Decompose that WAR!
A pattern language for
microservices
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson
[email protected]
http://plainoldobjects.com
http://microservices.io
@crichardson
Presentation goal
Why patterns and pattern
languages?
A pattern language for
microservices
@crichardson
About Chris
@crichardson
About Chris
Founder of a startup that’s creating
a platform for developing
event-driven microservices
@crichardson
For more information
https://github.com/cer/event-sourcing-examples
http://microservices.io
http://plainoldobjects.com/
https://twitter.com/crichardson
@crichardson
Agenda
Why a pattern language for microservices?
Core patterns
Deployment patterns
Communication patterns
@crichardson
In 1986…
http://en.wikipedia.org/wiki/Fred_Brooks
@crichardson
Yet almost 30 years later
developers are still
passionately arguing over
“silver bullets”
@crichardson
Suck/Rock Dichotomy
JavaScript vs. Java
Spring vs. Java EE
Functional programming vs. Object-oriented
Containers vs. Virtual Machines
http://nealford.com/memeagora/2009/08/05/suck-rock-dichotomy.html
@crichardson
Gartner Hype Cycle
It’s
awesome
It’s not
awesome
Trade-offs
understood
http://upload.wikimedia.org/wikipedia/commons/b/bf/Hype-Cycle-General.png
@crichardson
How we make decisions
Rationalize with
our intellect
Decide
using
emotions
http://en.wikipedia.org/wiki/Mahout
@crichardson
We need a better way
to discuss and think
about technology
@crichardson
What’s a pattern?
Reusable solution
to a problem
occurring
in a particular context
@crichardson
The structure of a pattern
=
Great framework for discussing
and thinking about technology
@crichardson
The structure of a pattern
Name
Context
aka the situation
Problem
(conflicting) issues
etc to address
Forces
Solution
Resulting context
Related patterns
@crichardson
Benefits
Resulting context
Drawbacks
Issues to resolve
@crichardson
Alternative solutions
Related patterns
Solutions to problems
introduced by this pattern
@crichardson
Pattern language
A collection of related
patterns that solve problems
in a particular domain
Relationships
Pattern A results in a
context that has a problem
solved by Pattern B
Access to Water
Promenade
Local townhall
Patterns A and B solve the
same problem
Intimacy gradient
Pattern A is a
specialization of pattern B
Light on two sides
http://en.wikipedia.org/wiki/A_Pattern_Language
Meta-pattern
Problem: How to talk/reason about technology?
Context: Emotional software development culture
Solution: Use the pattern format
Benefit: More objective
Drawback: Less exciting
Related patterns: It’s awesome!
@crichardson
Work
in
progress
Motivating
Pattern
Solution
Pattern
Solution A
Solution B
General
Specific
Service-perContainer
Deployment
Service-per-VM
Multiple Services
per host
Single Service per
Host
Core
Monolithic
architecture
Microservice
architecture
Partitioning
API gateway
Communication
Server-side
discovery
Client-side discovery
Style
Messaging
Remote Procedure
Invocation
Service registry
Self registration
Discovery
3rd party registration
@crichardson
Agenda
Why a pattern language for microservices?
Core patterns
Deployment patterns
Communication patterns
@crichardson
@crichardson
Let’s imagine you are
building an online store
Browser/
Client
REST/JSON
HTML
StoreFrontUI
Product Info
Service
Recommendation
Service
SQL
Database
Review Service
Order Service
@crichardson
Problem: what’s the
deployment architecture?
@crichardson
Forces
There is a team of developers that must be productive
The application must be easy to understand and modify
Do continuous deployment
Run multiple instances for scalability and availability
Use emerging technologies (frameworks, programming
languages, etc)
@crichardson
Pattern: Monolithic
architecture
WAR/EAR
StoreFrontUI
HTML
Browser/
Client
REST/JSON
Product Info
Service
Recommendation
Service
Simple to
develop
test
deploy
scale
MySQL
Database
Review Service
Order Service
Tomcat
@crichardson
Examples everywhere
@crichardson
But when the application is
large …
@crichardson
Intimidates developers
@crichardson
Obstacle to frequent
deployments
Need to redeploy everything to change one component
Interrupts long running background (e.g. Quartz) jobs
Increases risk of failure
Eggs in
one basket
Fear of change
Updates will happen less often - really long QA cycles
e.g. Makes A/B testing UI really difficult
@crichardson
Overloads your IDE and
container
Slows down development
@crichardson
Obstacle to scaling
development
I want to update
the UI
But
the backend is not working
yet!
Lots of coordination and
communication required
@crichardson
Requires long-term commitment
to a technology stack
@crichardson
Pattern: Microservice
architecture
@crichardson
Apply functional
decomposition
X axis
- horizontal duplication
Sc
Z
ax
is
Scale by
splitting
different things
ale - d
by ata
sp pa
th litt rtit
in in ion
gs g
sim ing
ila
r
Y axis functional
decomposition
@crichardson
Microservice architecture
Browse Products
UI
Product
ProductInfo
Info
Service
Checkout UI
Recommendation
Service
Order management
UI
Review
Service
Account
management UI
Order
Service
Apply X-axis and Z-axis scaling
to each service independently @crichardson
Examples
http://techblog.netflix.com/
~600 services
http://highscalability.com/amazon-architecture
100-150 services to build a page
http://www.addsimplicity.com/downloads/
eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128
@crichardson
Benefits
Smaller, simpler apps
Easier to understand and develop
Less jar/classpath hell - who needs OSGI?
Faster to build and deploy
Scales development: develop, deploy and scale each service independently
Improves fault isolation
Eliminates long-term commitment to a single technology stack
System level architecture vs. service level architecture
Easily and safely experiment with new technologies
@crichardson
Drawbacks
Complexity of developing a distributed system
Implementing inter-process communication
Handling partial failures
Implementing business transactions that span multiple databases
(without 2PC)
Complexity of testing a distributed system
Complexity of deploying and operating a distributed system
Managing the development and deployment of features that span
multiple services
Fortunately solutions exists
@crichardson
The benefits typically
outweigh the drawbacks
for
large, complex applications
@crichardson
Issues to address
How to deploy the services?
How do the services communicate?
How do clients of the application communicate with the
services?
How to partition the system into services?
….
@crichardson
Agenda
Why a pattern language for microservices?
Core patterns
Deployment patterns
Communication patterns
@crichardson
We have applied the
microservices pattern:
How to deploy the services?
@crichardson
Forces
Services are written using a variety of languages, frameworks, and
framework versions
Each service consists of multiple service instances for throughput and
availability
Building and deploying a service must be fast
Service must be deployed and scaled independently
Service instances need to be isolated
Resources consumed by a service must be constrained
Deployment must be cost-effective
@crichardson
@crichardson
Pattern: Multiple service
instances per host
Host (Physical or VM)
Service-A
Instance-1
Service-B
Instance-2
Service-C
Instance-2
Process
WAR
OSGI bundle
@crichardson
Benefits and drawbacks
Benefits
Drawbacks
Efficient resource utilization
Poor/Terrible isolation
Fast deployment
Poor visibility (with WAR/OSGI
deployment)
Difficult to limit resource utilization
Risk of dependency version
conflicts
Poor encapsulation of
implementation technology
Pattern: Service instance per
host
@crichardson
Pattern: Service per VM host
VM
Service
deployed as
Service
packaged as
VM
image
VM
Service
VM
Service
@crichardson
Example
http://techblog.netflix.com/
~600 services
packer.io is a great tool
@crichardson
Benefits and drawbacks
Benefits
Drawbacks
Great isolation
Less efficient resource
utilization
Great manageability
VM encapsulates
implementation technology
Leverage AWS
infrastructure for
Autoscaling/Load balancing Slow deployment
Pattern: Service per Container
VM
host
Container
Service
deployed as
Service
packaged as
Container
image
Container
Service
VM
Container
Service
@crichardson
Examples
@crichardson
Benefits and drawbacks
Benefits
Drawbacks
Great isolation
Immature infrastructure for
deploying containers
Great manageability
Container encapsulates
implementation technology
Efficient resource utilization
Fast deployment
Agenda
Why a pattern language for microservices?
Core patterns
Deployment patterns
Communication patterns
@crichardson
Communication issues
How do clients of the
system interact with the
services?
The System
Service A
System Client
How do services
within the system
interact?
Service B
Service C
@crichardson
@crichardson
How do clients of the system
interact with the services?
@crichardson
Forces
Mismatch between the fine-grained microservices and needs
of the clients
Different clients need different data
LAN vs. WAN vs. Mobile network performance
The number of service instances and their locations (host
+port) is changes dynamically
Partitioning into services can change over time and should be
hidden from clients
@crichardson
Directly connecting the front-end to the backend
Chatty API
View
What’s the host/port??
Controller
Model
Traditional server-side
web application
View
Controller
Model
Browser/Native App
Web unfriendly
protocols
REST
Product Info
service
REST
Recommendation
Service
Thrift
Review
service
@crichardson
Pattern: API gateway
Single entry point
View
Controller
REST
Product Info
service
REST
Recommendation
Service
Thrift
Review
service
Model
Traditional server-side
web application
View
API
Gateway
Controller
Model
Browser/Native App
Client
specific APIs
Protocol
translation
@crichardson
Example: Netflix API
Device specific
end points
http://techblog.netflix.com/2013/01/optimizing-netflix-api.html
@crichardson
Benefits of the API gateway
Insulates the clients from the partitioning
Insulates the clients from the problem of discovery
Provides the optimal API for each client
Reduces the number of requests/roundtrips
Simplifies the client by moving logic for calling multiple
services from the client to API gateway
Gateway translates between web-unfriendly protocols and
HTTP/WebSockets
@crichardson
Drawbacks of the API
gateway
Increased complexity - the API gateway is yet another highly
available component that must be developed, deployed and
managed
Increased response time due to the additional network hop
through the API gateway
@crichardson
@crichardson
Benefits and drawbacks of
messaging
Benefits
Drawbacks
Decouples client from
services
Additional complexity of
message broker
Message broker buffers
messages
Request/reply-style
communication is more
complex
Supports a variety of
communication patterns Client needs to discover
location of message
broker
Benefits and drawbacks of
RPC
Benefits
Drawbacks
Simple and familiar
Only supports request/
reply
Request/reply is easy
No intermediate broker
Service must be
available
Client needs to discover
locations of service
instances
@crichardson
The problem of discovery
Dynamically
assigned
Client or API
gateway
Service
Client
10.4.3.1:8756
?
10.4.3.99:4545
10.4.3.20:333
How to load
balance?
Dynamically
changing
Service
Instance A
Service
Instance B
Service
Instance C
@crichardson
Pattern: Client-side discovery
Service
Client
Registryaware
HTTP
Client
load
balance
request
10.4.3.1:8756
10.4.3.99:4545
10.4.3.20:333
query
register
Service
Instance A
Service
Instance B
Service
Instance C
Service
Registry
@crichardson
Example: Netflix Eureka and Ribbon
https://github.com/Netflix/eureka/wiki/Eureka-at-a-glance
https://github.com/Netflix/ribbon
@crichardson
Benefits and drawbacks
Benefits
Drawbacks
Flexible, application-specific
load balancing
Couples the client to the
Service Registry
Fewer network hops and
moving parts compared to
Server-side discovery Need implement client-side
discovery and load balancing
logic in multiple languages/
frameworks
Service Registry is yet
another moving part to setup
and operate - highly available
Pattern: Server-side discovery
10.4.3.1:8756
Service
Client
request
Router
load 10.4.3.99:4545
balance
request
10.4.3.20:333
query
register
Service
Instance A
Service
Instance B
Service
Instance C
Service
Registry
@crichardson
Example
Internal
ELB
Public
ELB
Internal
ELB
http://docs.aws.amazon.com/ElasticLoadBalancing/latest/
DeveloperGuide/vpc-loadbalancer-types.html
@crichardson
Benefits and drawbacks
Benefits
Drawbacks
Simpler client code
Limited to (generic) load
balancing algorithms
provided by router
Built-in to some cloud/
container environments, e.g.
AWS ELB, Kubernetes,
Marathon
More network hops
Router and Service Registry
is yet another moving part
to setup and operate highly available
The problem of registration
10.4.3.1:8756
10.4.3.99:4545
10.4.3.20:333
Service
Instance A
Service
Instance B
Service
Instance C
register
Service
Registry
How does registration
happen?
@crichardson
@crichardson
Forces
Service instances must be
registered with the service registry on startup
unregistered on shutdown
Service instances that crash must be unregistered from the
service registry
Service instances that are running but incapable of handling
requests must be unregistered from the service registry
@crichardson
Pattern: Self registration
10.4.3.1:8756
Service
Instance A
register
heart-beat
unregister
Service
Registry
@crichardson
Examples
Netflix Eureka Java client
API for registering/unregistering with Eureka server
Specify state: STARTING, UP, DOWN, OUT_OF_SERVICE
Register a periodically invoked health check callback
Zookeeper-based Java applications
Service = znode
Service instance = ephemeral child of znode
@crichardson
Benefits and drawbacks
Benefits
Drawbacks
Simple - no extra
components
Couples the service to
service registry
Service can implement a
more sophisticated state
model
Must implement registration
logic in multiple languages/
frameworks
Service might lack the selfawareness to unregister
itself
Pattern: 3rd party registration
10.4.3.1:8756
Service
Instance A
healthcheck
Registrar
register
heart-beat
unregister
Service
Registry
@crichardson
Examples
AWS Autoscaling groups
Automatically register/unregister EC2 instance with ELB
Registrator
Registers and unregisters Docker containers with service registry
Kubernetes/Marathon
Automatically register/unregister services
Netflix Eureka Prana
Sidecar application for non-Java clients
Registers/unregisters service with Eureka
Performs health checks
@crichardson
Benefits and drawbacks
Benefits
Drawbacks
Simpler service
Registrar is yet another
component that must be
setup/operated. Must be
highly available
Registrar can perform health
checks
Some cloud/container
environments provide a
Registrar
Summary: Patterns and pattern
languages are a great way to …
Think about technology
Discuss technology
Apply technology
@crichardson
Summary: The Microservices pattern
language is a great way to …
Think about microservices
Discuss microservices
Apply microservices (or not)
@crichardson
@crichardson [email protected]
http://plainoldobjects.com
http://microservices.io
@crichardson