Docker, data &
extensions
Luke Marsden
@lmarsden
CTO, ClusterHQ
clusterhq.com
github.com/clusterhq/flocker
github.com/clusterhq/powerstrip
Docker is great!
Linux containers are a better
abstraction for apps
Some things are difficult
to do with Docker today
Multi-host apps require
orchestration
But none of these tools
handle state
Modern applications
are complex distributed systems
Containers and PaaS today
capture the easy, data-free bits
We should aim to capture the entire app
enabling larger class of apps to move to containers
This is the big problem we’re
solving at ClusterHQ:
Managing stateful containers
in production
Volumes in Docker: the problem
Cattle, not pets
We care about ops:
robustness
eliminate SPOFs
scalability
reduce complexity
You need to be able to move
things around
Moving stateful containers
Take initial snapshot, replicate
Shut down container
Replicate last snapshot
Start container on other side
What is Flocker?
Application manifest
Deployment manifest
Network proxy (iptables)
Storage layer
CLI (flocker-deploy) + server-side
Flocker
demo!
Next up:
API, multi-tenancy for
devops teams &
pluggable backends
But there’s a
problem…
Want to solve data
problems, not solve
scheduling, orchestration
& networking…
Docker extensions
goal:
Choice for ops
and
Portable developer
experience
Docker is great:
fundamentally better
abstraction, but…
Real apps run on
> 1 computer
Composition
Scheduling
Networking
Storage
Pieces coming together
Orchestration +
scheduling
Do
ck
Docker
Docker
Docker
er
AP
I
Docker
Orchestration +
scheduling sions
g
n
i
ork
w
t
e
N
Sto
n
e
t
ex
Do
ck
rag
ee
er
AP
I
???
xte
n
sio
n
Docker
s
Docker
Docker
Docker
Wrapping sucks :(
Orchestration +
scheduling
Do
ck
N
S
Docker
N
S
Docker
N
S
Docker
er
AP
I
N
S
Docker
Plugin mechanism
Late-bound
Composable
Optional
Healthy ecosystem
Lots of innovation
Powerstrip is a
prototyping tool for
Docker extensions
Powerstrip enables
composition of prototypical
Docker extensions
Flocker
Portable volumes for Docker
Move database containers around
Weave Virtual overlay network: IP address can
follow a container to another machine
Both tools wrap Docker :(
Both tools can work
together if they are both
Docker extensions*…
* Docker extensions don’t exist yet, so we’ll use
Powerstrip
powerstrip-flocker
observes “create” request, creates or
moves volume
powerstrip-weave
observes “create” and “start” request,
hijacks entrypoint
Powerstrip demo
flocker + weave
composed with
powerstrip
Extensions taxonomy
Extension type
Messages
API
Network
Volume
???
pre-hook
create
create
?
post-hook
destroy
destroy
?
events
attach
move
detach
snapshot
Extensions taxonomy
Extension type
Messages
API
Network
Volume
???
pre-hook
create
create
?
post-hook
destroy
destroy
?
events
attach
move
detach
snapshot
Powerstrip only gets us as far as the API extensions, but
it’s better to have “vertical” extension types…
Extensions taxonomy
Extension type
Messages
API
Network
Volume
???
pre-hook
create
create
?
post-hook
destroy
destroy
?
events
attach
move
detach
snapshot
Hypothesis: API extensions will be useful to prototype the
next extension types
What’s next?
Let’s build real Docker extensions!
Define loading/handshake mechanism
Agree on protocol & events
Get into stable release of Docker - 1.7?
WIP at cpuguy83/docker#3
Thank you - questions?
Check out Flocker & Powerstrip!
clusterhq.com/blog
@lmarsden @clusterhq #clusterhq #docker-extensions
on Freenode
Get rewarded to play with Flocker!
clusterhq.com/research
We are hiring in Bristol + SF Bay Area!
clusterhq.com/careers
t-s
t.ly hi
/c rts!
hq
te
es
bi