OpenStack Summit Container Day, April 2016
clouds, containers, infrastructure, applications
and their management
(Stay tuned for updates to presentation)
One size does not fit all.
A strict apples-to-apples comparison is inappropriate and not the objective, hence characterizing and contrasting.
Host Health Monitoring
Orchestrator Updates and Host Maintenance
Networking and Load-Balancing
Application Health Monitoring
Announced as production-ready 5 months ago (Nov 2015)
image: digital trends
Swarm’s scheduler is pluggable
Swarm scheduling is a combination of strategies and filters/constraint:
container constraints (affinity, dependency, port) are defined as environment variables in the specification file
node constraints (health, constraint) must be specified when starting the docker daemon and define which nodes a container may be scheduled on.
Ability to remove batteries is a strength for Swarm:
image: Alan Chia
Docker Swarm is fully compatible with Docker’s networking
Docker multi-host networking (requires a K/V store) provides for user-defined overlay networks that are micro-segmentable
uses a gossip protocol for quick convergence of neighbor table
facilitates container name resolution via embedded DNS server (previously via etc/hosts)
You may bring your own network driver
Load-balancing is new in 1.11.0
- no documentation available
Rebalancing is not available
Rescheduled containers lose access to volumes mounted on the former host.
use volume plugins like Flocker to avoid this
Scaling swarm to 1,000 AWS nodes and 50,000 containers
Suitable for orchestrating a combination of infrastructure containers
Less built-in functionality for application containers
Swarm is a young project and lacks advanced features.
High Availability out of experimental as of latest release (1.11.0)
Load-balancing in 1.11.0? Swarm needs to be used with third-party software.
Only schedules Docker containers, not containers using other specifications.
While dependency and affinity filters are available, Swarm does not provide the ability to enforce scheduling of two containers onto the same host or not at all.
Swarm works. Swarm is simple and easy to deploy.
If you already know Docker CLI, using Swarm is straight-forward
facilitates earlier stages of adoption by organizations viewing containers as faster VMs
Less built-in functionality for applications
Swarm is easy to extend, if can already know Docker APIs, you can customize Swarm
Pluggable K/V store for both node and multi-host networking
considered a third generation container orchestrator led by Google, Red Hat and others.
bakes in load-balancing, scale, volumes, deployments and secret management among other features.
Declaratively, opinionated with many key features included
Two primary modes of finding a Service
Predicates (node resources and characteristics):
PodFitPorts , PodFitsResources, NoDiskConflict , MatchNodeSelector, HostName , ServiceAffinit, LabelsPresence
Priorities (weighted strategies used to identify “best fit” node):
LeastRequestedPriority, BalancedResourceAllocation, ServiceSpreadingPriority, EqualPriority
Deployment objects automate deploying and rolling updating applications.
Upgrading the Kubernetes components and hosts is done via shell script
cAdvisor, Heapster, InfluxDB, Grafana
three types of user-defined application health-checks and uses the Kubelet agent as the the health check monitor
HTTP Health Checks, Container Exec, TCP Socket
collect logs which persist beyond the lifetime of the pod’s container images or the lifetime of the pod or even cluster
standard output and standard error output of each container can be ingested using a Fluentd agent running on each node
…enter the Pod
For those familiar with Docker-only, Kubernetes requires understanding of new concepts
Powerful frameworks with more moving pieces beget complicated cluster deployment and management.
Lightweight graphical user interface
Mesos is a distributed systems kernel
stitches together many different machines into a logical computer
Mesos has been around the longest (launched in 2009)
and is arguably the most stable, with highest (proven) scale currently
Frameworks have a scheduler and executor. Schedulers get resource offers. Executors run tasks.
Marathon is written in Scala
Mesos-DNS generates an SRV record for each Mesos task
including Marathon application instances
Marathon will ensure that all dynamically assigned service ports are unique
Mesos-DNS is particularly useful when:
apps are launched through multiple frameworks (not just Marathon)
you are using an IP-per-container solution like Project Calico
you use random host port assignments in Marathon
may run multiple frameworks
has three types of Modules
Replacement, Hook, Anonymous
- Mesos has maintenance mode
An IP per Container
No longer share the node's IP
Helps remove port conflicts
Enables 3rd party network drivers
Marathon offers two TCP/HTTP proxies.
A simple shell script and a more complex one called marathon-lb that has more features.
Pluggable (e.g. Traefik for load-balancing)
Great at asynchronous jobs. High availability built-in.
Referred to as the “golden standard” by Solomon Hykes, Docker CTO.
A high-level perspective of the container orchestrator spectrum.