layer5.io
cloud native and its management
Service Mesh Patterns
calcotestudios.com/talks
Kush Trivedi
Layer5, Maintainer
@kush_1814
Join the community
slack.layer5.io
Container
Orchestrator
Mesh
5.5 years
(Jun 2014)
4.5 years
(Jul 2015)
3 years
(Apr 2017)
5.5 years ago
(Jun 2014)
7 years ago
(Mar 2013)
4 years ago
(Feb 2016)
layer5.io/landscape
It's meshy out there.
Different tools for different use cases
a sample
Meshery is interoperable with these abstractions.
Service Mesh Interface
(SMI)
Multi-Vendor Service Mesh Interoperation (Hamlet)
Service Mesh Performance Specification (SMPS)
A standard interface for service meshes on Kubernetes.
A set of API standards for enabling service mesh federation.
A format for describing and capturing service mesh performance.
to the rescue
where Dev and Ops meet
Empowered and independent teams can iterate faster
where Dev and Ops meet
Empowered and independent teams can iterate faster
Service Mesh Performance (SMP)
Configuration
Security
Telemetry
Control Plane
Data
Plane
service mesh ns
Foo Pod
Proxy Sidecar
Service Foo
App Container
Bar Pod
Proxy Sidecar
Service Bar
App Container
Out-of-band telemetry propagation
Control flow
application
traffic
Application traffic
application namespace
Ingress Gateway
Egress Gateway
Management
Plane
meshery
adapters
gRPC
kube-api
kube-system
generated load
http / gRPC traffic
fortio
wrk2
nighthawk
UI
API
workloads
Meshery WASM Filter
CLI
perf analysis
patterns
@mesheryio
operator
Four API specifications
Service Mesh Interface aims to provide:
• Standard interface for service mesh on Kubernetes
• Basic feature set for most common mesh use cases
• Extensible to support new features
• Space for the ecosystem to innovate
Traffic Split
Traffic Specs
Traffic Metrics
Traffic Access Control
APIs:
Operate and upgrade with confirmation of SMI compatibility
✔︎ Learn Layer5 sample application used for validating test assertions.
✔︎ Defines compliant behavior.
✔︎ Produces compatibility matrix.
✔︎ Ensures provenance of results.
✔︎ Runs a set of conformance tests.
✔︎ Built into participating service mesh’s release pipeline.
SMI Conformance application - github.com/layer5io/learn-layer5
benchmarking of service mesh performance
exchange of performance information from system-to-system / mesh-to-mesh
apples-to-apples performance comparisons of service mesh deployments.
MeshMark - a universal performance index to gauge a service mesh’s efficiency against deployments in other organizations’ environments
https://smp-spec.io
Directly provides:
Indirectly facilitates:
Latency, throughput, and the proxies’ CPU and memory consumption affected by these factors
Data Plane
Proxy sidecar
App Container
Pod
with many variables
Data plane performance depends on many factors, for example:
Data Plane
Pod
Proxy sidecar
App Container
Comparing approaches to data plane filtering
Data Plane
App Container
Pod
Client Library
Proxy sidecar
Power
Speed
Data Plane
Proxy sidecar
App Container
Pod
Understanding the trade-off between power and speed
Data Plane
Proxy sidecar
App Container
Pod
Speed
Data Plane
Proxy sidecar
App Container
Pod
Data Plane
Proxy sidecar
App Container
Pod
Power
Speed
https://smp-spec.io
Use Meshery's powerful performance management features
- easily reproduce tests
- persist test results
- use different load generators
- baseline and compare over time
- test your workloads on and off the mesh
- tweak configurations and try again
- manage 9 different service meshes and counting...
https://meshery.io
Hamlet specifies a set of API standards for enabling service mesh federation.
Specifications:
service catalog federation
https://github.com/vmware/hamlet
​​API to authenticate and securely distribute resources between federated service meshes.
API to discover, reach, authenticate, and securely communicate with federated services.
Service Mesh Patterns
SIG Network: Service Mesh Working Group
Join the community
slack.layer5.io
Kush Trivedi
@kush_1814
control over chaos
content-based traffic steering
what gets people hooked on service metrics
identity and policy
Expect more from your infrastructure
Bloated service (application) code
Duplicating work to make services production-ready
Load balancing, auto scaling, rate limiting, traffic routing...
Inconsistency across services
Retry, tls, failover, deadlines, cancellation, etc., for each language, framework
Siloed implementations lead to fragmented, non-uniform policy application and difficult debugging
Diffusing responsibility of service management
to avoid...
Control Plane
Data Plane
Touches every packet/request in the system.
Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
Provides policy, configuration, and platform integration.
Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.
Does not touch any packets/requests in the data path.
You need a management plane.
Ingress Gateway
Management
Plane
Provides multi-mesh federation, backend system integration, expanded policy and governance, continuous delivery integration, workflow, chaos engineering, configuration and performance management.
Egress Gateway
github.com/layer5io/image-hub
Functionality | In the app | In the filter |
---|---|---|
User / Token | ||
Subscription Plans | ||
Plan Enforcement |
a sample app
Two
application containers
Hub UI Pod
Image Storage Container
Image Storage Pod
Hub UI
Container
Image Storage Service
Hub UI Service
github.com/layer5io/image-hub
Hub UI Pod
Image Access Container
Image Access Pod
Hub UI
Container
Image Access Service
Hub UI Service
Envoy sidecar
Envoy sidecar
github.com/layer5io/image-hub
with Consul
Leader
Follower
Consul Servers
Follower
agent
Consul Client
node
Leader
Agent
Control Plane
Data Plane
intentions
Foo Pod
Proxy Sidecar
Service Foo
discovery, config,
tls certs
Foo Container
Bar Pod
Proxy Sidecar
Service Bar
Bar Container
Control flow
application traffic
Application traffic
application namespace
Follower
Consul Client
Consul Servers
Follower
policy
check
layer5.io/service-mesh-architectures
WASM Filter
node
Image Access Container
Image Access Pod
Image Access Service
Envoy sidecar
github.com/layer5io/image-hub
WASM Filter
with a Rust-based WASM filter
apiVersion: apps/v1
kind: Deployment
spec:
template:
metadata:
labels:
app: api-v1
annotations:
"consul.hashicorp.com/connect-inject": "true"
"consul.hashicorp.com/service-meta-version": "1"
"consul.hashicorp.com/service-tags": "v1"
"consul.hashicorp.com/connect-service-protocol": "http"
"consul.hashicorp.com/connect-wasm-filter-add_header": "/filters/optimized.wasm"
spec:
containers:
- name: api
image: layer5/image-hub-api:latest
Leader
Follower
Consul Servers
Follower
agent
node
for the web, malware and beyond
webassembly.org
identifying your optimal configuration for most requests
In the presence of Bucket 1...
...take your largest segment by count and divide by your number of cores
Bucket 2
Bucket 1
Bucket 3
Bucket 4
Understand value vs overhead
Data Plane
Touches every packet/request in the system.
Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
Ingress Gateway
Egress Gateway
No control plane? Not a service mesh.
Control Plane
Provides policy, configuration, and platform integration.
Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.
Does not touch any packets/requests in the data path.
Data Plane
Touches every packet/request in the system.
Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
Ingress Gateway
Egress Gateway
Data Plane
Pod
Proxy sidecar
App Container
+
Comparing approaches to data plane filtering
Data Plane
Proxy sidecar
App Container
Data Plane
App Container
Pod
Pod
Client Library
Proxy sidecar
Proxy sidecar
Power
Speed