layer5.io

cloud native and its management

calcotestudios.com/talks

leecalcote

Service Mesh Patterns

calcotestudios.com/talks

Join the community

slack.layer5.io

Third step in Cloud Native journey

Container

Orchestrator

Mesh

5.5 years
(Jun 2014)

4.5 years
(Jul 2015)

3 years
(Apr 2017)

5.5 years ago
(Jun 2014)

7 years ago
(Mar 2013)

4 years ago
(Feb 2016)

v1.0

Announced

layer5.io/landscape

It's meshy out there.

Strengths of Service Mesh Implementations

Different tools for different use cases

a sample

Service mesh abstractions

Meshery is interoperable with these abstractions.

Service Mesh Interface
(SMI)

Multi-Vendor Service Mesh Interoperation (Hamlet)

Service Mesh Performance Specification (SMPS)

A standard interface for service meshes on Kubernetes.

A set of API standards for enabling service mesh federation.

A format for describing and capturing service mesh performance.

to the rescue

Service Mesh Deployment Models

Client

Edge Cache

Istio Gateway

(envoy)

Cache Generator

Collection of VMs running APIs

service mesh

Istio VirtualService

Istio ServiceEntry

Situation:

existing services running on VMs (that have little to no service-to-service traffic).
nearly all traffic flows from client to the service and back to client.

Benefits:

gain granular traffic control (e.g path rewrites).
detailed service monitoring without immediately deploying a thousand sidecars.

Ingress

Istio - The Weather Company's Journey

Out-of-band telemetry propagation

Application traffic

Control flow

Proxy per Node

Service A

maesh

Node (server)

Service A

Service B

maesh

Node (server)

Service A

Service C

maesh

Node (server)

Advantages:

Less (memory) overhead.
Simpler distribution of configuration information.
primarily physical or virtual server based; good for large monolithic applications.

Disadvantages:

Coarse support for encryption of service-to-service communication, instead host-to-host encryption and authentication policies.
Blast radius of a proxy failure includes all applications on the node, which is essentially equivalent to losing the node itself.
Not a transparent entity, services must be aware of its existence.

Advantages:

Good starting point for building a brand-new microservices architecture or for migrating from a monolith.

Disadvantages:

When the number of services increase, it becomes difficult to manage.

Router "Mesh"

Fabric Model

Advantages:

Granular encryption of service-to-service communication.
Can be gradually added to an existing cluster without central coordination.

Disadvantages:

Lack of central coordination. Difficult to scale operationally.

Ingress or Edge Proxy

Advantages:

Works with existing services that can be broken down over time.

Disadvantages:

Is missing the benefits of service-to-service visibility and control.

Service Mesh Architectures

Data Plane

Touches every packet/request in the system.
Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.

Ingress Gateway

Egress Gateway

Service Mesh Architecture

No control plane? Not a service mesh.

Control Plane

Provides policy, configuration, and platform integration.
Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.
Does not touch any packets/requests in the data path.

Data Plane

Touches every packet/request in the system.
Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.

Ingress Gateway

Egress Gateway

Service Mesh Architecture

Control Plane

Data Plane

Touches every packet/request in the system.
Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.

Provides policy, configuration, and platform integration.
Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.
Does not touch any packets/requests in the data path.

You need a management plane.

Ingress Gateway

Management
Plane

Provides multi-mesh federation, backend system integration, expanded policy and governance, continuous delivery integration, workflow, chaos engineering, configuration and performance management.

Egress Gateway

Service Mesh Architecture

Service Mesh Interface (SMI)

Service Mesh Performance (SMP)

SIG Network:
Service Mesh WG

Configuration

Security

Telemetry

Control Plane

Data
Plane

service mesh ns

Foo Pod

Proxy Sidecar

Service Foo

App Container

Bar Pod

Proxy Sidecar

Service Bar

App Container

Out-of-band telemetry propagation

Control flow

application

traffic

Application traffic

application namespace

Meshery Architecture

Ingress Gateway

Egress Gateway

Management
Plane

meshery

adapters

gRPC

kube-api

kube-system

generated load

http / gRPC traffic

fortio

wrk2

nighthawk

API

workloads

Meshery WASM Filter

CLI

perf analysis

patterns

@mesheryio

operator

Configuration Best Practices

Meshery analyzes your service mesh and workload configuration

operate with confidence

Assess your service mesh configuration against deployment and operational best practices with Meshery's configuration validator.

Service Mesh Interface

Four API specifications

Service Mesh Interface aims to provide:

• Standard interface for service mesh on Kubernetes

• Basic feature set for most common mesh use cases

• Extensible to support new features

• Space for the ecosystem to innovate

Traffic Split

Traffic Specs

Traffic Metrics

Traffic Access Control

APIs:

Service Mesh Interface (SMI) Conformance

Operate and upgrade with confirmation of SMI compatibility

✔︎ Learn Layer5 sample application used for validating test assertions.

✔︎ Defines compliant behavior.

✔︎ Produces compatibility matrix.

✔︎ Ensures provenance of results.

✔︎ Runs a set of conformance tests.

✔︎ Built into participating service mesh’s release pipeline.

Demo

SMI Conformance application - github.com/layer5io/learn-layer5

SMI Conformance

benchmarking of service mesh performance
exchange of performance information from system-to-system / mesh-to-mesh
apples-to-apples performance comparisons of service mesh deployments.
MeshMark - a universal performance index to gauge a service mesh’s efficiency against deployments in other organizations’ environments

Service Mesh Performance

https://smp-spec.io

Directly provides:

Indirectly facilitates:

a vendor neutral specification for capturing details of infrastructure capacity, service mesh configuration, and workload metadata.

Performance Management

Understand value vs overhead

Demo

https://smp-spec.io

Service Mesh Performance

What is WebAssembly?

for the web, malware and beyond

A small, fast binary format that promises near-native performance for web applications.
Most modern browsers support it.
Safe and sandboxed execution environment.
Over 40 languages that support WASM as a compilation target.
Originally used to speed up large web-applications.

webassembly.org

Comparing types of Data Plane filtering

Data Plane

Pod

Proxy sidecar

App Container

Comparing approaches to data plane filtering

Data Plane

Proxy sidecar

App Container

Data Plane

App Container

Pod

Client Library

Proxy sidecar

Power

Speed

An optimization game

Latency, throughput, and the proxies’ CPU and memory consumption affected by these factors

Data Plane

Proxy sidecar

App Container

Pod

Number of client connections
Target request rate
Request size and Response size
Number of proxy worker threads
Protocol
CPU cores
Number and types of proxy filters

with many variables

Data plane performance depends on many factors, for example:

Service Mesh Patterns

List of service mesh patterns

Service Mesh Patterns

SIG Network: Service Mesh Working Group

by the book

Join the community

slack.layer5.io

Lee Calcote

@lcalcote

DEV

OPS

Decoupling at Layer 5

where Dev and Ops meet

Empowered and independent teams can iterate faster

DEV

OPS

Decoupling at Layer 5

where Dev and Ops meet

Empowered and independent teams can iterate faster

Context-based routing

Understanding the trade-off between power and speed

Data Plane

Proxy sidecar

App Container

Pod

Speed

Data Plane

Proxy sidecar

App Container

Pod

Round robin load balancing

Data Plane

Proxy sidecar

App Container

Pod

Path-based routing

Comparing types of functions

Power

Speed

Performance Testing Best Practices

Use Meshery's powerful performance management features

- easily reproduce tests

- persist test results

- use different load generators

- baseline and compare over time

- test your workloads on and off the mesh

- tweak configurations and try again

- manage 9 different service meshes and counting...

https://meshery.io

Hamlet

Hamlet specifies a set of API standards for enabling service mesh federation.

Specifications:

service catalog federation

https://github.com/vmware/hamlet

API to authenticate and securely distribute resources between federated service meshes.

API to discover, reach, authenticate, and securely communicate with federated services.

Federated Resource Discovery API
Federated Service Discovery API

Traffic Control

control over chaos

Resilency

content-based traffic steering

Observability

what gets people hooked on service metrics

Security

identity and policy

Service Mesh Functionality

Expect more from your infrastructure

What is a Service Mesh?

a services-first network

layer5.io/books/the-enterprise-path-to-service-mesh-architectures

@lcalcote

Image Hub

github.com/layer5io/image-hub

Functionality	In the app	In the filter
User / Token
Subscription Plans
Plan Enforcement

a sample app

Two

application containers

Hub UI Pod

Image Storage Container

Image Storage Pod

Hub UI
Container

Image Storage Service

Hub UI Service

github.com/layer5io/image-hub

Image Hub on Docker Desktop

Hub UI Pod

Image Access Container

Image Access Pod

Hub UI
Container

Image Access Service

Hub UI Service

Image Hub on a Service Mesh

Envoy sidecar

github.com/layer5io/image-hub

with Consul

Leader

Follower

Consul Servers

Follower

agent

Consul Client

node

Leader

Agent

Control Plane

Data Plane

intentions

Foo Pod

Proxy Sidecar

Service Foo

discovery, config,

tls certs

Foo Container

Bar Pod

Proxy Sidecar

Service Bar

Bar Container

Control flow

application traffic

Application traffic

application namespace

Follower

Consul Client

Consul Servers

Follower

policy

check

Consul Architecture

layer5.io/service-mesh-architectures

WASM Filter

node

Image Access Container

Image Access Pod

Image Access Service

Image Hub on Consul

Envoy sidecar

github.com/layer5io/image-hub

WASM Filter

with a Rust-based WASM filter

apiVersion: apps/v1
kind: Deployment
spec:
  template:
    metadata:
      labels:
        app: api-v1
      annotations:
        "consul.hashicorp.com/connect-inject": "true"
        "consul.hashicorp.com/service-meta-version": "1"
        "consul.hashicorp.com/service-tags": "v1"
        "consul.hashicorp.com/connect-service-protocol": "http"
        "consul.hashicorp.com/connect-wasm-filter-add_header": "/filters/optimized.wasm"
    spec:
      containers:
      - name: api
        image: layer5/image-hub-api:latest

Leader

Follower

Consul Servers

Follower

agent

node

PERFORMANCE

Optimizing your average response time

identifying your optimal configuration for most requests

In the presence of Bucket 1...

...take your largest segment by count and divide by your number of cores

Bucket 2

Bucket 1

Bucket 3

Bucket 4

Why use a Service Mesh?

Bloated service (application) code
Duplicating work to make services production-ready
- Load balancing, auto scaling, rate limiting, traffic routing...
Inconsistency across services
- Retry, tls, failover, deadlines, cancellation, etc., for each language, framework
- Siloed implementations lead to fragmented, non-uniform policy application and difficult debugging
Diffusing responsibility of service management

to avoid...