It's a Multi-Mesh World

Lee Calcote

Founder, Layer5

@lcalcote

Lee Calcote

@lcalcote

cloud native and its management

calcotestudios.com/talks

Service Mesh Patterns

slack.layer5.io

slack.layer5.io

What is a Service Mesh?

a services-first network

layer5.io/books/the-enterprise-path-to-service-mesh-architectures

Traffic Control

control over chaos

Resilency

content-based traffic steering

Observability

what gets people hooked on service metrics

Security

identity and policy

Service Mesh Functionality

Expect more from your infrastructure

Business Logic

in-network application logic

DEVELOPER

OPERATOR

Decoupling at Layer 5

where Dev, Ops, Product meet

Empowered and independent teams can iterate faster

PRODUCT OWNER

DEVELOPER

OPERATOR

Decoupling at Layer 5

where Dev, Ops, Product meet

Empowered and independent teams can iterate faster

PRODUCT OWNER

layer5.io/landscape

It's meshy out there.

Strengths of Service Mesh Implementations

Different tools for different use cases

a sample

Service Mesh Architectures

Data Plane

  • Touches every packet/request in the system.

  • Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.

Ingress Gateway

Egress Gateway

Service Mesh Architecture

No control plane? Not a service mesh.

Control Plane

  • Provides policy, configuration, and platform integration.

  • Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.

  • Does not touch any packets/requests in the data path.

Data Plane

  • Touches every packet/request in the system.

  • Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.

Ingress Gateway

Egress Gateway

Service Mesh Architecture

Control Plane

Data Plane

  • Touches every packet/request in the system.

  • Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.

  • Provides policy, configuration, and platform integration.

  • Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.

  • Does not touch any packets/requests in the data path.

You need a management plane.

Ingress Gateway

Management
Plane

  • Provides backend system integration, expanded policy and governance, continuous delivery integration, workflow, chaos engineering,  configuration and performance management and multi-mesh federation.

Egress Gateway

Service Mesh Architecture

Pilot

Citadel

Mixer

Control Plane

Data Plane

istio-system namespace

policy check

Foo Pod

Proxy Sidecar

Service Foo

tls certs

discovery & config

Foo Container

Bar Pod

Proxy Sidecar

Service Bar

Bar Container

Out-of-band telemetry propagation

telemetry

 

reports

Control flow

application traffic

Application traffic

application namespace

telemetry reports

Istio Architecture

Galley

Ingress Gateway

Egress Gateway

@layer5

Leader

Agent

Control Plane

Data Plane

intentions

Foo Pod

Proxy Sidecar

Service Foo

discovery, config,

 

tls certs

Foo Container

Bar Pod

Proxy Sidecar

Service Bar

Bar Container

Control flow

application traffic

Application traffic

application namespace

Follower

Consul Client

Consul Servers

Follower

policy

 

check

Consul Architecture

layer5.io/service-mesh-architectures

WASM Filter

node

Control Plane

Data Plane

linkerd-system namespace

Foo Pod

Proxy Sidecar

Service Foo

Foo Container

Bar Pod

Proxy Sidecar

Service Bar

Bar Container

Out-of-band telemetry propagation

telemetry

 

scarping

Control flow during request processing

application traffic

Application traffic

application namespace

telemetry scraping

Architecture

destination

Prometheus

Grafana

tap

web

CLI

proxy-api

public-api

Linkerd

proxy-injector

@layer5

Control Plane

Data Plane

octa-system namespace

policy check

Foo Pod

Proxy
Sidecar

Service Foo

discovery & config

Foo Container

Bar Pod

Service Bar

Bar Container

Out-of-band telemetry propagation

telemetry

 

reports

Control flow

application traffic

Application traffic

application namespace

telemetry reports

Octarine Architecture

Policy
Engine

Security Engine

Visibility
Engine

+

Proxy
Sidecar

+

@layer5

Service Mesh Deployment Models

Client

Edge Cache

Istio Gateway

(envoy)

Cache Generator

Collection of VMs running APIs

service mesh

Istio VirtualService

Istio VirtualService

Istio ServiceEntry

Situation:

  • existing services running on VMs (that have little to no service-to-service traffic).

  • nearly all traffic flows from client to the service and back to client.

 

Benefits:

  • gain granular traffic control (e.g path rewrites).

  • detailed service monitoring without immediately deploying a thousand sidecars.

Ingress

Istio - The Weather Company's Journey

Out-of-band telemetry propagation

Application traffic

@lcalcote

Control flow

Proxy per Node

Service A

Service A

Service A

maesh

Node (server)

Service A

Service A

Service B

maesh

Node (server)

Service A

Service A

Service C

maesh

Node (server)

Advantages:

  • Less (memory) overhead.

  • Simpler distribution of configuration information.

  • primarily physical or virtual server based; good for large monolithic applications.
     

Disadvantages:

  • Coarse support for encryption of service-to-service communication, instead host-to-host encryption and authentication policies.

  • Blast radius of a proxy failure includes all applications on the node, which is essentially equivalent to losing the node itself.

  • Not a transparent entity, services must be aware of its existence.

layer5.io/books

Advantages:

  • Good starting point for building a brand-new microservices architecture or for migrating from a monolith.

Disadvantages:

  • When the number of services increase, it becomes difficult to manage.

Router "Mesh"

Fabric Model

Advantages:

  • Granular encryption of service-to-service communication.

  • Can be gradually added to an existing cluster without central coordination.

Disadvantages:

  • Lack of central coordination. Difficult to scale operationally.

Ingress or Edge Proxy

Advantages:

  • Works with existing services that can be broken down over time.

Disadvantages:

  • Is missing the benefits of service-to-service visibility and control.

Service mesh abstractions 

Meshery is interoperable with these abstractions.

Service Mesh Interface
(SMI)

Multi-Vendor Service Mesh Interoperation (Hamlet)

Service Mesh Performance Specification (SMPS)

A standard interface for service meshes on Kubernetes.

A set of API standards for enabling service mesh federation.

A format for describing and capturing service mesh performance.

to the rescue

  • benchmarking of service mesh performance
     

  • exchange of performance information from system-to-system / mesh-to-mesh
     

  • apples-to-apples performance comparisons of service mesh deployments.
     

  • MeshMark - a universal performance index to gauge a service mesh’s efficiency against deployments in other organizations’ environments

 

Service Mesh Performance

https://smp-spec.io

Directly provides:

Indirectly facilitates:

  • a vendor neutral specification for capturing details of infrastructure capacity, service mesh configuration, and workload metadata.

An optimization game

Latency, throughput, and the proxies’ CPU and memory consumption affected by these factors

Data Plane

Proxy sidecar

App Container

Pod

 

  • Number of client connections
  • Target request rate
  • Request size and Response size
  • Number of proxy worker threads
  • Protocol
  • CPU cores
  • Number and types of proxy filters

 

with many variables

Data plane performance depends on many factors, for example:

Context-based routing

Understanding the trade-off between power and speed

Data Plane

Proxy sidecar

App Container

Pod

Speed

Data Plane

Proxy sidecar

App Container

Pod

Round robin load balancing

Data Plane

Proxy sidecar

App Container

Pod

Path-based routing

Comparing types of functions

Power

Speed

Comparing types of Data Plane filtering

Data Plane

Pod

Proxy sidecar

App Container

Comparing approaches to data plane filtering

Data Plane

App Container

Pod

Client Library  

Proxy sidecar

Rate limiting with Go client library

  • 100 RPS
    • p50: 3.19ms
  • 500 RPS
    • p50: 2.44ms
  • Unlimited RPS - 4417
    • p50: 0.66ms

Rate limiting with WASM module (Rust filter)

  • 100 RPS
    • p50: 2.1ms
  • 500 RPS
    • p50: 2.22ms
  • Unlimited RPS - 5781
    • p50: 0.62ms

Power

Speed

Data Plane

Proxy sidecar

App Container

Pod

What is WebAssembly?

for the web, malware and beyond

  • A small, fast binary format that promises near-native performance for web applications.
  • Most modern browsers support it.
  • Safe and sandboxed execution environment.
  • Over 40 languages that support WASM as a compilation target.
  • Originally used to speed up large web-applications.

webassembly.org

Image Access Container

Image Access Pod

Image Access Service

Image Hub on Consul

Envoy sidecar

github.com/layer5io/image-hub

WASM Filter

with a Rust-based WASM filter

apiVersion: apps/v1
kind: Deployment
spec:
  template:
    metadata:
      labels:
        app: api-v1
      annotations:
        "consul.hashicorp.com/connect-inject": "true"
        "consul.hashicorp.com/service-meta-version": "1"
        "consul.hashicorp.com/service-tags": "v1"
        "consul.hashicorp.com/connect-service-protocol": "http"
        "consul.hashicorp.com/connect-wasm-filter-add_header": "/filters/optimized.wasm"
    spec:
      containers:
      - name: api
        image: layer5/image-hub-api:latest

Leader

Follower

Consul Servers

Follower

agent

node

Service Mesh Interface (SMI)

Service Mesh Performance (SMP)

Performance Management

Understand value vs overhead

Configuration Best Practices

Meshery analyzes your service mesh and workload configuration

operate with confidence

Assess your service mesh configuration against deployment and operational best practices with Meshery's configuration validator.

Service Mesh Interface (SMI) Conformance

Operate and upgrade with confirmation of SMI compatibility

✔︎ Learn Layer5 sample application used for validating test assertions.
 

 âœ”︎ Defines compliant behavior.

 âœ”︎ Produces compatibility matrix.

 âœ”︎ Ensures provenance of results.

 âœ”︎ Runs a set of conformance tests.

 âœ”︎ Built into participating service mesh’s release pipeline.

Configuration

Security

Telemetry

Control Plane

Data
Plane

service mesh ns

Foo Pod

Proxy Sidecar

Service Foo

Foo Container

Bar Pod

Proxy Sidecar

Service Bar

Bar Container

Out-of-band telemetry propagation

Control flow

application

traffic

Application traffic

application namespace

Meshery Architecture

Ingress Gateway

Egress Gateway

Management
Plane

meshery

adapter

gRPC

kube-api

kube-system

generated load

http / gRPC traffic

fortio

wrk2

nighthawk

UI

server

workloads

Meshery WASM Filter

CLI

perf analysis

Lee Calcote

@lcalcote

Join the discussion

slack.layer5.io

Optimizing your average response time

identifying your optimal configuration for most requests

In the presence of Bucket 1...

...take your largest segment by count and divide by your number of cores

Bucket 2

Bucket 1

Bucket 3

Bucket 4

Performance Testing Best Practices

meshery.io

Use Meshery's powerful performance management features

- easily reproduce tests

- persist test results

- use different load generators

- baseline and compare over time

- test your workloads on and off the mesh

- tweak configurations and try again

- compare 6 different service meshes and counting...

Image Hub

github.com/layer5io/image-hub

Functionality In the app In the filter
User / Token
Subscription Plans
Plan Enforcement

a sample app

Two 

application containers

Hub UI Pod

Image Storage Container

Image Storage Pod

Hub UI
Container

Image Storage Service

Hub UI Service

github.com/layer5io/image-hub

Image Hub on Docker Desktop

Hub UI Pod

Image Access Container

Image Access Pod

Hub UI
Container

Image Access Service

Hub UI Service

Image Hub on a Service Mesh

Envoy sidecar

Envoy sidecar

github.com/layer5io/image-hub

with Consul

Leader

Follower

Consul Servers

Follower

agent

Consul Client

node

What is a Service Mesh?

a services-first network

layer5.io/books/the-enterprise-path-to-service-mesh-architectures

Third step in Cloud Native journey

Container

Orchestrator

Mesh

5.5 years
(Jun 2014)

4.5 years
(Jul 2015)

3 years
(Apr 2017)

5.5 years ago
(Jun 2014)

7 years ago
(Mar 2013)

4 years ago
(Feb 2016)

v1.0

Announced