Discreetly Studying the Effects of Individual Traffic Control Functions

Lee Calcote

Founder, Layer5

Prateek Sahu

Contributor, Layer5

@lcalcote

@PrateekSahu22

Lee Calcote

linkedin.com/in/leecalcote

@lcalcote

layer5.io

cloud native and its management

calcotestudios.com/talks

github.com/leecalcote

Service Mesh Patterns

calcotestudios.com/talks

slack.layer5.io

Prateek Sahu

@PrateekSahu22

slack.layer5.io

@PrateekSahu22

prateeksahu.github.io

Traffic Control

control over chaos

Resilency

content-based traffic steering

Observability

what gets people hooked on service metrics

Security

identity and policy

Service Mesh Functionality

Expect more from your infrastructure

Help with Modernization

 

  • Can modernize your IT inventory without:

    • Rewriting your applications

    • Adopting microservices, regular services are fine

    • Adopting new frameworks

    • Moving to the cloud

address the long-tail of IT services

Get there for free

Why use a Service Mesh?

  • Bloated service (application) code

  • Duplicating work to make services production-ready

    • Load balancing, auto scaling, rate limiting, traffic routing...

  • Inconsistency across services

    • Retry, tls, failover, deadlines, cancellation, etc., for each language, framework

    • Siloed implementations lead to fragmented, non-uniform policy application and difficult debugging

  • Diffusing responsibility of service management

to avoid...

DEV

OPS

Decoupling at Layer 5

where Dev and Ops meet

Empowered and independent teams can iterate faster

DEV

OPS

Decoupling at Layer 5

where Dev and Ops meet

Empowered and independent teams can iterate faster

Service Mesh Architectures

Control Plane

Data Plane

  • Touches every packet/request in the system.

  • Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.

  • Provides policy, configuration, and platform integration.

  • Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.

  • Does not touch any packets/requests in the data path.

You need a management plane.

Ingress Gateway

Management
Plane

  • Provides multi-mesh federation, backend system integration, expanded policy and governance, continuous delivery integration, workflow, chaos engineering,  configuration and performance management.

Egress Gateway

Service Mesh Architecture

layer5.io/landscape

It's meshy out there.

Service mesh abstractions 

Meshery is interoperable with these abstractions.

Service Mesh Interface
(SMI)

Multi-Vendor Service Mesh Interoperation (Hamlet)

Service Mesh Performance Specification (SMPS)

A standard interface for service meshes on Kubernetes.

A set of API standards for enabling service mesh federation.

A format for describing and capturing service mesh performance.

to the rescue

 

  • A vendor neutral specification for capturing details of infrastructure capacity, service mesh configuration, and workload metadata.

     

  • a universal performance index to gauge a service mesh’s efficiency against deployments in other organizations’ environments.

  • apples-to-apples performance comparisons of service mesh deployments.

 

Service Mesh Performance Specification

vendor neutral service mesh performance measurement standard

github.com/layer5io/service-mesh-performance-specification

Directly provides:

Facilitates:

SIG Network: Service Mesh WG

Service Mesh Interface (SMI)

Service Mesh Performance Spec (SMPS)

Performance Management

Understand value vs overhead

Configuration Best Practices

Meshery analyzes your service mesh and workload configuration

operate with confidence

Assess your service mesh configuration against deployment and operational best practices with Meshery's configuration validator.

Configuration

Security

Telemetry

Control Plane

Data
Plane

service mesh ns

Foo Pod

Proxy Sidecar

Service Foo

Foo Container

Bar Pod

Proxy Sidecar

Service Bar

Bar Container

Out-of-band telemetry propagation

Control flow

application

traffic

Application traffic

application namespace

Meshery Architecture

Ingress Gateway

Egress Gateway

Management
Plane

meshery

adapter

gRPC

kube-api

kube-system

generated load

http / gRPC traffic

load generator

fortio

wrk2

nighthawk

UI

server

workloads

Meshery WASM Filter

CLI

Image Hub

github.com/layer5io/image-hub

Functionality In the app In the filter
User / Token
Subscription Plans
Plan Enforcement

a sample app

Two 

application containers

Hub UI Pod

Image Storage Container

Image Storage Pod

Hub UI
Container

Image Storage Service

Hub UI Service

github.com/layer5io/image-hub

Image Hub on Docker Desktop

Hub UI Pod

Image Access Container

Image Access Pod

Hub UI
Container

Image Access Service

Hub UI Service

Image Hub on a Service Mesh

Envoy sidecar

Envoy sidecar

github.com/layer5io/image-hub

with Consul

Leader

Follower

Consul Servers

Follower

agent

Consul Client

node

Leader

Agent

Control Plane

Data Plane

intentions

Foo Pod

Proxy Sidecar

Service Foo

discovery, config,

 

tls certs

Foo Container

Bar Pod

Proxy Sidecar

Service Bar

Bar Container

Control flow

application traffic

Application traffic

application namespace

Follower

Consul Client

Consul Servers

Follower

policy

 

check

Consul Architecture

layer5.io/service-mesh-architectures

WASM Filter

node

Image Access Container

Image Access Pod

Image Access Service

Image Hub on Consul

Envoy sidecar

github.com/layer5io/image-hub

WASM Filter

with a Rust-based WASM filter

apiVersion: apps/v1
kind: Deployment
spec:
  template:
    metadata:
      labels:
        app: api-v1
      annotations:
        "consul.hashicorp.com/connect-inject": "true"
        "consul.hashicorp.com/service-meta-version": "1"
        "consul.hashicorp.com/service-tags": "v1"
        "consul.hashicorp.com/connect-service-protocol": "http"
        "consul.hashicorp.com/connect-wasm-filter-add_header": "/filters/optimized.wasm"
    spec:
      containers:
      - name: api
        image: layer5/image-hub-api:latest

Leader

Follower

Consul Servers

Follower

agent

node

PERFORMANCE

An optimization game

Latency, throughput, and the proxies’ CPU and memory consumption affected by these factors

Data Plane

Proxy sidecar

App Container

Pod

 

  • Number of client connections
  • Target request rate
  • Request size and Response size
  • Number of proxy worker threads
  • Protocol
  • CPU cores
  • Number and types of proxy filters

 

with many variables

Data plane performance depends on many factors, for example:

What is WebAssembly?

for the web, malware and beyond

  • A small, fast binary format that promises near-native performance for web applications.
  • Most modern browsers support it.
  • Safe and sandboxed execution environment.
  • Over 40 languages that support WASM as a compilation target.
  • Originally used to speed up large web-applications.

webassembly.org

Comparing types of Data Plane filtering

Data Plane

Pod

Proxy sidecar

App Container

Comparing approaches to data plane filtering

Data Plane

App Container

Pod

Client Library  

Proxy sidecar

Rate limiting with Go client library

  • 100 RPS
    • p50: 3.19ms
  • 500 RPS
    • p50: 2.44ms
  • Unlimited RPS - 4417
    • p50: 0.66ms

Rate limiting with WASM module (Rust filter)

  • 100 RPS
    • p50: 2.1ms
  • 500 RPS
    • p50: 2.22ms
  • Unlimited RPS - 5781
    • p50: 0.62ms

Power

Speed

Data Plane

Proxy sidecar

App Container

Pod

Context-based routing

Understanding the trade-off between power and speed

Data Plane

Proxy sidecar

App Container

Pod

Speed

Data Plane

Proxy sidecar

App Container

Pod

Round robin load balancing

Data Plane

Proxy sidecar

App Container

Pod

Path-based routing

Comparing types of functions

Power

Speed

Optimizing your average response time

identifying your optimal configuration for most requests

In the presence of Bucket 1...

...take your largest segment by count and divide by your number of cores

Bucket 2

Bucket 1

Bucket 3

Bucket 4

Performance Testing Best Practices

meshery.io

Use Meshery's powerful performance management features

- easily reproduce tests

- persist test results

- use different load generators

- baseline and compare over time

- test your workloads on and off the mesh

- tweak configurations and try again

- compare 6 different service meshes and counting...

Lee Calcote

@lcalcote

Join the discussion

slack.layer5.io

Prateek Sahu

@PrateekSahu22

Data Plane

  • Touches every packet/request in the system.

  • Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.

Ingress Gateway

Egress Gateway

Service Mesh Architecture

No control plane? Not a service mesh.

Control Plane

  • Provides policy, configuration, and platform integration.

  • Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.

  • Does not touch any packets/requests in the data path.

Data Plane

  • Touches every packet/request in the system.

  • Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.

Ingress Gateway

Egress Gateway

Service Mesh Architecture

Comparing types of Data Plane filtering

Data Plane

Pod

Proxy sidecar

App Container

+

Comparing approaches to data plane filtering

Data Plane

Proxy sidecar

App Container

Data Plane

App Container

Pod

Pod

Client Library

Proxy sidecar

Proxy sidecar

Power

Speed

What is a Service Mesh?

a services-first network

layer5.io/books/the-enterprise-path-to-service-mesh-architectures

Third step in Cloud Native journey

Container

Orchestrator

Mesh

5.5 years
(Jun 2014)

4.5 years
(Jul 2015)

3 years
(Apr 2017)

5.5 years ago
(Jun 2014)

7 years ago
(Mar 2013)

4 years ago
(Feb 2016)

v1.0

Announced