Lee Calcote

Layer5, Founder​

Open Source Summit:
Service Meshes

calcotestudios.com/talks

Lee Calcote

linkedin.com/in/leecalcote

@lcalcote

lee@layer5.io

clouds, containers, functions, applications,  and their management

calcotestudios.com/talks

gingergeek.com

github.com/leecalcote

@lcalcote

layer5.io/books

Subscribe for Early Release

at

https://layer5.io/subscribe

Benefits

First few services are relatively easy

 

 

Democratization of language and technology choice

 

Faster delivery, service teams running independently, rolling updates

@lcalcote

Challenges

Next 10 or so may introduce pain

 

 

Language and framework-specific libraries

 

 

Distributed environments, ephemeral infrastructure, out-moded tooling

Microservices

@lcalcote

layer5.io/landscape

Which is why...

 I have a container orchestrator.

@lcalcote

Core

Capabilities

  • Cluster Management

    • Host Discovery

    • Host Health Monitoring

  • Scheduling

  • Orchestrator Updates and Host Maintenance

  • Service Discovery

  • Networking and Load Balancing

  • Stateful Services

  • Multi-Tenant, Multi-Region

Additional

Key Capabilities

  • Application Health and Performance Monitoring

  • Application Deployments

  • Application Secrets

@lcalcote

minimal capabilities required to qualify as a container orchestrator

Service meshes generally rely on these underlying layers.

Which is why...

 I have an API gateway.

@lcalcote

Microservices API Gateways

north-south vs. east-west

What do we need?

• Observability

• Logging
• Metrics
• Tracing

• Traffic Control

• Resiliency

• Efficiency
• Security

• Policy

@lcalcote

a Service Mesh

What is a Service Mesh?

a dedicated layer for managing service-to-service communication

@lcalcote

So, a microservices platform?

obviously.

Orchestrators don't bring all that you need

and neither do service meshes,

but they do get you closer.

Missing: application lifecycle management, but not by much

partially.

a services-first network

Missing: distributed debugging; provide nascent visibility (topology)

Why use a Service Mesh?

to avoid...

  • Bloated service code

  • Duplicating work to make services production-ready

    • Load balancing, auto scaling, rate limiting, traffic routing...

  • Inconsistency across services

    • Retry, tls, failover, deadlines, cancellation, etc., for each language, framework

    • Siloed implementations lead to fragmented, non-uniform policy application and difficult debugging

  • Diffusing responsibility of service management

@lcalcote

Observability

what gets people hooked on service metrics

@lcalcote

Goals

  • Metrics without instrumenting apps

  • Consistent metrics across fleet

  • Trace flow of requests across services

  • Portable across metric back-end providers

You get a metric!  You get a metric!  Everyone gets a metric!

© 2018 SolarWinds Worldwide, LLC. All rights reserved.

Traffic Control

control over chaos

@lcalcote

  • Traffic splitting
    • L7 tag based routing?
  • Traffic steering
    • Look at the contents of a request and route it to a specific set of instances.
  • Ingress and egress routing

Resilency

 

  • Systematic fault injection

  • Timeouts and Retries with timeout budget

  • Control connection pool size and request load

  • Circuit breakers and Health checks

 

content-based traffic steering

layer5.io

Timeouts & Retries

@lcalcote

Web

Service Foo

Timeout = 600ms
Retries = 3

Timeout = 300ms
Retries = 3

Timeout = 900ms
Retries = 3

Service Bar

Database

Timeout = 500ms
Retries = 3

Timeout = 300ms
Retries = 3

Timeout = 900ms
Retries = 3

Deadlines

Web

Service Foo

Deadline = 600ms

Deadline = 496ms

Service Bar

Database

Deadline = 428ms

Deadline=180ms

Elapsed=104ms

Elapsed=68ms

layer5.io

Elapsed=248ms

@lcalcote

DEV

OPS

Layer 5

where Dev and Ops meet

Problem: too much infrastructure code in services

Service Mesh Architecture

Service Mesh Architecture

@lcalcote

Data Plane

  • Touches every packet/request in the system.
  • Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.

No control plane? Not a service mesh.

Ingress Gateway

Egress Gateway

Service Mesh Architecture

@lcalcote

Control Plane

Data Plane

  • Touches every packet/request in the system.
  • Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
  • Provides policy, configuration, and platform integration.
  • Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.
  • Does not touch any packets/requests in the data path.

No control plane? Not a service mesh.

Ingress Gateway

Egress Gateway

Service Mesh Architecture

@lcalcote

Control Plane

Data Plane

  • Touches every packet/request in the system.
  • Responsible for service discovery, health checking, routing, load balancing, authentication, authorization, and observability.
  • Provides policy, configuration, and platform integration.
  • Takes a set of isolated stateless sidecar proxies and turns them into a service mesh.
  • Does not touch any packets/requests in the data path.

No control plane? Not a service mesh.

Ingress Gateway

Egress Gateway

Management
Plane

  • Provides monitoring, backend system integration, expanded policy and application configuration.

BookInfo Sample App

Reviews v1

Reviews Pod

Reviews v2

Reviews v3

Product Pod

Details Container

Details Pod

Ratings Container

Ratings Pod

Product Container

Reviews Service

Ratings Service

Details Service

Product Service

BookInfo Sample App on Service Mesh

Reviews v1

Reviews Pod

@lcalcote

Reviews v2

Reviews v3

Product Pod

Details Container

Details Pod

Ratings Container

Ratings Pod

Product Container

Envoy sidecar

Envoy sidecar

Envoy sidecar

Envoy sidecar

Envoy sidecar

Reviews Service

Enovy sidecar

Envoy ingress

Product Service

Ratings Service

Details Service

Pilot

Citadel

Mixer

Control Plane

Data Plane

istio-system namespace

policy check

Foo Pod

Proxy Sidecar

Service Foo

tls certs

discovery & config

Foo Container

Bar Pod

Proxy Sidecar

Service Bar

Bar Container

Out-of-band telemetry propagation

telemetry

 

reports

Control flow during request processing

application traffic

Application traffic

application namespace

telemetry reports

Istio Architecture

@lcalcote

Galley

Ingress Gateway

Egress Gateway

Control Plane

Data Plane

linkerd-system namespace

Foo Pod

Proxy Sidecar

Service Foo

Foo Container

Bar Pod

Proxy Sidecar

Service Bar

Bar Container

Out-of-band telemetry propagation

telemetry

 

scarping

Control flow during request processing

application traffic

Application traffic

application namespace

telemetry scraping

@lcalcote

Architecture

destination

Prometheus

Grafana

tap

dashboard

CLI

proxy-api

public-api

Linkerd

Deployment Models

Freely Available

at

gingergeek.com

Adopting a service mesh

Adopter’s Dilemma

Which service mesh to use?

What's the catch? Nothing's for free.

Playground

WHICH SERVICE MESH SHOULD I USE AND HOW DO I GET STARTED?

 

Learn about the functionality of different service meshes and visually manipulate mesh configuration.

Performance Benchmark

WHAT OVERHEAD DOES BEING ON THE SERVICE MESH INCUR?

 

Benchmark the performance of your application across different service meshes and compare their overhead.

layer5.io/meshery

@lcalcote

Meshery

a multi-service mesh performance benchmark and playground

Side-by-Side Performance Comparison

Istio

Linkerd

Octarine

NSM

App Mesh

@lcalcote

results coming forthcoming at KubeCon EU...

Consul

Up next...

(service meshes contributing adapters)

Kubernetes
(no mesh)

Demo

Relative

Showdown!
Slowdown

@lcalcote

layer5.io/meshery

Cores Threads Istio (2) Linkerd
8 8 1 1
8 16 1.7 1.8
8 32 3.2 3.4
8 100 9.3 9.6

(2) mTLS on, tracing off

Relative

Showdown! Slowdown

@lcalcote

layer5.io/meshery

Cores Threads Istio (1) Istio (2) Linkerd
8 8 1 1 1
8 16 1.4 1.7 1.8
8 32 18.4 3.2 3.4
8 100 52.2 9.3 9.6

(1) mTLS on, tracing on

(2) mTLS on, tracing off

Mixer

@lcalcote

Mixer

Control Plane

Data Plane

istio-system namespace

Foo Pod

Proxy sidecar

Service Foo

Foo Container

Out-of-band telemetry propagation

Control flow during request processing

application traffic

application traffic

application namespace

telemetry reports

an attribute processing engine

Different Response Sizes

@lcalcote

layer5.io/meshery

@lcalcote

layer5.io/meshery

Different Response Sizes

@lcalcote

layer5.io/meshery

Different Response Sizes

Service Mesh Benchmark Specification

A project and vendor-neutral specification for capturing details of:

  1. Environment / Infrastructure

    • Number and size of nodes, orchestrator

  2. Service mesh and its configuration

  3. Service / application details

Bundled with test results.

 

github.com/layer5io/service-mesh-benchmark-spec

@lcalcote

layer5.io/meshery

@lcalcote

layer5.io/meshery

LEE CALCOTE

Layer5, Founder​

Open Source Summit:
Service Meshes

layer5.io/books