Lee Calcote
February 2018

as the service proxy


the extensible service mesh

clouds, containers, functions, applications  and their management


The more, the more merrier?


The first few services are relatively easy



Democratization of language and technology choice


Faster delivery, service teams running independently, rolling updates


The next 10 or so may introduce pain



Language and framework specific libraries



Distributed environments, ephemeral infrastructure, out-moded tooling

Which is why...

 I have a container orchestrator.



  • Cluster Management

    • Host Discovery

    • Host Health Monitoring

  • Scheduling

  • Orchestrator Updates and Host Maintenance

  • Service Discovery

  • Networking and Load-Balancing

  • Stateful services

  • Multi-tenant, multi-region


Key Capabilities

  • Application Health & Performance Monitoring

  • Application Deployments

  • Application Secrets

What do we need?

• Observability

• Logging
• Metrics
• Tracing

• Traffic Control

• Resiliency

• Efficiency
• Security


a Service Mesh

What is a Service Mesh?

a dedicated layer for managing service-to-service communication

so, a microservices platform?


Orchestrators don't bring all that you need

and neither do service meshes,

but they do get you closer.

Missing: application lifecycle management, but not by much


a services-first network

Missing: distributed debugging; provide nascent visibility (topology)



Layer 5

where Dev and Ops meet

Problem: too much infrastructure code in services

Why use a Service Mesh?

to avoid...

  • Bloated service code

  • Duplicating work to make services production-ready

    • load balancing, auto scaling, rate limiting, traffic routing, ...

  • Inconsistency across services

    • retry, tls, failover, deadlines, cancellation, etc, for each language, framework

    • silo'ed implementations lead to fragmented, non-uniform policy application and difficult debugging

  • Diffusing responsibility of service management

Help with Modernization


  • Can modernize your IT inventory without:

    • Rewriting your applications

    • Adopting microservices, regular services are fine

    • Adopting new frameworks

    • Moving to the cloud

Address the long-tail of IT services

Get there for free

What is Istio?

An open platform to connect, manage, and secure microservices

  • Observability

  • Resiliency

  • Traffic Control

  • Security

  • Policy Enforcement



is what gets people hooked on service metrics


  • Metrics without instrumenting apps

  • Consistent metrics across fleet

  • Trace flow of requests across services

  • Portable across metric backend providers

You get a metric!  You get a metric!  Everyone gets a metric!

Traffic Control

control over chaos

  • Traffic splitting
    • L7 tag based routing?
  • Traffic steering
    • Look at the contents of a request and route it to a specific set of instances.
  • Ingress and egress routing



  • Systematic fault injection
  • Timeouts and Retries with timeout budget

  • Circuit breakers and Health checks

  • Control connection pool size and request load


content-based traffic steering

Istio Architecture

Istio Architecture

Control Plane

Data Plane

Touches every packet/request in the system. Responsible for service discovery, health checking, routing, load balancing, authentication, authorization and observability.

Provides policy and configuration for services in the mesh.

Takes a set of isolated stateless sidecar proxies and turns them into a distributed system.

Does not touch any packets/requests in the system.

Istio Architecture




Control Plane

Data Plane

istio-system namespace

policy check

Foo Pod

Proxy sidecar

Service Foo

tls certs

discovery & config

Foo Container

Bar Pod

Proxy sidecar

Service Bar

Bar Container

Out-of-band telemetry propagation




Control flow during request processing

application traffic

application traffic

application namespace

telemetry reports

What's Pilot for?

provides service discovery to sidecars

manages sidecar configuration



Control Plane

the head of the ship


istio-system namespace

system of record for service mesh


provides abstraction from underlying platforms

What's Mixer for?

  • Point of integration with infrastructure backends
    • Intermediates between Istio and backends, under operator control
    • Enables platform & environment mobility
  • Responsible for policy evaluation and telemetry reporting
    • Provides granular control over operational policies and telemetry
  • Has a rich configuration model
    • Intent-based config abstracts most infrastructure concerns




Control Plane

istio-system namespace

an attribute-processing and routing machine


  1. Precondition checking
  2. Quota management
  3. Telemetry reporting

What's Auth for?

  • Verifiable identity
    • Issues certs
    • Certs distributed to service proxies
    • Mounted as a Kubernetes secret
  • Secure naming / addressing
  • Traffic encryption



Control Plane

security at scale


istio-system namespace

security by default

Orchestrate Key & Certificate:

  • Generation
  • Deployment
  • Rotation
  • Revocation


Service Proxy Sidecar

A C++ based L4/L7 proxy

Low memory footprint

In production at Lyft


  • API driven config updates → no reloads
  • Zone-aware load balancing w/ failover
  • Traffic routing and splitting
  • Health checks, circuit breakers, timeouts, retry budgets, fault injection…
  • HTTP/2 & gRPC
  • Transparent proxying
  • Designed for observability


the included battery

Data Plane


Proxy sidecar

App Container

Extensibility of Istio


  • Uses pluggable adapters to extend its functionality
    • Adapters run within the Mixer process
  • Adapters are modules that interface to infrastructure backends
    • (logging, metrics, quotas, etc.)
  • Multi-interface adapters are possible
    • (e.g. AppOptics adapter exposing logs & metrics)

Mixer Adapters

sending telemetry






Swapping Proxies - Envoy, Linkerd, Nginx, Conduit

Forms of Extensions

Why use another service proxy?

Based on your operational expertise and need for battle-tested proxy. You may be looking for caching, WAF or other functionality available in NGINX Plus.

If you're already running Linkerd and want to start adopting Istio control APIs like CheckRequest.

Conduit not currently designed a general-purpose proxy, but lightweight and focused with extensibility via gRPC plugin.


  • Support for rules, policies, mtls encryption, monitoring & tracing
  • Compatible with Mixer adaptors
  • Transparent sidecar injection
  • Compatible with Istio 0.3.0


  • Support for gRPC traffic
  • Support for ingress proxy
  • Support for Quota Checks
  • Expanding the mesh beyond Kubernetes

See sidecar-related limitations as well as supported traffic management rules --> here .

Considered beta quality

Soliciting feedback and participation from community

Istio & nginMesh






Control Plane

  • Translator agent Istio to Nginx (in go)
  • Loadable module Nginx to Mixer (in rust)


config file

Data Plane

Mixer Module

"istio-proxy" container

route rules

tcp server

istio-system namespace








Out-of-band telemetry propagation

Control flow during request processing

application traffic

application traffic

http servers


Let's look at Istio's canonical sample app.

BookInfo Sample App

Reviews v1

Reviews Pod

Reviews v2

Reviews v3

Product Pod

Details Container

Details Pod

Ratings Container

Ratings Pod

Product Container

Reviews Service

BookInfo Sample App

Reviews v1

Reviews Pod

Reviews v2

Reviews v3

Product Pod

Details Container

Details Pod

Ratings Container

Ratings Pod

Product Container

Nginx sidecar

Nginx sidecar

Nginx sidecar

Nginx sidecar

Nginx sidecar

Reviews Service

Nginx sidecar

Envoy ingress

                                $ kubectl apply -f istio-nginmesh-0.4.0-v2.yaml
$ kubectl apply -f nginmesh-0.4.0-migration/istio/release/install/kubernetes/istio-initializer.yaml

deploy Istio and nginMesh

                                kubectl get ns
watch kubectl get po,svc -n istio-system
kubectl apply -f nginmesh-0.4.0-migration/istio/release/samples/kubernetes/bookinfo.yaml

confirm deployment Istio; deploy sample app

                                watch kubectl get po,svc
kubectl get svc istio-ingress -n istio-system -o jsonpath='{.spec.ports[0].nodePort}';echo ''

confirm sample app


running nginMesh


running nginMesh

                                echo "http://$(kubectl get nodes -o template --template='{{range.items}}{{range.status.addresses}}{{if eq .type "InternalIP"}}{{.address}}{{end}}{{end}}{{end}}'):$(kubectl get svc istio-ingress -n istio-system -o jsonpath='{.spec.ports[0].nodePort}')/productpage"

See "reviews" v1, v2 and v3

                                # From Docker's perspective
docker ps | grep nginmesh

# From Kubernetes' perspective
kubectl get po  
kubectl describe 

Verify nginMesh deployment

                                # exec into 'istio-proxy'
kubectl exec -it pod -c istio-proxy /bin/bash

Connect to Nginx sidecar

                                #takes every request/response and sends to Mixer control plane 
See load_module /etc/nginx/modules/; 

# Mixer adapters for telemetry
docker run --rm istio/fortio load -c 1 -t 10m \
`echo "http://$(kubectl get nodes -o template --template='{{range.items}}{{range.status.addresses}}{{if eq .type "InternalIP"}}{{.address}}{{end}}{{end}}{{end}}'):$(kubectl get svc istio-ingress -n istio-system -o jsonpath='{.spec.ports[0].nodePort}')/productpage"`

Verify mesh configuration


running nginMesh

                                #Deploy new configuration to Nginx
istioctl create -f route-rule-all-v1.yaml 
istioctl delete -f route-rule-all-v1.yaml

#A/B testing for a user
kubectl apply -f route-rule-reviews-test-v2.yaml

#More for user 'lee'
kubectl exec -it pod -c istio-proxy /bin/bash
more /etc/istio/proxy/conf.d/http_0.0.0.0_9080.conf

Apply traffic routing policy

See Mixer telemetry

Lee Calcote

Thank you. Questions?

clouds, containers, functions,

applications and their management