Load-Balancing in the Cloud using

Lee Calcote
March 16th, 2017

http://calcotestudios.com/talks

Nginx

Kubernetes

&

Lee Calcote

linkedin.com/in/leecalcote

@lcalcote

blog.gingergeek.com

lee@calcotestudios.com

clouds, containers, infrastructure, applications and their management

Public

Private

Application Delivery Controllers

Load-Balancers

have evolved.

@lcalcote

Our Bloat-a-lith

A 1GB jar!

Our Case Study

is an IDE for game developers
created by game development studio based in Austin
creates a real-time, collaborative game development engine

“Google Docs for game development”

@lcalcote

Our Bloat-a-lith

EventBus Handler

Persistence Verticle

Vert.x

SockJS

Analytics

Analytics UI

Hazelcast

GridConnection

Telemetry

Telemetry Analysis

Projects, Streams,

Users, Sessions, Tenants

Kafka Producer

Platform Client Producer

Spark, R

GCE Network

Load Balancer

Kafka

Rendering Engines

Platform Publishing Tool

Game Analysis

Single instance

@lcalcote

beloved

Shaping Up

Faster delivery, rolling updates
Horizontal scale out on-demand;
- on an individual service basis
Modular architecture
Easy integration and deployments
Service isolation, resilience and fail-safe recovery

Benefits of Microservices

Democratization of language and technology choice

@lcalcote

a quick review

it's an excellent time to be a developer

The promise of...

Characteristics of Microservices

how small is small?

Who has a system that is too big and that you’d like to break down?

Can a small team manage it?

More moving parts increases complexity.

Can you make a change to a service and deploy it by itself without changing anything else?

@lcalcote

small, autonomous services that work together.

independent, autonomous service
self-contained functional unit

App is Reactive
Leverages sockets
Limited resources
Culture (DevOps / Cloud maturity)
Support containers AND VMs
Propagation of huge data sets

The Challenge

On-the-fly intelligent rendering
Distribution of content
Extremely low latency for Reactive services
Blocking vs non-blocking
On-premises telemetry collection and analytics

@lcalcote

Our Microbloat v2

Analytics UI

Authentication

Telemetry

Telemetry Analysis

Spark, R

Kafka

Rendering Engines

Platform Publishing Tool

Game Analysis

Master

etcd

Locking

Projects

Authorization

kube-proxy

SockJS

Authorization

kube-proxy

DaemonSet

Node

kube-api, etc.

@lcalcote

Node

Pod

Deployment

Authorization

DaemonSet

Service

Comparing Services

(not shoes)

SSL Termination

Kubernetes 1.5	No
Swarm 1.13	No
Mesos+Marathon	Yes

ELB Classic	Yes
ELB L7	Yes
Beanstalk	Yes (EC2)
IOT	Yes
ECS	Yes (EC2)

Load-Balancer	No
App Gateway	Yes
Container Service	?

Cloud LB (HTTP)	Yes
Cloud LB (Network)	Yes
GKE	No

Container Orchestrators

Clouds

AWS

Azure

GCP

Link

SSL Proxy

@lcalcote

Link

Websocket Support

Kubernetes 1.5	No
Swarm 1.13	No
Mesos+Marathon	Yes

Container Orchestrators

Clouds

AWS

Azure

GCP

Link

@lcalcote

ELB Classic	Yes
ELB L7	Yes
Beanstalk	Yes
IOT	Yes
ECS	Yes

Load-Balancer	No
App Gateway	Yes
Container Service	?

Cloud LB (HTTP)	No
Cloud LB (Network)	Yes
GKE	No

Link

Kubernetes & Nginx

to the Rescue

@lcalcote

There are soooo many ways to skin this cat.

Microbloat v3

Going deeper with Nginx & Kubernetes

As an ingress controller in Kubernetes
- SSL termination
- Path-based rules
- Web socket support

@lcalcote

Service Discovery with Nginx Plus
- Need for locating service instances instantly without reconfiguring
- On-the-fly Reconfiguration API
- Work with etcd

Kubernetes

Pod - group of co-scheduled containers and volumes
Replication Controller - reconciliation loop to keep current state congruent with desired state
Service - a set of pods that comprise a common function
Deployment - manages updates for Pods and Replica Sets
Secrets - store and retrieve sensitive data

\ˈnō-mən-ˌklā-chər

a brief Kubernetes construct review

@lcalcote

Exposing Kubernetes Services

ClusterIP - service to be reachable only from inside of the cluster.
ExternalName - It serves as a way to return an alias to an external service residing outside the cluster.
NodePort - exposes service on a port on each node of the cluster.
LoadBalancer - cluster-internal IP and exposing service on a NodePort, also ask the cloud provider for a load balancer which forwards requests to the Service exposed as a <NodeIP>:NodePort for each Node.

@lcalcote

kube-proxy runs on each node in the cluster
a network proxy that represents Services on each node
integral to how services are exposed in the cluster
limited to layer 4 (tcp/udp) load-balancing

kube-proxy

no·men·cla·ture

@lcalcote

kube-proxy

iptables

Container AA

Container A

kube-proxy

Node A

Node B

Client

Pod A

Service A

iptables

Inbound

Outbound

NodePort

@lcalcote

Traffic flow with

NodePort/LoadBalancer

Container BB

Container B

Pod B

Service B

An Ingress is a collection of rules that allow inbound connections to reach the cluster services. - how you expose and route to the service .

Ingress

no·men·cla·ture

@lcalcote

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: projects
spec:
  tls:
  - hosts:
    - api.maxplay.io
    secretName: api-secret
  rules:
  - host: api.maxplay.io
    http:
      paths:
      - path: /projects
        backend:
          serviceName: tenant-svc
          servicePort: 80
      - path: /tenants
        backend:
          serviceName: user-svc
          servicePort: 80
      - path: /users
        backend:
          serviceName: user-svc
          servicePort: 80

Secrets

$ kubectl create secret generic api-secret --from-file nginx.conf

@lcalcote

An Ingress Controller is a control loop that manages rules enabling inbound traffic to applications.

Ingress Controller

specification

@lcalcote

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-ingress-dp
  labels:
    app: nginx-ingress
spec:
  replicas: 1
  selector:
    app: nginx-ingress
  template:
    metadata:
      labels:
        app: nginx-ingress
    spec:
      containers:
      - image: maxplay/nginx-ingress:latest
        imagePullPolicy: Always
        name: nginx-ingress
        ports:
        - containerPort: 80
          hostPort: 80
        - containerPort: 443
          hostPort: 443
    args:
    - /nginx-ingress-controller
    - --default-backend-service= \
    $(POD_NAMESPACE)/nginx-default-backend

Make sure you review controller specific docs so you understand the caveats of each one.

iptables

Container AA

Container A

Ingress
Controller

kube-proxy

Node A

Node B

Client

Pod A

Ingress B

Service A

iptables

Inbound

Outbound

@lcalcote

Traffic flow with

Ingress Controller

Annotation in the Ingress resource definition specifies which services are web socket services -

"socks-svc"

Web Socket Support

Ingress Resource Annotation

@lcalcote

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: api-ingress
  annotations:
    nginx.org/websocket-services: "sockjs-svc"
spec:
  tls:
  - hosts:
    - api.maxplay.io
    secretName: api-secret
  rules:
  - host: api.maxplay.io
    http:
      paths:
      - path: /sockjs
        backend:
          serviceName: sockjs-svc
          servicePort: 8181

NGINX

Why Nginx

Hybrid needs
- On-premises and Google Cloud Platform
Consistent administration and capabilities
- central load balancing and proxy platform
Support for VM and container-based technologies
- with minimal configuration change
Deeper feature set available as services/team matures
- use as an Application Delivery Controller

Platform independence

@lcalcote

Common Administration w/Nginx Plus

App

@lcalcote

A/B Testing Using Nginx

Ingress Controller and Deployments

Requests

App v0.1

App v0.2

90% of requests go to v0.1

10% of requests go to v0.2

@lcalcote

GSLB & Content Caching w/Nginx Plus

App

GeoDNS

US Regions

Session Persistence and Sticky Routing help in performance of request routing and localized content
Content Caching provides faster retrieval of data
Performance, reliability and availability

Global Regions

@lcalcote

Lee Calcote

linkedin.com/in/leecalcote

@lcalcote

blog.gingergeek.com

lee@calcotestudios.com

Thank you. Questions?

clouds, containers, infrastructure,

applications and their management

http://calcotestudios.com/ talks