Bringing Istio to Openstack Mesh expansion across Openstack instances

Bringing Istio to Openstack Mesh expansion across Openstack instances Tim Swanson/Arvind Somya Technical Leader May 24, 2018

Agenda Introduction of Istio and Service Mesh Istio Kubernetes on OpenStack Expanding the Istio Mesh across Openstack VMs Istio Mesh Expansion Demo Future Improvements References Q&A

Introduction

What is Istio * A uniform way to connect, manage, and secure microservices. Automatic load balancing for HTTP, gRPC, WebSocket, and TCP traffic. Fine-grained control of traffic behavior with rich routing rules, retries, failovers, and fault injection. A pluggable policy layer and configuration API supporting access controls, rate limits and quotas. Automatic metrics, logs, and traces for all traffic within a cluster, including cluster ingress and egress. Secure service-to-service authentication with strong identity assertions between services in a cluster. * Directly from https://istio.io/about/intro.html

What is a Service Mesh API External Requests Web UI Billing DB API Proxy Accounts DB Proxy Proxy Gateway/ Ingress Proxy API Business Logic Proxy API Istio’s Multicluster enhancement allows cloud boundaries anywhere in above mesh Infrastructure layer for service-to-service communication A mesh of proxies Proxies injected as sidecars Supports numerous protocols (HTTP 1/2, gRPC, TCP, UDP) Can inspect API transactions at Layer 7 or layer 3/4. Intelligent routing rules can be applied between endpoints 5

Istio Architecture Source: https://istio.io/docs/concepts/what-is-istio/overview.html

Istio Components: Envoy – a proxy injected in every application Pod (as a sidecar) Envoy is a separate open source project in the CNCF Pilot – Serves as a control plane for the proxies to configure them and provide them their routing rules Citadel (previously istio-auth) – Propagates certificates and keys to all the pods to secure both control plane and app exchanges Mixer – Provides telemetry collection as well as sophisticated receive side policy checks. 7

API Pilot transmits Config data to all Envoys Auth sends TLS Certificates to all Envoys. Control Plane Mixer enforces policy And gathers telemetry from all envoys Envoy Productpage Productpage Productpage Pilot Mixer Envoy Reviews Reviews Reviews Auth Envoy Ratings Ratings Ratings Bookinfo Application 2018 Cisco and/or its affiliates. All rights reserved. 8

Bookinfo Application Microservice Architecture V1 No stars V2 Black stars V3 Red stars

Kubernetes on Openstack

Create K8s cluster k8s Admin Openstack API

U se k8s User K8 s Openstack API cl us te r Master s Minion s K8s cluster

Kubernetes Openstack Cloud Provider Performs Openstack API operations based on kubernetes events Main Scenario: K8s loadbalancer service is created Openstack LBaaS loadbalancer created and endpoints are setup Istio Ingress Depends on Kubernetes LoadBalancer 13

apiVersion: v1 kind: Service metadata: name: istio-ingress namespace: istiosystem labels: istio: ingress spec: type: LoadBalancer . selector: istio: ingress Load Balance r Neutron Router K8s Service VIP Create Service VIP LB endpoints Openstac k Cloud provider Master s Minion s K8s cluster

LoadBalancers Created for 2 k8s LB Services Created by Kubernetes’ Openstack Cloud Provider Hosting 2 k8s services of type “LoadBalancer” LoadBalancer VMs K8s cluster VMs

Neutron Router Each Nodes’ Pod CIDR Routes Needed Openstac k Cloud provider Master s Minion s K8s cluster Node 1 router Node 2 router

Neutron Router Pod CIDR Routes (extraroutes) CNI plugin Openstac k Cloud provider Master s Pod CIDR Routes Minion s K8s cluster Node 1 router Node 2 router

VM to Pods through Router Neutron Router App VM Openstac k Cloud provider Master s Pod CIDR Routes Minion s K8s cluster Node 1 router Node 2 router

Installing Kubernetes and Istio on Openstack 2018 Cisco and/or its affiliates. All rights reserved. 19

apiVersion: v1 kind: Service metadata: name: istio-ingress namespace: istiosystem labels: istio: ingress spec: type: LoadBalancer . selector: istio: ingress Load Balance r Neutron Router K8s Service VIP Create Service VIP LB endpoints Openstac k Cloud provider Master s Minion s K8s cluster

Expanding the Istio mesh across Openstack VMs 2018 Cisco and/or its affiliates. All rights reserved. 21

Why expand the mesh? A lot of enterprise workloads still run in VMs Containerization of apps might not be possible due to resource constraints Enterprises are concerned about security in some cases Vast infrastructure investments in virtualization already Benefits of deploying apps on a mesh Transition to a container only infrastructure 2018 Cisco and/or its affiliates. All rights reserved. 22

What this demo does Spin up Bookinfo, Istio's sample application, on the Kubernetes cluster with all services in containers Migrate the backend to a MySql container Start an instance on Openstack Provision the VM to join the Istio Mesh Move the backend from a containerized MySql to a VM based MySql instance using Istio 2018 Cisco and/or its affiliates. All rights reserved. 23

Create bookinfo basic Modify route rule to route drain all traffic from Reviews and send 100% to v3 Reviews v3 Create new rule Create to send Reviews 80% v3 ofv2 pod the traffic tov2 Reviews Deploy the Ratings pod Default route rules split queries among all versions of a service Apply route rules to route all queries to Ratings pods Envoy Productpage Productpage Productpage 50% 20% Envoy Reviews Reviews Reviews Envoy Ratings Ratings Ratings 80% 50% 100% Envoy Reviews Reviews Reviews v3 Envoy Ratings Ratings Ratings v2 Bookinfo Application 2018 Cisco and/or its affiliates. All rights reserved. 24

The MySql backend service will bethe replaced by an Apply route rules to route allthe traffic to backed Ratings pod Deploy Ratings v2MySql Delete the MySql backend service Openstack MySqlMySql VM Pod Envoy Productpage Productpage Productpage Envoy Productpage Productpage MySql Envoy Reviews Reviews Reviews v3 Envoy Ratings Ratings Ratings v2 MySql Envoy Ratings Ratings Ratings v2 Bookinfo Application 2018 Cisco and/or its affiliates. All rights reserved. 25

Spin up an Register Openstack the Openstack VM with MySql VM asconfigured the MySqlfor backend the Bookinfo in Istio backend Envoy Productpage Productpage Productpage Envoy Reviews Reviews Reviews v3 Envoy Ratings Ratings Ratings v2 MySql Envoy MySql VM Openstack Bookinfo Application 2018 Cisco and/or its affiliates. All rights reserved. 26

Demo

What needs to be supplied to the VM The VM needs to be able to talk to the Istio service and Pod endpoints. This can be achieved using a netplugin like Calico to redistribute routes. The VM needs the Kubernetes secrets in order to communicate to the Kubernetes services and pods The VM should be able to discover services on the Kubernetes cluster. We can do this via either a DNS relay, Dnsmasq or static entries in the host file. Using a DNS relay pointing to the Kubernetes DNS would be the ideal solution as this will protect the connections in cases of container restart and moves. 2018 Cisco and/or its affiliates. All rights reserved. 28

Future work Better integration with major cloud providers AWS, GCE, Openstack etc. There isn’t an easy and native way to extend the Istio service mesh to conventional VM based clouds. So, we have to use hacky ways like bash scripts to extend the mesh. It would be ideal to have the ability to define service meshes per tenant/project and have VM’s electively join using a single toggle. K8s/Istio should be able to treat the VM endpoint as another version of a service to help seamlessly transition service to and from a VM (currently we have to delete local services to replace). This might be fixed with 0.8 release of Istio. 2018 Cisco and/or its affiliates. All rights reserved. 29

References: Istio Docs: https://istio.io/docs/ Istio Integrating VMs: https://istio.io/docs/guides/integrating-vms.html Istio on OpenStack Magnum Example: https://tiswanso.github.io/istio/istio on magnum.html Kubernetes OpenStack Cloud Provider: https://kubernetes.io/docs/concepts/cluster-administration/clou d-providers/ 30

Questions/ Comments?