Service mesh & Istio for K8 cluster

Monolithic design to micro service design

Let’s say we have following monolithic app. Since all module is part of same binary therefore each module interaction is a simply in-process execution therefore we don’t need to worry about security, service discovery etc.

But as our user base grows, we can’t scale the monolitic app. We will need to adopt the microservice design pattern and deploy it on Kubernetes cluster (let’s assume K8 is our choice) as below.

Since now these are no more in-process execution i.e. each microservice is making over the network call to other microservice therefore it comes with following overhead.

1. Service discovery
2. Retry logic
3. Secure the incoming request to service
4. Monitoring to track the latency/error rate of each services
5. Tracing to track the life of each request.
6. Canary rollout to find the error on new version before rolling out 100% traffic
7. Many other problems.

Since these problems are common to every microservices therefore instead of changing code for every microservices to implement it, we can adopt the Service mesh architecture to provide the central solution without changing anything to the microservice code.

Istio is one of the implementation of Servish mesh architecture.

Service mesh architecture

It introduce a sidecar proxy with each pod to implement these common features. It also has control plane which takes care of injecting this sidecar proxy to each pod.

Now microservices can talk to each other through these proxy. Network layer consist of control plane and proxy is called Service mesh.

Istio architecture

Service mesh is a design pattern and Istio as implemented it.

The Istio architecture is composed of two main components:

1. Control plane: This is responsible for managing and configuring the proxies in the data plane. It provides a central point of control for the service mesh and allows administrators to define policies and rules that govern the behavior of the proxies. The core components of the control plane include:

• Pilot: Provides service discovery, traffic management, and resiliency capabilities to the Envoy proxies.
• Citadel: Provides security features, including certificate management and authentication.
• Galley: Configures, validates, and distributes Istio configuration to the data plane.

2. Data plane: This consists of a set of intelligent proxies (Envoy) deployed as sidecars alongside each microservice in the application. These proxies intercept and manage all network traffic between microservices, providing features such as:

• Traffic management: Routing traffic between services, including load balancing, fault injection, and traffic mirroring.
• Security: Enforcing security policies, including authentication, authorization, and encryption.
• Observability: Collecting metrics and traces on the behavior of microservices, providing insights into the health and performance of the application.

istiod is the core component of Istio's control plane. Think of it as the brain of the Istio service mesh. Before istiod, the control plane was made up of multiple separate components. But since Istio 1.5, all these components have been combined into this single binary, istiod.

VirtualService

In Istio, a VirtualService acts like a smart traffic manager for your microservices. It lets you control how requests reach your services, going beyond the basic routing provided by Kubernetes.

Here’s a breakdown of what a VirtualService does:

1. Traffic Routing: This is the core function. You can define rules to route traffic based on various criteria like:

• HTTP headers: Route traffic based on the User-Agent header (e.g., route mobile users to a different version).
• URI paths: Route requests to /api/v1 to one service and /api/v2 to another.
• Weights: Distribute traffic across different versions of your service (e.g., 70% to v1, 30% to v2).

2. Fault Injection: Want to test the resilience of your application? VirtualService allows you to simulate network issues like delays or aborts. This helps you understand how your system behaves under stress.

3. Traffic Mirroring: You can mirror traffic to a separate service for testing or debugging purposes. This allows you to analyze requests without affecting the actual traffic flow.

Following is sample yaml file for reference.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: my-virtual-service
spec:
  hosts:
  - my-service.default.svc.cluster.local # The host that the virtual service applies to
  http:
  - route:
    - destination:
        host: my-service.default.svc.cluster.local # The destination service for this route
        subset: v1 # The subset of the destination service to route to
      weight: 75 # Percentage of traffic to route to this destination
    - destination:
        host: my-service.default.svc.cluster.local
        subset: v2
      weight: 25

Dynamic service discovery

Istio’s dynamic service discovery is a key mechanism that allows services within the mesh to find and communicate with each other, even as they change and scale.

Here’s how it works:

1. Platform Adapters: Istio doesn’t reinvent the wheel when it comes to service discovery. It leverages the existing mechanisms of Kubernetes and gather information about available services.
2. Service Registry: Istio maintains its own internal service registry, which is essentially a database of all the services and their endpoints (IP addresses and ports) within the mesh. This registry is populated by the platform adapters, which continuously monitor the underlying environment for changes.
3. Pilot and Envoy: Pilot, a core component of Istio’s control plane, takes this service information and translates it into a format that Envoy proxies can understand. Envoy uses this information to route traffic to the appropriate services.
4. xDS Protocol: The communication between Pilot and Envoy happens through the xDS protocol. This protocol allows Pilot to dynamically push updates to Envoy whenever there are changes in the service registry, such as a new service being deployed or an existing service scaling up or down.

Certificate management

Istio offers robust certificate management capabilities to ensure secure communication within the service mesh. Here’s how it works:

1. Istiod as the Certificate Authority (CA):

• By default, Istio’s control plane component, istiod, acts as a CA. It generates and manages certificates for all the workloads (services) in the mesh.
• This built-in CA functionality simplifies certificate management by providing a centralized and automated way to issue and rotate certificates.

2. Certificate Signing Requests (CSRs):

• When a workload needs a certificate, its Envoy proxy generates a CSR and sends it to istiod.
• istiod validates the request and, if approved, issues a certificate signed by its own root certificate.

3. Mutual TLS (mTLS):

• Istio strongly encourages the use of mTLS to encrypt traffic between services.
• With mTLS, both the client and server present certificates to authenticate each other before exchanging any data.
• Istio’s certificate management system facilitates mTLS by automatically provisioning and rotating certificates for all workloads.

4. Integration with External CAs:

• While istiod can act as a CA, Istio also supports integration with external certificate authorities.
• This is particularly useful in enterprise environments where organizations have their own PKI infrastructure.
• You can configure Istio to use an external CA by providing it with the necessary certificates and keys.

5. Certificate Rotation:

• Istio automatically handles certificate rotation to ensure that certificates are renewed before they expire.
• This helps maintain security and prevent disruptions caused by expired certificates.

6. Citadel (deprecated):

• In earlier versions of Istio, a component called Citadel was responsible for certificate management.
• However, Citadel has been deprecated and its functionality has been integrated into istiod.

Metrics and Tracing

Istio provides robust observability features through metrics and distributed tracing.

Metrics

• Built-in metrics generation: Istio automatically generates a comprehensive set of metrics based on the four “golden signals” of monitoring: latency, traffic, errors, and saturation. These metrics provide valuable information about the health and performance of your services and the mesh itself.
• Customization: You can customize the metrics collected by Istio to suit your specific needs. This allows you to focus on the metrics that are most important to you.
• Integration with monitoring systems: Istio can integrate with popular monitoring systems like Prometheus and Grafana, allowing you to visualize and analyze your metrics data.
• Types of Metrics:
• Control Plane Metrics: Provide insights into the health and performance of Istio components like istiod, Pilot, and Galley.
• Data Plane Metrics: These are collected by Envoy proxies and provide detailed information about service-to-service communication, including request volume, latency, error rates, and more.

Distributed Tracing

• End-to-end request tracking: Istio enables distributed tracing, which allows you to track requests as they flow through your microservices architecture. This helps you understand the complex interactions between services and identify performance bottlenecks or errors.
• Automatic injection of tracing headers: Istio automatically injects tracing headers into requests, so you don’t need to manually instrument your applications.
• Integration with tracing systems: Istio integrates with popular tracing systems like Jaeger and Zipkin, providing a user-friendly interface for visualizing and analyzing traces.
• Understanding Service Dependencies: Traces help visualize the relationships between services, making it easier to identify dependencies and potential points of failure.

Istio ingress gateway

The Istio Ingress Gateway is like a doorman for your service mesh, controlling all incoming traffic from the outside world. It’s a specialized load balancer that sits at the edge of your mesh and decides how to route traffic to your internal services.

Following is sample example.

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: my-gateway
spec:
  selector:
    istio: ingressgateway # Selects the Istio ingress gateway deployment
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*" # Allows traffic to all hosts
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE # Use simple TLS termination
      credentialName: my-credential # Replace with your TLS secret name
    hosts:
    - "*"

Reference

This post is based on great following YouTube content. Feel free to watch it.

https://www.youtube.com/watch?v=16fgzklcF7Y

Enjoy learning :-)