Kubernetes Ingress Controller Deep Dive

1. Introduction

Kubernetes Ingress¹ is an API object that manages external access to services in a cluster, typically HTTP and HTTPS traffic. It is managed by the Ingress Controller², that watches and controls the Ingress resources. There are multiple well-known Ingress Controller Types:

• Cloud-native Controllers: Provision and integrate with cloud-provider specific load balancers, like Network Load Balancers in AWS, etc.
• Software-based Controllers: Provision software load balancers and reverse proxies, like nginx, traefik, envoy, etc.

2. Architecture

2.1. Components

Ingress controllers run as k8s pods and implement the following control loop:

1. Watch loop: watches for changes to Ingress, Service, and Endpoints resources
2. Configuration sync: Translates Kubernetes resources into load balancer configuration
3. Health Management: Monitors backend pod health and updates routing accordingly

2.2. Config Map Management

The controller dynamically updates nginx.conf based on Ingress resources. Go templating is utilized to generate nginx config on the fly, and SIGHUP³.

2.3. Service Discovery

Controllers discover backend pods through multiple mechanisms, like Endpoints API, EndpointSlices, and DNS Resolution.

3. Request Flow

{{< mermaid >}} graph LR A[Client] —> B[DNS] B —> C[Cloud LB] C —> D[Ingress Pod] D —> E[K8s Service] E —> F[Backend Pod] F —> E E —> D D —> C C —> A {{< /mermaid >}}

4. Cloud Provider Integration

Cloud provider ingress controllers automatically configure some default settings, such as:

1. Security groups and Network ACLs
2. Health checks
3. Backend configuration: Services as registered as load balancer targets
4. Multi-AZ target distribution

5. Advanced features

5.1. Path-based Routing

Ingress controllers support sophisticated routing patterns, like:

1. Exact Path Matching: Like matching /api/v1/users to user-service.
2. Path Prefix Matching
3. Regex Patterns: specific to nginx

5.2. Load Balancing Algorithms

Multiple load balancing algorithms are also supported:

1. Round Robin (Default): evenly distributed traffic to all pods
2. Least Connections: more traffic routed to pods with fewer connections
3. IP Hash (Session Affinity): a hash of multiple parameters, like source IP, source port, etc are computed and then mapped to a target.
4. Weighted Distribution: user-defined priorities for different IPs

5.3. More advanced features

1. cert-manager integration for automatic certificate provisioning and renewal
2. Session affinity: like cookie-based using nginx, or IP-based.
3. Rate limiting
4. WAF (Web App Firewall) integration
5. Multi-zone deployments

6. What Next

I enjoyed this deep dive into ingress controllers. Next, I want to explore how Kubernetes controllers work in general and learn how to build custom controllers.