Load Balancing Traffic to Applications in Kubernetes Cluster

Kubernetes (also known as K8s), the container orchestration tool originally developed by Google, has fast become the platform of choice for deploying containerised applications in public clouds and private clouds.

For the DevOps teams, K8s provides a common platform for deploying their applications across the different cloud environments, abstracting the intricacies of the underlying cloud infrastructure, and allowing them to focus on their tasks. For organisations, this translates into flexibility to deploy the applications in the cloud that best meets the needs of customers, while optimising costs at the same time.

This flexibility to deploy applications in any cloud, however, creates the challenge of making them accessible to end users in a reliable and consistent manner. For example, as an application is moved from a private to public cloud (such as AWS or Azure), how do you ensure the same level of accessibility, performance, reliability and security, as in the private cloud?

For making applications accessible to the end users, Kubernetes supports the following options:

NodePort: In this, a port is allocated on each node (known as NodePort) and the end users can access the application at the node’s IP address and port value. With this option, you have to manually configure a load balancer to distribute traffic among the nodes.

LoadBalancer: Like NodePort, this option allocates a port on each node and additionally connects to an external load balancer. This option requires integration with the underlying cloud provider infrastructure and hence, is typically used with public cloud providers that have such an integration. This tight integration, however, makes moving an application from one cloud provider to another difficult and error prone.

Ingress Controller: K8s defines an Ingress Controller that can be used to route HTTP and HTTPS traffic to applications running inside the cluster. An Ingress Controller, however, does not do away with the requirement of an external load balancer. As in the case of the load balancer, each public cloud provider has its own Ingress Controller that works in conjunction with its own load balancer. For example, Azure’s AKS Application Gateway Ingress Controller is an Ingress Controller that works in conjunction with the Azure Application Gateway. This again makes the access solution specific to a cloud deployment.

Clearly, none of the above options provide a truly cloud-agnostic solution. Also, while using a cloud provider’s custom load balancing or Ingress Controller solution may be quick and easy in the short term, overall, it increases the management complexity and inhibits automation as you now have to deal with multiple different solutions.

A desired solution would have the following key attributes:

Cloud-agnostic: The solution should work in both public and private clouds. This means it should be available in different form factors (such as virtual, physical and container), so that it can be deployed in a form that is optimal for that environment.

Dynamic configuration of load balancer: The solution should be able to dynamically configure the load balancer to route traffic to the Pods running inside the Kubernetes cluster as the Pods are created and scaled up/down.

Support automation tools: The solution should support automation tools for integration into existing DevOps processes such as CI/CD pipelines.

Centralised visibility and analytics: The solution should provide centralised visibility and analytics. This would enable proactive troubleshooting, fast root-cause analysis, leading to a higher application uptime.