According to Gartner estimates, by 2027 more than 90% of global organizations will be running containerized applications in production. A significant increase from fewer than 40% in 2021. As containerization gains traction and becomes the de facto standard for deploying applications, Kubernetes is seeing a sharp increase in usage across enterprises.
Having said that, a recent survey conducted by CNCF in partnership with FinOps foundation revealed an increase in Kubernetes cloud cost across the board during 2023. A surprising finding is that most respondents needed help tracking and predicting Kubernetes cost-effectively. A wise man once said, “A penny saved is a penny earned,” but measuring costs correctly in Kubernetes environments is complex.
This article discusses best practices and tools for implementing cost optimization and gaining accurate visibility for your Kubernetes workloads.
Challenges in Managing Kubernetes Cloud Cost
Let’s quickly take a look at some of the challenges of managing costs with Kubernetes:
- Kubernetes workloads are dynamic, making it difficult to accurately calculate the cost of the resources used by your apps.
- We frequently deploy applications to a multi-tenant Kubernetes cloud cluster, which complicates accurately estimating resource costs. While cloud vendors make it relatively easy to know that your Kubernetes cluster is costing you $100 per day, it can be difficult to estimate how many resources, i.e. compute and memory, a team or project is using.
- Even if everything is reduced to CPU, memory, and load balancers, various external considerations such as shared resources and cluster management fees can complicate your Kubernetes cluster cost estimation.
- Overprovisioning of resources and a lack of awareness and responsibility among individuals and teams can increase the cost estimate of your Kubernetes cluster cloud. Overprovisioning can also lead to the tragedy of commons situation where there is no incentive to be modest while dimensioning deployments.
- In the absence of standardized processes and tools to visualize Kubernetes cost, budgeting or spending analysis becomes difficult.
- Sometimes, when you are unsure about your workload consumption trends, which fluctuate often, it is hard to get accurate information on Kubernetes cloud cost.
Splitting Down Costs
Knowing how much your managed Kubernetes cloud cluster will cost ahead of time is preferable to being surprised when you receive your monthly bills. The following components make up the typical cost of running an application on a managed Kubernetes cloud cluster:
- CPU and memory
- Persistent storage
- Ingress/Egress
- Network
- Common services (i.e., control plane and service mesh)
- Cluster management fee
How Cloud Providers Charge
Your organization’s decision to harness the power of Kubernetes depends on a set of parameters, and cost is crucial. But do you know how different cloud providers charge you for managed Kubernetes instances?
Every cloud provider charges you differently for the managed control plane. For example, Digital Ocean comes with a free control plane. This free service includes management for processes like etcd, kube-apiserver, kube-controller-manager, kube-scheduler, cloud-controller-manager, and other services. Civo is another managed Kubernetes service vendor that offers the control plane for free
While on the other hand, Google Kubernetes Engine and AWS Elastic Kubernetes Service charge for the control plane usage. Keeping track of all factors impacting Kubernetes cluster cost can be overwhelming.
Strategies to Manage Kubernetes Cloud Costs
The following are some best practices teams can follow to optimize their Kubernetes cloud costs.
- Right-sizing your Kubernetes cluster nodes is crucial in optimizing resource utilization. If your containers are too large and consume most of your node resources, adding more containers on that node would be a task.
- You should right-size pods using Kubernetes resource requests and limits by specifying the minimum and maximum amount of resources your application can request.
- Leveraging autoscaling and downscaling features of Kubernetes using the horizontal pod autoscaler and cluster autoscalercan drastically reduce your cloud expenditures.
- Kubernetes resource bin packing is another technique you should consider for cost-efficient utilization of your infrastructure. In this technique, multiple containers are consolidated into fewer nodes to increase the utilization of your cluster nodes.
- Cloud discounts (i.e., AWS saving plans)
- Startup programs (i.e., Y Combinator companies get $100k in GCP credits)
- If your Kubernetes workloads are stateless, you can run them on spot instances to save cost.
- Negotiate and ask for volume-based discounts
Tools to Manage Kubernetes Costs
Ascertaining and managing your Kubernetes costs is a complex task, and you need multiple tools in your arsenal to do some of the heavy lifting.
Cast AI
CAST AI is a Kubernetes automation platform that claims to cut cloud costs by half.
- CAST AI can scan your Kubernetes cluster within a few minutes and give you a cost-saving report to know how much you could have saved using its features. Furthermore, if you agree with the report’s findings, you can structure CAST AI to implement those changes to cut costs immediately and automatically.
- CAST AI also has an AI-driven instance selection mechanism that, based on your requirements, decides on the best instance type at the minimum cost.
- CAST AI has bin packing features that allow you to compact pods in fewer nodes and later on delete nodes when they no longer have workloads scheduled on them. In this approach, your nodes reach high utilization, and you don’t have to pay for resources that aren’t utilized.
Kube-downscaler
kube-downscaler is an open-source framework that can reduce your Kubernetes resources to zero when they are not being used actively.
- kube-downscaler scales down Kubernetes resources during non-working hours. It’s like Thanos snapping his fingers, and all your Kubernetes pods are effectively scaled down to zero.
- You can define schedules based on your usage pattern, and then it will scale down your cluster. For example, you can scale down the cluster on weekends.
- It is great for your lower environments, where you might not need the infrastructure to be available 24/7.
Harness
Harness is a modern software delivery platform powered by AI. It has built-in capabilities to manage your cloud bills with its Cloud Cost Management feature.
- Harness provides in-depth, actionable insights into your dynamic idle Kubernetes resources by effectively shutting down non-production idle resources with its cloud AutoStopping feature. Harness claims that by using this feature, teams can save up to 70% on non-production cloud costs, all with a simple one-time setup that doesn’t need maintenance.
- Estimate how much of your existing cloud costs you can save using its savings calculator.
- Detecting and flagging anomalies in your cloud consumptions by analyzing historical spending patterns.
Kubecost
Kubecost is the most popular open-source tool for getting a real-time bird’s eye view of Kubernetes costs. It uses preconfigured Grafana dashboards with its opinionated setup (tailored to your needs) to monitor Kubernetes cluster costs. You can install it using a Helm chart.
- It is a unified Kubernetes cost-monitoring solution that supports all major cloud providers.
- Automatically generates Kubernetes cost optimization recommendations via its UI or API. These recommendations can be dynamically applied for immediate results.
- Provides real-time alerting capabilities so that teams can take control of their Kubernetes-related costs and stay within budgeted limits.
- Breaks down the cost allocation to Kubernetes resources such as deployments, namespaces, labels, pods, and containers (but not limited to these). You can transparently and accurately share the cost among your teams and tally it with your cloud bills.
OpenCost
OpenCost is an open-source cost monitoring tool for cloud-native environments. It is a CNCF sandbox project created by Kubecost.
- If you have budget constraints, this is the only tool built by Kubernetes experts and practitioners that is freely available in the market.
- It relies on vendor-neutral OpenCost Specification to measure and allocate Kubernetes clusters’ infrastructure and container costs.
- OpenCost API enables real-time and historical reporting on Kubernetes cloud costs based on on-demand list pricing.
Loft
Loft’s Kubernetes platform has excellent features to help you manage resource costs.
- Sleep mode can scale down your Kubernetes environment by putting namespaces to sleep and purging all pods inside a namespace during periods of inactivity. Kubernetes resource configuration is not purged and remains in the cluster.
- Auto-delete, as the name implies, deletes namespaces and virtual clusters that have been idle, old, or unused for some time.
Harness Case Study
Relativity, a SaaS data company, reduced its cloud costs by millions in five months using Harness cloud cost management features. Relativity doubled the density of Kubernetes pods per node from 40 to 100 by analyzing the poor utilization of Kubernetes pods operating on cloud infrastructure (nodes), and in some large nodes, 100 pods per node. The financial impact of this was a 40% reduction in Kubernetes costs per day. They saved one full-time employee from the expenditures of developing an in-house cost management platform for product and engineering teams using Harness Cloud Cost Management.
Final Thoughts
While it is customary for engineering teams to overprovision resources to run their Kubernetes clusters, this approach may not be sustainable—someone must pay in the end. So, whether you are a seasoned expert or just starting with Kubernetes, you must consider the costs of maintaining a Kubernetes cluster.
