Kubernetes Resource Requests and Limits for Cost Efficiency

Kubernetes resource requests and limits are not a configuration detail. They are the financial architecture of your cluster. Get them wrong and you are not just risking instability; you are systematically overpaying for compute you will never use, while simultaneously creating silent failure modes that your dashboards will never surface. There are four specific failure modes: scheduling waste, runtime OOM kills, CPU throttling, and eviction under memory pressure. Each has a different cause and a different fix. Treating them as a single "resource tuning" problem is precisely why most teams solve one and quietly worsen another.

The terminology actively misleads people, so it is worth being precise before touching anything. Requests are what the scheduler reads when deciding where to place a pod. Reserved capacity, not a measurement of actual usage. If a pod requests 1 CPU, the scheduler treats 1 CPU as consumed on that node whether the pod is doing real work or sitting completely idle. Limits are invisible to the scheduler entirely. They only become relevant after the pod is running, enforced at runtime by the kubelet. And here is where CPU and memory diverge in ways that matter: exceed a CPU limit and the container gets throttled, it slows down but survives. Exceed a memory limit and the container is terminated immediately. No warning, no graceful shutdown, no paging to disk. Engineers coming from a VM background, where the OS can page under pressure and the process lives, are often genuinely surprised when the container just disappears — it is a bit like expecting a revolving door and walking into a wall.

QoS class is the third piece of this. Kubernetes assigns each pod a class based on how requests and limits are configured. Guaranteed means request equals limit for every resource on every container, making that pod the last to be evicted under node pressure. Burstable means requests and limits differ, or only one is set. This directly shapes survivability during memory pressure events and should inform how you configure critical services versus background jobs. Most teams I have seen treat these classes as incidental output rather than deliberate design.

Now, the utilization numbers. According to the Cast AI 2026 State of Kubernetes Optimization Report, average CPU utilization across tens of thousands of clusters fell to 8% in 2025. Memory utilization sat at 20%. These are not dev environments or neglected staging clusters. These are production workloads actively serving traffic, running at a fraction of their reserved capacity.

The directional trend makes it worse. CPU overprovisioning rose from 40% to 69% year over year in that same dataset. Memory overprovisioning stands at 79%. The Cast AI 2025 Kubernetes Cost Benchmark Report found that 99.94% of clusters are over-provisioned. This is not an edge case. This is the default state of production Kubernetes.

The mechanism is straightforward. A pod requesting 1 CPU but using 0.2 CPU causes the scheduler to treat that full CPU as unavailable to every other workload on the node. A four-CPU node fills up at four pods, even though actual compute consumed is less than a single core. Teams are paying for headroom that is not protecting any workload; it is just occupying space and forcing the cluster to scale out earlier than it needs to. Per a 2025 CNCF survey, 68% of organizations running production Kubernetes clusters overspend by 30 to 45% due to overprovisioned nodes, idle pods, and misconfigured autoscaling. That is not a rounding error. That is a structural budget leak repeating every billing cycle.

Why do engineers do this? Usually because they have been burned by throttling before. Setting requests high feels safe, a hedge against the invisible degradation of CPU limits that are too tight. The problem is that this choice accumulates across every deployment, every namespace, and every team, until the collective waste becomes load-bearing. It is like every developer on the team leaving their car idling in the parking lot just in case they need to leave quickly — individually reasonable, collectively absurd.

What a Single Misconfigured Pod Actually Costs at Scale

Consider a production pod requesting 4,000m CPU and 8Gi of memory. Actual P99 usage: 450m CPU and 950Mi memory. Roughly nine times the CPU and memory reserved compared to what the workload demonstrably uses under real conditions.

The cost consequence of that single pod is $792 per month in pure scheduling waste. Across ten replicas, $7,920 per month, every month, with the application behaving perfectly fine. The pod passes health checks. It serves traffic. Every dashboard shows green. The waste is entirely invisible unless someone is watching utilization next to requests, side by side, doing the arithmetic. Most teams are not doing that arithmetic.

The compounding is where it gets genuinely punishing. The scheduler is reserving that capacity on real nodes, which means other pods cannot be placed there. The cluster is forced to provision additional nodes to accommodate demand that would fit on existing hardware if requests reflected actual usage. Waste at the pod level becomes waste at the node level, then at the cluster level. It propagates rather than accumulates, the way a single blocked lane backs up traffic miles behind an incident that cleared twenty minutes ago.

There is a mirror failure mode worth naming here, because teams that identify this problem and overcorrect can create something worse. Cutting requests to reclaim cost while leaving limits too low produces workloads that run but silently degrade under load. Latency climbs, requests time out, and engineers spend days investigating application code when the actual cause is CPU throttling that produces no alert, no error log, and no obvious signal anywhere people typically look.

The Right-Sizing Process: Measuring Actual Usage Before Touching Any Manifests

The first rule of right-sizing: configure nothing until you have real consumption data. This sounds obvious. It is routinely ignored. Estimation, intuition, and "what we set last time" are the primary inputs behind most resource configurations in production today, and the Cast AI numbers are the predictable result.

Observation windows matter more than people account for. Capture at least 7 to 14 days of utilization data, specifically chosen to include realistic traffic variation: weekday versus weekend patterns, any known spikes, and batch processing windows if your workload has them. A one-day window during low-traffic hours will produce numbers that cause OOM kills during peak load. I have seen this happen on services that had been running for over a year, re-configured during a quiet Tuesday afternoon when nothing was happening, and then killed the following Monday morning when traffic came back. The team spent two days looking at application code before someone pulled up the memory limits.

Target P95 or P99 usage as your baseline, not averages. Averages are mathematically guaranteed to underrepresent the peaks that cause terminations and throttling. Setting requests to average usage means roughly half of traffic bursts will exceed what the scheduler reserved. That is not a configuration; that is a scheduled failure.

CPU requests should land at P95/P99 demand plus workload-appropriate headroom, targeting a utilization ratio of 60 to 80% of requested resources. The industry-typical 20 to 30% ratio is the signature of the overprovisioning problem the Cast AI data documents. The gap between those two numbers is money.

Memory is the more consequential dimension, because the failure mode is termination, not slowdown. Memory requests and limits should be set close together, often equal, because the apparent flexibility of a gap between them is largely illusory. If a container's memory usage climbs past its request but below its limit, it is still subject to eviction under node pressure before ever reaching the limit. And if it hits the limit, it is killed. The gap provides minimal protection and real risk.

CPU limits are a genuinely nuanced question. For latency-sensitive production services, many experienced teams remove CPU limits entirely and rely on requests for fair scheduling. CPU throttling is a subtle and frequently invisible tax on performance, and the Prometheus metric that exposes it, containercpucfsthrottledperiods_total, is absent from most teams' default alert configurations. Unless that metric is being actively monitored, cutting limits to save cost creates a latency problem that never surfaces as an alert, which means it persists indefinitely.

How Autoscalers Use Requests, and Why Misconfigured Requests Break Both HPA and VPA

The Horizontal Pod Autoscaler calculates its scaling threshold as a percentage of requested resources, not of actual node capacity. This is the detail that breaks autoscaling in both directions. Inflated requests cause HPA to perceive headroom that does not exist in usable terms, so it scales out later than workload conditions warrant, or triggers on load that reflects misconfigured baselines rather than real demand. According to Datadog's 2025 State of Containers report, 86% of HPA users apply it across most of their clusters. At that adoption rate, misconfigured requests are corrupting autoscaling behavior for the substantial majority of production Kubernetes environments.

The Vertical Pod Autoscaler observes historical usage and recommends right-sized requests, or in Auto mode, applies them directly by recreating pods. That recreation behavior is why VPA in Auto mode is inappropriate for stateful workloads. Databases, caches, and message queues should use VPA in Off or Initial mode, with recommendations applied manually during planned maintenance windows where pod disruption is acceptable and expected.

There is also a well-documented conflict between VPA and CPU-based HPA running simultaneously on the same deployment. VPA resizes the very resource that HPA uses to calculate its scaling threshold. The result is unpredictable scaling behavior from the interaction between the two systems. Running them together on the same deployment produces outcomes neither system was designed for; pick one or architect around the conflict deliberately.

Karpenter, which reached version 1.1 as a CNCF incubating project in late 2025, provisions new nodes in 30 to 60 seconds versus the 3 to 5 minutes typical of legacy Cluster Autoscaler, and selects instance types based on actual pending pod requirements rather than a fixed configuration. It actively consolidates underutilized nodes by evicting pods and terminating unnecessary instances. That consolidation behavior is only effective when pod requests reflect real usage. Karpenter cannot consolidate effectively around inflated requests because the scheduler legitimately cannot fit more workloads onto those nodes, even when actual utilization would permit it. The right-sizing problem and the autoscaling problem are the same problem.

Namespace-Level Governance That Keeps Over-Provisioning From Creeping Back

Right-sizing without governance is a one-time improvement that decays within a few deployment cycles. The overprovisioning returns as developers set new requests by habit, copy configurations from old manifests, or omit resource specs entirely because nobody ever explicitly told them not to.

LimitRanges establish default and maximum resource requests and limits at the namespace level. Any container deployed without explicit resource specifications inherits the namespace defaults. This eliminates pods running with no requests set at all, which is more common than it should be and has scheduling consequences that accumulate quietly over months.

ResourceQuotas cap total CPU, memory, and storage consumption within a namespace, preventing any single team or environment from consuming disproportionate cluster capacity. In multi-tenant clusters without quotas, the typical outcome is resource hoarding: teams set inflated requests to protect themselves against capacity contention they have experienced before, other teams cannot get pods scheduled, and Finance cannot produce accurate chargeback data because consumption is indistinguishable from waste. The problem has a technical root cause and a technical fix. The reason it persists is that the fix requires deliberate implementation.

Policy-as-code tools, specifically OPA and Kyverno, both CNCF projects, allow cost governance to be enforced as admission policies. Misconfigured requests, missing resource specs, or requests that exceed defined thresholds are rejected at deploy time, before they ever reach a running cluster. Catching a misconfigured manifest in CI costs essentially nothing. Discovering the same misconfiguration after ten replicas have been running for a month costs considerably more than nothing.

Non-production environments deserve their own governance policy. Running development and staging clusters only during business hours, roughly 60 hours per week against continuous operation, reduces non-production compute costs by 64%. Depending on cluster size, that is $5,000 to $15,000 per year from scheduling decisions alone, before any workload optimization.

GPU Workloads: The Highest-Stakes Version of the Same Right-Sizing Problem

GPU resource requests follow the same scheduling logic as CPU and memory. The scheduler reserves the requested GPU count against node capacity regardless of actual utilization. The difference is cost per unit of idle time. A wasted GPU instance hour compounds faster and is harder to absorb than an equivalent CPU hour, which means right-sizing errors are more consequential here, not less.

GPU sharing and fractional GPU allocation, available through tools like NVIDIA's device plugin with time-slicing, allow multiple pods to share a single GPU. The right-sizing logic is identical: request only what the workload demonstrably needs, validated against observed usage rather than theoretical maximums that the model will never actually encounter in production.

Inference workloads and training jobs have meaningfully different resource profiles. Inference is typically latency-bound, benefits from Guaranteed QoS to avoid eviction under memory pressure, and warrants precise resource specifications that prioritize consistent performance. Batch training tolerates eviction and is well-suited for spot or preemptible GPU instances, which reduce cost without impacting job completion, only completion time. Configuring both workload types identically is waste in one direction or brittleness in the other.

The P95/P99 measurement discipline applies here exactly as it does for CPU and memory. GPU memory requests should be based on observed peak usage during representative inference runs. The gap between theoretical maximum and actual P99 peak is frequently large, and every hour that gap is provisioned but unused is a direct cost with no corresponding benefit.

What Realistic Savings Look Like After a Structured Right-Sizing Program

For clusters with unoptimized resource requests, right-sizing alone typically delivers 20 to 40% savings on cluster node costs. Combined with Karpenter consolidation and spot adoption for stateless workloads, total savings reach 30 to 60% of cluster spend. Structured optimization programs covering right-sizing, autoscaler tuning, spot adoption, and non-production cleanup typically recover 30 to 50% of cluster spend. Autonomous optimization platforms that manage clusters continuously report reductions in the 50 to 75% range on the clusters they manage.

The structural nature of those savings is what matters most. Correctly set requests change what the scheduler reserves on a permanent basis. Every subsequent pod placement benefits. That is categorically different from a one-time audit that decays as soon as the next deploy cycle runs without governance in place.

Teams that skip measurement and right-size by intuition tend to reintroduce the same waste within one or two deployment cycles. The manifests look better. The habits have not changed. The governance is absent. Six months later, the bill is back.

For teams without dedicated DevOps capacity, the operational overhead of sustained right-sizing is itself a cost that needs accounting. Monitoring utilization, tuning VPA recommendations, managing Karpenter NodePools, and enforcing admission policies requires consistent attention that product engineering teams rarely have bandwidth for. Teams running Kubernetes on shared-tenant platforms often lack visibility into utilization metrics entirely, which makes the measurement step impossible before any other decision can be made. Some managed platforms deploy into a customer's own AWS, GCP, or Azure account, giving engineering teams direct access to their cluster's resource data and cost signals, and handle cluster management, autoscaling configuration, and CVE patching, removing that operational overhead from product engineering teams.

The savings from right-sizing are real, well-documented, and available to nearly every team running Kubernetes at any meaningful scale. The reason 99.94% of clusters remain over-provisioned is not technical difficulty. Nobody made the configuration decision deliberately the first time, and governance was never put in place to prevent the defaults from compounding into a structural cost problem. That is a solvable problem. Most teams just have not solved it yet.