How Kubernetes HPA scales

"Autoscaling" gets used as a single word for three different things, and conflating them is where most confusion starts.

• Horizontal scaling adds or removes running instances (Pods) to match demand.
• Vertical scaling changes the resources — CPU, memory — allocated to instances that are already running.
• Cluster-level scaling adds or removes the underlying nodes a cluster runs on, usually triggered when Pods can't be scheduled due to insufficient node capacity.

This piece covers the first one — Kubernetes' Horizontal Pod Autoscaler (HPA) — since it's the mechanism most people mean when they say "autoscaling," and its logic is publicly documented in enough detail to describe precisely rather than approximately.

What is the Horizontal Pod Autoscaler (HPA)?

The HPA is a Kubernetes controller that automatically adjusts the number of Pod replicas in a Deployment or StatefulSet based on observed metrics like CPU and memory utilization.

How does the HPA control loop work mechanically?

The HPA is not a continuous, always-watching process. It's a control loop that runs intermittently — defaulting to every 15 seconds per Kubernetes' current documentation — and on each cycle it does four things in sequence:

1. Finds the target workload (a Deployment or StatefulSet) via the HPA's scaleTargetRef
2. Selects the Pods belonging to that workload
3. Pulls the relevant metric for each Pod — CPU and memory from the built-in metrics API; custom metrics from a custom or external metrics API
4. Computes a ratio between the current metric value and the target value, and uses that ratio to decide the new replica count

Nothing scales between cycles. If a traffic spike hits at second 1 and resolves by second 14, the HPA may never even register it happened.

What is the actual HPA scaling formula?

Kubernetes' own documentation states the calculation directly:

desiredReplicas = ceil(currentReplicas × (currentMetricValue / desiredMetricValue))

If current CPU usage is 200m against a target of 100m, replicas double (200 ÷ 100 = 2.0). If current usage drops to 50m against the same target, replicas halve (50 ÷ 100 = 0.5).

Two details matter more than the formula itself:

• A tolerance band prevents constant micro-adjustments. The control plane skips scaling entirely if the ratio is close enough to 1.0 — within a configurable tolerance that defaults to 0.1 (10%). Without this, a workload sitting almost exactly at target would flap up and down every cycle over noise.
• Not-ready and missing-metric Pods get handled conservatively. If a Pod's metrics are missing, the controller sets it aside and recomputes more cautiously — assuming it's using 100% of target during scale-down, and 0% during scale-up — specifically to avoid overreacting to incomplete data.

Why is scaling down deliberately slower than scaling up?

Kubernetes doesn't act on a single scaling recommendation the moment it's computed. Before the HPA actually scales the target, the controller records the recommendation, then chooses the highest recommendation from within a configurable window — defaulting to 5 minutes — before acting on a scale-down. That's the downscale stabilization window, and its purpose is to prevent thrashing: without it, a brief dip in traffic would trigger an immediate scale-down, only for the next cycle's spike to trigger an immediate scale-up.

There's no equivalent forced delay on scale-ups by default — the asymmetry is intentional. Under-provisioning during real demand is a worse failure mode than briefly over-provisioning during a lull, so Kubernetes' defaults are biased toward reacting fast when load increases and reacting cautiously when load decreases.

Where does HPA break down in practice?

The mechanics are precise, but the defaults have real limits:

1. The reaction is never instant. Between the 15-second poll interval, the time new Pods take to start serving traffic, and the tolerance band absorbing small fluctuations, there's unavoidable lag between "load increased" and "capacity caught up." New Pods also get a grace period — initial readiness delay (default 30s) and CPU-initialization period (default 5 minutes) — before metrics are trusted.
2. Averages hide spikes. The default calculation uses the mean metric across all targeted Pods. A workload with uneven request costs can have a healthy average while a subset of Pods are overloaded. HPA has no visibility into that distribution by default.
3. CPU and memory aren't always the metric that matters. For request-latency-sensitive services, scaling on CPU alone can miss the actual bottleneck; Kubernetes supports custom and external metrics for this, but it must be configured deliberately — it isn't the default.

Quick Reference: Key HPA Defaults

Version Note

Older Kubernetes docs and many blog posts state the default sync period as 30 seconds. Current upstream docs state 15 seconds. If you're working against a specific cluster, check that cluster's actual kube-controller-manager flags — defaults have changed across versions.

References

• Kubernetes, "Horizontal Pod Autoscaling" — official documentation, including algorithm details
• Google Cloud, "Horizontal Pod autoscaling | GKE"