Why Traffic Shaping Is the Missing Piece in Your Kubernetes Scaling Strategy

Author: Thomas Mathew (opens in a new tab) Date: March 12, 2026

In 2019, Netflix experienced an outage that left members unable to play content. Every request, video playback, UI animations, analytics, A/B test logging competed for the same overwhelmed infrastructure. The fix wasn't more capacity. It was prioritized load shedding: a system that automatically drops low-priority traffic (logging, analytics, UI enhancements) while keeping the core experience - video playback running. Days after deploying it, a similar incident occurred. This time, members never noticed. (opens in a new tab)

Netflix isn't alone. Google, Uber, LinkedIn, Shopify, and Agoda all run priority-based traffic shaping in production. It's become a standard reliability pattern at scale. But in the Kubernetes ecosystem, most teams still don't have it.

Your autoscaler handles capacity. Traffic shaping handles priority. Without both, you're forced into an impossible choice: over-provision for safety (wasting 30-50% of resources) or accept degraded performance during spikes.

A Proven Pattern: How the Industry Leaders Do It

Priority-based load shedding isn't a new idea. It's a battle-tested reliability pattern used by companies processing millions of requests per second. What they all have in common: not all traffic is created equal, and treating it equally during overload is the wrong default.

Netflix: Progressive Priority Scoring

Netflix assigns every request a priority score from 0 (highest) to 100 (lowest) (opens in a new tab). As system load increases, shedding progressively expands from the lowest-priority traffic upward. At 35% overload, only requests scored above 95 are shed. At 80% overload, the threshold drops to ~50. Video playback requests sit near 0, they are never shed. This approach evolved from gateway-level (Zuul) to service-level, giving each service control over its own priority decisions.

Google: Request Criticality Propagation

Google's SRE practices mark every request with a criticality level from "critical user-facing" down to "non-critical background" (opens in a new tab). Criticality propagates automatically through their RPC system: if Request A calls Service B and Service C, both inherit A's priority. When a service is overloaded, it sheds from the lowest criticality first (opens in a new tab). Google also employs graceful degradation under load, a search may return cached results instead of querying the full index, preserving the user experience at reduced quality rather than failing entirely.

Uber: Dynamic Load Shedding (Cinnamon)

Uber built QALM (QoS-Aware Load Management) (opens in a new tab): a framework that ranks requests into tiers and dynamically tunes concurrency limits using P99 latency metrics. When their Cinnamon load shedder activates, lower-priority traffic is dropped before latency-sensitive operations are affected. Results under overload: throughput increased ~80%, P99 latency for critical operations dropped ~70%.

LinkedIn: Hodor - Three Traffic Tiers

LinkedIn's Hodor system (opens in a new tab) classifies all traffic into three tiers: non-degradable (member-facing requests must never be dropped), degradable (can serve reduced quality), and optional (offline/nearline systems can be dropped with zero user impact). Overload detectors run inside each process and query every inbound request against these tiers.

The Common Pattern

Despite different implementations, every company converges on the same principles:

1. Classify requests by business priority: not all traffic matters equally
2. Shed progressively from lowest priority: protect critical paths at all costs
3. Use real-time signals: concurrency, latency, error rate, resource pressure
4. Automate the response: humans are too slow during a traffic spike

The Gap in Kubernetes

Despite the industry consensus, most Kubernetes clusters run without any priority-based traffic shaping. Teams rely on tools that weren't designed for this problem:

Rate limiting is blind to business context. It caps requests per second regardless of what those requests are. A rate limiter treats a payment submission the same as a prefetch call, exactly the problem Netflix, Google, and Uber solved by adding priority awareness.

Circuit breakers are binary. They trip when a service is unhealthy, cutting off all traffic. There's no graduated response. A circuit breaker can't say "shed analytics traffic but keep checkout flowing", which is exactly what LinkedIn's Hodor does with its three-tier model.

EnvoyFilter YAML is static and manual. Istio's EnvoyFilter lets you write custom traffic rules, but they're static configurations with no awareness of real-time workload pressure. Uber's Cinnamon dynamically tunes thresholds using live P99 latency, static YAML can't do that.

The gap isn't knowledge, it's implementation. Building a Netflix-style load shedder requires deep infrastructure engineering. Wave Flow brings this proven pattern to any Kubernetes cluster as a deployable solution.

Wave Flow: The Industry Pattern, Ready for Kubernetes

Wave Flow implements the same priority-based load shedding pattern used by Netflix, Google, and Uber, but as a deployable WebAssembly (WASM) module that integrates directly into your existing service mesh proxies (Istio, Envoy, Kong, NGINX). No custom infrastructure. ~0.1ms decision overhead.

It classifies every incoming request into four priority tiers modeled after the tiered approaches proven at scale:

Requests are classified by HTTP headers, URL path prefixes, and HTTP methods, the same classification dimensions Google uses (hostname, URL path, user ID). Unmatched traffic defaults to BULK, ensuring critical transactions are always protected first.

How It Works in Practice

Consider an e-commerce platform during a flash sale. A scenario Shopify has written extensively about (opens in a new tab), where traffic can spike 100x above normal:

• /checkout/* and /payment/* paths → CRITICAL (never shed)
• /cart/* and X-User-Tier: premium headers → IMPORTANT (shed only above 95%)
• /products/* and /search → MODERATE (shed above 80%)
• /recommendations, /analytics → BULK (shed above 60%)

When CPU utilization hits 65%, Wave Flow starts shedding recommendation and analytics calls, similar to how Netflix sheds logging and A/B test traffic first. As pressure climbs to 85%, product browsing gets throttled too but payments keep flowing at full capacity. By the time the autoscaler's new pods are ready, critical revenue has been protected throughout the transition.

Three Deployment Modes

Wave Flow integrates with your existing infrastructure, no service mesh replacement needed:

1. Ingress Gateway Mode: Protect all traffic entering the cluster. Works with Istio Ingress Gateway, Kong, NGINX, Envoy Gateway, and APISIX.
2. Sidecar Mode: Per-service traffic shaping for granular control, similar to Netflix's evolution from gateway-level to service-level load shedding.
3. Ambient Mesh Mode: Zero-sidecar deployments with Istio ambient mesh (1.18+), combining per-service control with shared proxy efficiency.

What This Means for Your Team

For Platform Engineering Teams

Netflix, Google, and LinkedIn all built custom load shedding into their internal platforms. Wave Flow gives your platform the same capability without the custom engineering. It becomes part of your platform contract. A standardized, 4-tier traffic priority model that every development team uses. Define once, deploy everywhere.

For SRE / Reliability Teams

At 3am, the page fires. With Wave Flow already running, BULK and MODERATE traffic are being throttled automatically as utilization rises, the same way Netflix's Zuul gateway automatically shed low-priority traffic during their 2020 incident. Your job shifts from "apply emergency rate limits under pressure" to "verify automation is working." Graceful degradation replaces cascading failure.

For Engineering Leadership

With e-commerce downtime costing an average of $14,056 per minute (opens in a new tab) and large enterprises averaging $23,750 per minute, protecting revenue-critical transactions during traffic spikes has direct financial impact. Wave Flow ensures that during the moments when traffic exceeds capacity, the requests that generate revenue are the ones that get through.

Getting Started

Wave Flow deploys as WASM modules into your existing Istio or Envoy infrastructure. No service mesh replacement, no additional sidecars, no custom code.

A practical rollout path:

1. Classify: Identify your critical services and assign priority tiers, the same exercise Netflix and LinkedIn did when defining their priority scoring systems.
2. Deploy: Apply Wave Flow policies in one cluster. Monitor shaping behavior for 1-2 weeks under normal traffic.
3. Expand: Roll out to remaining clusters with consistent priority policies.

Priority-based load shedding is a proven pattern at Netflix, Google, Uber, LinkedIn, and Shopify. The difference is that those companies spent years building custom systems. Wave Flow brings the same pattern to your Kubernetes cluster as a deployable, configurable solution.

Autoscaling handles capacity. Traffic shaping handles priority. Together, they close the gap that neither can solve alone.

Take Kubernetes Traffic Management to the Next Level

Ready to add priority-based traffic intelligence to your Kubernetes stack?

Explore:

• Wave Flow — Priority-based traffic shaping for Kubernetes
• Contact us — See Wave Flow in action

Wave Autoscale is developed by STCLab, a CNCF Silver Member and AWS EKS Service Ready Partner.