Memory Leak Detection
Watches the hourly minimum memory of each container and flags a floor that keeps rising, so you can act before the container is OOMKilled.
Memory grows 50 MB a day, and Kubernetes says nothing until OOMKilled
kubernetes.io, Assign Memory Resources to Containers and Pods, and Resource Management for Pods and Containers. The 50 MB a day is illustrative, from an example in the Wave Memory Leak Detection docs.
A container that allocates more memory than its limit "becomes a candidate for termination", and the kernel enforces memory limits with OOM kills (kubernetes.io). Until that moment, the Pod stays Running, and nothing in its status warns about the rising memory.
A slow leak hides inside the normal sawtooth. Each garbage collection frees memory, but the low point of each cycle keeps rising, a little every day.
OOM kill at the memory limit: kubernetes.io. Hourly minimum and Steady Growth: Wave Memory Leak Detection docs. The curve, 50 MB a day, the limit, and the days are illustrative.
How Wave does it
- Containermemory goes past its limit
- over limitLinux kernelenforces the limit with an OOM kill
- OOM killkubeletrestarts the container with a back-off
- statuskubectl describe podshows OOMKilled after the crash
- Wave Agentmemory per container, with cAdvisor
- gRPCWave Corekeeps the hourly minimum, up to about 5 days
- hourly floorLeak patternsSteady Growth or Trend Based Growth, with a confidence score
- flaggedWave Consolea leak tab and an OOM risk tab
Watch the floor
Wave tracks the hourly minimum memory of each container, over up to about 5 days. Normal use moves up and down; a leak keeps raising that floor.
Name the pattern
Each detection is Steady Growth or Trend Based Growth, with the baseline memory, the recent memory, the increase, and a confidence score. A bubble chart puts the leaks that are both fast and large in one corner.
Check the limit too
The Possible OOM Risk tab lists workloads running close to their memory limit now, leak or not. You set what counts as close in OOM Risk Settings.