← all discussions

cgroup v2 memory.high vs memory.max — is Kubernetes using the right one?

cocoldboot5 hours ago4 replies

The cgroups page now assumes v2 everywhere. Under v2 there's memory.high (throttle/reclaim) and memory.max (OOM kill). Pod memory limits map to memory.max.

MemoryQoS was supposed to use memory.high to give a soft ceiling before the kill. Where does that stand, and does anyone run with it enabled?

Discussion

kekernel_ravi4 hours ago

I ran the MemoryQoS feature gate on a staging cluster for about three months. Some notes:

- memory.high is set to a fraction of the limit (memoryThrottlingFactor, default 0.9). - Workloads with bursty allocation (JVMs during GC, Python with big pandas frames) got *throttled* hard instead of OOM-killed. That sounds better, but in practice latency went through the roof and the pod looked "alive" while being useless. - Workloads with steady memory were unaffected either way.

So it trades a loud failure for a quiet one. For latency-sensitive services I'd rather have the OOM kill and a restart.

Also check your kernel version — some older 5.x kernels had reclaim behavior under memory.high that was quite pathological.

Reply
cocoldboot3 hours ago

"Trades a loud failure for a quiet one" is the most useful summary of this I've read.

Reply
ararunv2 hours ago

Did you watch PSI metrics during this? memory.pressure would show the throttling clearly.

Reply
kekernel_ravi2 hours ago

Yes, some=60%+ on the JVM pods. That's how we noticed.

Reply