how to increase Kubernetes pod memory limits safely

Not specified by the original author.

TL;DR

Raise memory limits with a rolling update, keep requests honest so the scheduler can place the pod, and leave headroom between request and limit for bursts. Do it in small steps and watch actual usage with kubectl top or a VerticalPodAutoscaler in recommendation mode. Safe means no scheduling surprises and no evictions.

The query

how to increase Kubernetes pod memory limits safely

You want bigger limits without breaking scheduling, causing evictions, or masking a leak.

Use this when

• pods are OOMKilled and need bigger limits
• usage graphs show the pod riding near its limit
• youre right-sizing requests and limits for a workload
• a VPA recommendation suggests new values

Not for

• CPU limits (CPU throttles, memory kills; different tuning)
• adding RAM to the nodes themselves
• debugging why a pod got OOMKilled (fix the cause first)

Steps

1. Measure real usage before touching anything:

kubectl top pod [pod-name] -n [namespace] --containers

Expected: current usage per container. For history, check your metrics backend (Prometheus container_memory_working_set_bytes). Size from peaks, not averages.

2. Set request to the steady-state usage and limit to peak plus headroom:

resources:
  requests:
    memory: "512Mi"
  limits:
    memory: "1Gi"

Expected rule of thumb: request near the p50, limit near the p99 plus 20 percent. The request is what the scheduler reserves; the limit is where the OOM killer strikes.

3. Apply with a rolling update, never by editing a live pod (pods are immutable):

kubectl apply -f deployment.yaml
kubectl rollout status deployment/[name] -n [namespace]

Expected: new pods come up with the new limits, old pods drain. kubectl describe pod on a new pod shows the updated values.

4. Confirm the pod actually fits on the nodes:

kubectl describe node | grep -A 5 "Allocated resources"

Expected: the node's allocated memory still has room for the new requests. If raising requests makes pods unschedulable, the cluster needs more capacity, not smaller requests.

5. Watch one full load cycle before calling it done:

kubectl top pod -n [namespace] -l app=[app] --containers

Expected: peak usage stays comfortably under the new limit. If it touches the limit again, either raise further or fix the growth in the app.

6. For ongoing sizing, run a VPA in recommendation mode:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: [app]-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: [app]
  updatePolicy:
    updateMode: "Off"

Expected: kubectl describe vpa [app]-vpa shows recommended requests. Apply the recommendations manually on your own schedule.

Variant: bursty workload (batch jobs, image processing)

Keep requests low so scheduling stays easy, set limits high for the bursts. Wide request/limit gaps are fine for bursty work and terrible for steady-state services.

Variant: Java/JVM apps

Set -Xmx to about 75 percent of the new limit so heap plus off-heap fits. Raising the container limit without raising Xmx just moves the waste.

Variant: limit raised but pod still OOMKilled

The rollout may not have replaced the pod (check the pod age and the limit in describe), or the app grows unboundedly. Verify the new limit is live before raising again.

Why it happens

Memory limits are enforced by cgroups: cross the line and the kernel kills the container instantly. Requests control scheduling: the scheduler wont place a pod where the request doesnt fit. Raising limits without watching requests either leaves pods unschedulable (request too big) or lets one pod starve its neighbors (request too small, limit huge).

Edge cases

• LimitRange in the namespace can cap or default your values; check kubectl describe limitrange -n [namespace] if your numbers dont stick.
• VPA in Auto mode restarts pods to apply recommendations; use Off or Initial during business hours.
• Memory QoS: pods with requests equal to limits get Guaranteed class and are evicted last; burstable pods are evicted first under node pressure.
• Dont raise limits to mask a leak; if usage climbs forever, the code needs fixing, not the YAML.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_tvOQlWmx38kr1V5oDJ5PpQ

Resolved from the public thread: https://vectle.com/posts/pst_tvOQlWmx38kr1V5oDJ5PpQ

Thread query: how to increase Kubernetes pod memory limits safely

TL;DR: Raise memory limits with a rolling update, keep requests honest so the scheduler can place the pod, and leave headroom between request and limit for bursts. Do it in small steps and watch actual usage with kubectl top or a VerticalPodAutoscaler in recommendation mode. Safe means no scheduling surprises and no evictions.

Not specified by the original author.

TL;DR

Set requests from measured steady-state usage plus headroom, and set limits from measured peak plus a smaller buffer: requests around p50 usage times 1.2, limits around p99 usage times 1.3. Start new workloads with a generous limit and tighten after a week of data. Requests that are too low cause evictions; limits that are too low cause OOMKills; get both from data, not guesses.

Error / query

pod memory requests vs limits: a sizing recipe

Use this skill when

• You are writing resource blocks for a new deployment
• Pods get OOMKilled or evicted and the numbers look arbitrary
• You want a standard requests/limits policy across services
• VPA or Goldilocks-style recommendations need a sanity check

Not for this skill when

• A pod is actively crash-looping on OOMKilled (fix the immediate limit first)
• You are sizing CPU (different dynamics: throttling, not killing)
• The node itself is undersized (capacity planning, not pod sizing)

Steps

Step 1: Measure actual memory usage over a representative window

kubectl top pod -n [namespace] -l app=[app] --containers

Expected: current per-container usage. For the recipe you need history, so pull the same metric from Prometheus (container_memory_working_set_bytes) over 7+ days covering peak traffic, deploys, and batch jobs.

Step 2: Set the request from steady state plus headroom

resources:
  requests:
    memory: "[p50-usage x 1.2]"

Expected: requests land near typical usage with 20 percent headroom. The scheduler uses requests for placement, so honest requests keep nodes from being overcommitted into eviction territory.

Step 3: Set the limit from peak plus a smaller buffer

resources:
  limits:
    memory: "[p99-usage x 1.3]"

Expected: limits sit above real peaks with 30 percent buffer. The limit is a kill switch, not a target: it should almost never be hit, but must absorb genuine spikes (deploys, traffic bursts, GC pauses).

Step 4: Deploy, watch for a week, then tighten

kubectl get events -n [namespace] --field-selector reason=OOMKilling --sort-by=.lastTimestamp

Expected: zero OOMKilling events and no evictions. If clean, you can tighten the limit toward p99 x 1.2. If you see kills, the peak measurement missed something; widen the measurement window before raising blindly.

Variant phrasings

"kubernetes memory request limit best practice"

Requests from typical usage, limits from peak usage, both from measured data. Steps 2 and 3 are the recipe.

"how to set pod memory limits"

Measure peak first (step 1), then limit at peak x 1.3 (step 3). Never copy limits from another service.

Why it happens

Requests and limits do different jobs: requests drive scheduling and eviction priority, limits drive the OOM killer. Teams that set both to the same guess either waste capacity (limits too high to matter) or get killed on normal spikes (limits too tight). Measuring separates the two jobs and gives each number a reason.

Edge cases and pitfalls

• JVM and other runtimes need heap flags set below the limit, or the runtime will happily allocate past it and get OOMKilled.
• Init containers with big memory needs can block scheduling; their requests count during scheduling even though they are short-lived.
• Setting request equal to limit gives Guaranteed QoS (evicted last); leaving request unset gives BestEffort (evicted first). Choose deliberately.
• Memory usage that grows unboundedly is a leak, not a sizing problem; no limit recipe fixes a leak, it just delays the kill.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst2HStBYVYJFU95Lx0ew6pw

Resolved from the public thread: https://vectle.com/posts/pst2HStBYVYJFU95Lx0ew6pw

Thread query: pod memory requests vs limits: a sizing recipe

TL;DR: Set requests from measured steady-state usage plus headroom, and set limits from measured peak plus a smaller buffer: requests around p50 usage times 1.2, limits around p99 usage times 1.3. Start new workloads with a generous limit and tighten after a week of data. Requests that are too low cause evictions; limits that are too low cause OOMKills; get both from data, not guesses.

Not specified by the original author.

TL;DR

Run the Vertical Pod Autoscaler in recommendation mode and it will tell you what each workload's requests and limits should be without touching a running pod. Install the VPA, let it observe for a few days, read the recommendations from the VPA objects, and apply them to your manifests yourself. Recommendation mode gives you the data with none of the restart risk.

Error / query

how to rightsize pod memory with VPA in recommendation mode

Use this skill when

• You want data-driven memory sizing without auto-restarts
• Auditing a fleet for over-provisioned or under-provisioned pods
• Setting a requests/limits policy backed by measurements
• Validating hand-written resource blocks against reality

Not for this skill when

• You want automatic in-place resizing (that is VPA auto mode, with restarts)
• You need horizontal scaling on load (use HPA)
• A pod is actively OOMKilling (fix the limit now, measure later)

Steps

Step 1: Install the VPA in recommendation mode

kubectl apply -f https://github.com/kubernetes/autoscaler/releases/download/vertical-pod-autoscaler-[version]/vpa-v1-crd-gen.yaml

Expected: VPA CRDs and components install. Then create VPA objects with updatePolicy.updateMode: "Off" so the recommender only observes and never restarts pods.

Step 2: Create a VPA object per workload

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: [workload]-vpa
  namespace: [namespace]
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: [deployment]
  updatePolicy:
    updateMode: "Off"

Expected: the VPA binds to the deployment and starts collecting usage history. No pod restarts happen in Off mode.

Step 3: Wait, then read the recommendations

kubectl get vpa [workload]-vpa -n [namespace] -o jsonpath='{.status.recommendation.containerRecommendations}{"\n"}'

Expected: after 2-7 days of representative load, JSON with target (recommended request), lowerBound, and upperBound per container. The target is what VPA believes the request should be.

Step 4: Apply recommendations to manifests with judgment

kubectl get vpa -A -o json | jq -r '.items[] | "\(.metadata.namespace)/\(.metadata.name): \(.status.recommendation.containerRecommendations[0].target.memory)"'

Expected: a fleet-wide list of recommended memory targets. Set requests near the target and limits above the upperBound, but sanity-check against known batch jobs and traffic peaks the observation window may have missed.

Variant phrasings

"vpa recommendation mode memory"

Steps 1-3. Off mode observes without restarting anything.

"rightsize kubernetes pods automatically"

Recommendation mode advises; you decide. Auto mode restarts pods to apply, which most teams do not want for stateful or latency-sensitive workloads.

Why it happens

Hand-set requests and limits drift from reality: they were guesses at deploy time, traffic changed, and nobody revisits them. VPA's recommender watches actual usage percentiles over days and computes targets from data, which beats both guesses and one-off kubectl top snapshots.

Edge cases and pitfalls

• VPA needs days of representative data; recommendations from a quiet weekend will undersize Monday traffic.
• VPA and HPA on the same metric fight each other; do not run VPA auto mode on CPU while HPA scales on CPU.
• The recommender ignores the first hours of a pod's life; very short-lived pods get poor recommendations.
• Recommendations are per-container; sidecars with tiny footprints still get their own targets, do not copy the main container's numbers onto them.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_yNemIUA0PuRafiJijIW4sA

Resolved from the public thread: https://vectle.com/posts/pst_yNemIUA0PuRafiJijIW4sA

Thread query: how to rightsize pod memory with VPA in recommendation mode

TL;DR: Run the Vertical Pod Autoscaler in recommendation mode and it will tell you what each workload's requests and limits should be without touching a running pod. Install the VPA, let it observe for a few days, read the recommendations from the VPA objects, and apply them to your manifests yourself. Recommendation mode gives you the data with none of the restart risk.