# agent forgot JVM warmup in the benchmark - the JIT compiled mid-test and skewed results

## TL;DR

The JVM interprets first and compiles hot code later, so early iterations are slow and later ones are fast. Warm up before measuring: run enough iterations to trigger compilation, then measure the steady state. JMH does this for you; hand-rolled loops usually do not.

```text
agent forgot JVM warmup in the benchmark - the JIT compiled mid-test and skewed results
```

## Steps

1. Detect it: plot per-iteration times; if the first iterations are much slower and it stabilizes, that is warmup, not the workload.
   Expected: The curve shows the classic warmup slope.

2. Add a warmup phase: run the workload for enough iterations (or seconds) to let the JIT compile before starting the timer.
   Expected: The measured window starts after compilation settles.

3. Use JMH for microbenchmarks: it handles warmup, forks, and dead-code elimination correctly.
   Expected: Results are stable across runs.

4. For macro benchmarks, warm the whole path: caches, pools, and connections, not just the JIT.
   Expected: Steady state is genuinely steady.

5. Report warmup separately from measurement in the methodology so reviewers can trust the numbers.
   Expected: The benchmark documents its own warmup.

## When to use

- benchmark iterations get faster over time
- results vary between runs
- a hand-rolled Java benchmark shows suspicious speedups

## When not to use

- the workload is a one-shot startup measurement (then cold performance is the point)
- using JMH already (warmup is handled)
- the JVM is GraalVM native image (no JIT at runtime; different model)

## Tool compatibility

- HotSpot JVM
- JMH

## Variant phrasings

- "JVM warmup forgotten benchmark"
- "JIT compiled mid-test skewed"
- "java benchmark warmup iterations"

## Why it happens

The JVM starts by interpreting bytecode and compiles hot methods with C1 then C2 as they warm up. A benchmark that measures from iteration zero mixes interpreted, C1, and C2 performance into one number. The agent's 'speedup' was often just the JIT finishing its job mid-test.

## Edge cases

- deoptimization can re-trigger compilation later; long benchmarks should watch for it
- tiered compilation means there are two warmup slopes, not one
- warmup iterations must exercise the same code paths as measurement; warming the wrong path measures nothing

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_PUuuqpDjMecMx8-gfrBdaA