## TL;DR
Pin the sampling temperature to 0, pin the model version, feed the model a canonicalized diff, and derive the verdict from the findings with a deterministic rule instead of asking the model for a free-text verdict. Verdict flips on identical input are a sampling problem, not a judgment problem.

## The problem

```text
review agent gave conflicting verdicts on identical diffs across runs: nondeterminism: how to fix
```

## When to use
- The same diff gets different verdicts across reruns
- A/B runs of the agent disagree with each other
- The verdict flips but the finding list is nearly identical
- Reviewers no longer trust the bot's verdict

## When not to use
- The verdict changed because the diff changed (correct behavior)
- Findings differ because the tool versions changed (pin those too, but it is a different fix)
- The verdict is consistent but wrong (a rubric problem, not nondeterminism)

## Steps

### 1. Pin temperature to 0 and set any seed the provider offers
Set temperature 0 on the review call, and pass a fixed seed parameter if the model API supports one. Expected output: the single biggest source of run-to-run variance is removed.

### 2. Pin the model version string and log it
Use an exact dated model version, not an alias like "latest". Log the version on every run.

Expected output: provider-side model swaps stop silently changing verdict behavior; any flip can be correlated with a version change.

### 3. Canonicalize the diff before feeding it to the model
Sort files, keep hunks in file order, strip timestamps and index lines that vary between fetches. The model must see byte-identical input for identical diffs.

Expected output: input variance is eliminated, so any remaining flip is isolated to sampling.

### 4. Derive the verdict from findings with a deterministic rule
Do not ask the model for a verdict. Instead, map findings to a verdict in code: for example, any high-or-critical finding means request changes, only medium-and-below means comment, no findings means approve.

Expected output: identical finding sets always produce identical verdicts, by construction.

### 5. Log the verdict inputs for replay
Store the finding-list hashes plus the rule version with each verdict. Expected output: any future flip is replayable and diagnosable instead of mysterious.

## Tool compatibility
- Model provider APIs: temperature and seed parameters vary by provider; temperature 0 is universal, seeds are provider-specific.
- git diff: `--no-color` and sorted file lists for canonical input; works with any git version.
- Any review pipeline: the deterministic verdict rule lives outside the model, so it ports across providers.

## Variant phrasings

### review verdict flips between runs on the same PR
Same fix. Check steps 1 and 2 first; most flips trace to temperature or a silent model-version change.

### agent approved then requested changes with no new commits
Confirm the diff was truly identical (step 3). If the diff differed, this is the rebase case, not nondeterminism.

### nondeterministic review outcomes
When temperature 0 still flips, the provider itself is nondeterministic at 0; strengthen step 4 so the verdict never depends on model prose at all.

## Why it happens
Language models sample from a probability distribution, so the same prompt can produce different text on different runs. A free-text verdict amplifies this: tiny wording differences in the findings cascade into different verdict words. Temperature 0 shrinks the variance but providers still change models under aliases, and diff bytes can vary between fetches - each a separate flip source.

## Edge cases
- Providers that stay nondeterministic at temperature 0: lean harder on step 4; the verdict must not depend on sampled prose.
- Rule thresholds need tuning per repo: a strict rule on a noisy repo requests changes on everything; calibrate against reviewer overrides.
- Mixed pipelines (model findings plus scanner findings): apply the deterministic rule to the merged set, not to each source separately.
- A/B testing the agent itself: intentionally vary one input at a time and log which one moved the verdict.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_P-PiRINwTjeGbQ4rVEXcbA