Monotonic settlement for shared durable operation records

When to use

Multiple client retries share one durable operation record keyed by an idempotency key. Attempts can finish in either order: one may observe committed success while another later times out. Naive last-write-wins settlement can overwrite completed with unknown and discard the stored result.

Core rule

Separate canonical operation outcome from per-attempt observations.

• The canonical record holds the authoritative status and immutable committed result.
• Each attempt may log its own terminal observation, but attempt logs must not blindly overwrite the canonical record.

Settlement is monotonic merge, not last-write-wins.

Status lattice

Assign each status an ordinal rank (higher wins on conflict):

1. pending / running — in flight
2. unknown — caller lost the response; effect may or may not have committed
3. failed — committed failure (terminal)
4. completed — committed success with stored result (terminal, absorbing)

Rules:

• Never downgrade from completed or failed because of a weaker observation.
• Always upgrade to completed when durable evidence shows the effect committed, even if unknown was written first.
• Store the result atomically in the same write that transitions to completed; treat it as immutable afterward.

Settlement writes

Use conditional updates (compare-and-set on status rank), for example:

• Write unknown only if current status rank is below unknown.
• Write completed plus result if current status is not already completed.
• If already completed, ignore duplicate success or late timeout writes.

Optional: keep a settled_at or monotonic outcome_version for audit, but do not use wall-clock write time to pick the winner across unequal evidence types.

Order A: success first, timeout second

1. Attempt A commits effect and writes canonical record: completed + result.
2. Attempt B times out and tries to write unknown.
3. Conditional guard rejects the downgrade; canonical record stays completed.
4. Attempt B should read the canonical record (poll or on retry) and return the stored result.

Order B: timeout first, success second

1. Attempt B times out and writes unknown on the canonical record (allowed while not yet completed).
2. Attempt A commits and writes completed + result.
3. Upgrade succeeds because completed outranks unknown.
4. Any caller reading the record sees success; no duplicate effect if execution is idempotent under the same key.

Client contract

• Timeout means response unknown, not failure.
• After timeout, re-read the durable record or retry with the same idempotency key.
• Do not treat a local timeout as permission to issue a new logical operation under a fresh key.

What to verify in tests (not design reasoning)

• Concurrent settlement: success then timeout leaves completed unchanged.
• Concurrent settlement: timeout then success ends at completed with result present.
• Result field never cleared after first successful commit.
• Duplicate retries after completion replay the stored outcome without re-running the effect.