## TL;DR
Concurrent agent writers are corrupting the Delta log because two writers commit against the same table version. Use optimistic concurrency: attempt the write, catch the version-conflict error, and retry the transaction with backoff. Never let two writers assume they hold the log exclusively.

## Symptom
An agent running parallel tasks writes to the same Delta table from multiple workers. Commits fail with version-conflict errors, or worse, the transaction log ends up with interleaved partial writes that leave the table in an inconsistent state.

## When to use
- Parallel agent tasks write to the same Delta table
- Commits fail with version-conflict errors
- The log shows interleaved or partial writes from competing workers

## When not to use
- A single writer corrupts the table (thats a logic bug, not concurrency)
- The storage backend itself is faulty (fix the storage layer)
- Writers only ever append to disjoint partitions and never conflict (then isolate the conflict source first)

## Steps

1. Confirm the failure is a version conflict and not something else. The error names the expected vs actual table version:

```text
version conflict: expected version 42, found version 43
```
Expected output: you see the conflict signature. If the error is different (permissions, schema), retries will not help.

2. Wrap every write in retry-on-conflict with backoff:

```python
import time, random
for attempt in range(5):
    try:
        write_to_delta(df)  # your commit goes here
        break
    except VersionConflictError:
        time.sleep(2 ** attempt + random.random())
else:
    raise RuntimeError("delta write conflicts exhausted retries")
```
Expected output: transient conflicts resolve on retry. The backoff with jitter keeps competing writers from retrying in lockstep.

3. Reduce contention by partitioning the writes so workers rarely touch the same files:

```python
df.write.format("delta").mode("append").partitionBy("worker_id").save(path)
```
Expected output: fewer overlapping commits, fewer conflicts. Partitioning by writer is the cheapest contention reducer.

4. For the critical section that must not interleave, serialize it: funnel those writes through a single writer task or a lock:

```python
with distributed_lock("delta-write-" + table):
    write_to_delta(df)
```
Expected output: zero conflicts on the serialized path. Reserve this for small, critical writes; serializing everything defeats parallelism.

## Variant phrasings

### concurrent spark jobs failing on delta commit
Same fix. Optimistic concurrency with retry is the standard pattern; the retry belongs in the job code, not in manual reruns.

### delta table corrupted after parallel agent writes
Stop all writers first, then assess. Check the table history for the last good version and restore from it before resuming with the retry guard in place.

## Why it happens
Delta Lake uses optimistic concurrency: each writer reads the current version, computes its changes, and commits against that version. When two writers do this simultaneously, the second commit finds the version moved and must retry. Agents that fire off parallel writes with no retry logic turn every collision into a failure, and writers that bypass the commit protocol can interleave partial log entries.

## Edge cases
- Retries must re-read the table state before recomputing, or the retry commits stale logic against the new version.
- Blind retries on a hot table can livelock: cap attempts and surface the failure to the orchestrator instead of looping forever.
- Schema changes during concurrent writes conflict differently than data conflicts. Serialize DDL operations; do not retry them like data writes.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_A5FWks8gpjV5wbDHSDS5xg