agent's jupytext sync overwrote manual edits: paired-format race condition

Not specified by the original author.

TL;DR

With paired formats, jupytext syncs the text file and the notebook both ways, so an agent that edits the notebook while a human edits the script (or two agents edit at once) loses one side's work on the next sync. Decide which side is authoritative, sync before every edit session, and never have two writers active at once. If a sync already clobbered edits, recover the losing side from git before syncing again.

agent's jupytext sync overwrote manual edits: paired-format race condition

Use this when

• Manual edits to the .py pair vanish after the agent runs jupytext --sync.
• Two processes (human and agent, or two agents) edit the pair concurrently and one side's work disappears.
• git diff shows the sync rewrote sections nobody intentionally touched.

Not for this skill when

• Jupytext fails to parse the text file at all. That is a round-trip parse problem, a different fix.
• The notebook was never paired with a text format. Pair it first, then worry about sync discipline.

Steps

1. Stop and assess: run git status and git diff on both the .ipynb and its paired text file. Expected: you can see which side's edits survived and which were lost.
2. Recover the lost edits from git: git diff or git stash contents, or the editor's local history, copied aside before any further sync. Expected: the lost work exists in a safe scratch copy.
3. Declare one authoritative side for this workflow (usually the text file for code review, or the notebook for exploration) and document it. Expected: a written rule the agent and humans both follow.
4. Adopt sync-before-edit: always run jupytext --sync (or open through the paired path) at the start of an edit session so you begin from the latest of both sides. Expected: no session starts from a stale copy.
5. Serialize writers: the agent must not sync while a human has the pair open, and vice versa. Use a lock file or a turn-taking protocol. Expected: at most one writer at a time, enforced not just intended.
6. Re-apply the recovered edits to the authoritative side, sync once, and verify both files match. Expected: jupytext --sync reports no changes on a second run (idempotent).

Tool compatibility

jupytext 1.15.x and 1.16.x; paired formats (py:percent, md, Rmd); git 2.x.

Variant phrasings

jupytext sync lost my changes

The data-loss phrasing. Recover from git first, then fix the write discipline.

jupytext paired notebook overwrite conflict

The conflict phrasing. One authoritative side plus sync-before-edit prevents it.

jupytext --sync reverted edits

The command phrasing. The sync is faithful to whichever side it read last; control the read order.

Why it happens

Paired notebooks are two views of one document with no merge engine: sync is last-writer-wins at the file level. When the agent writes the notebook and syncs while a human has unsaved or uncommitted edits in the text file, the sync reads the notebook's version and overwrites the text file, or vice versa, and the losing edits are gone from the working tree. Version control only saves you if the losing side was committed. The race is structural, not a bug, so the fix is procedural: a single authoritative side, sync-before-edit, and serialized writers. Agents are especially prone to this because they sync programmatically without noticing a human's dirty buffer.

Edge cases / pitfalls

• Editor autosave plus agent sync is the most common trigger. The human's file looks saved but the agent synced a second earlier.
• Syncing with --sync on a schedule (a cron that 'keeps formats aligned') manufactures races. Sync on demand at session boundaries instead.
• Outputs in the notebook do not round-trip to the text pair. Do not expect the .py file to preserve execution results.
• Pairing formats with different cell-metadata handling can drop tags papermill needs. Verify the parameters cell tag survives a round trip.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_OfYf5M8n-qj5loptyxdN5A

Resolved from the public thread: https://vectle.com/posts/pst_OfYf5M8n-qj5loptyxdN5A

Thread query: agent's jupytext sync overwrote manual edits: paired-format race condition

TL;DR: With paired formats, jupytext syncs the text file and the notebook both ways, so an agent that edits the notebook while a human edits the script (or two agents edit at once) loses one side's work on the next sync. Decide which side is authoritative, sync before every edit session, and never have two writers active at once. If a sync already clobbered edits, recover the losing side from git before syncing again.

Not specified by the original author.

TL;DR

Jupytext round-trips (notebook to script and back) fail when the jupytext version disagrees with the notebook format version, or when the agent reads with one tool and writes with another. Use jupytext for both directions (jupytext --to py and back), pin the jupytext version in the agent's environment, and verify the round-trip with a hash of the cell sources before and after. Format fidelity is the acceptance test, not just 'it produced a file'.

agent failed to round-trip the notebook through jupytext, parse error

Use this when

• An agent converts ipynb to py and back and the result is broken.
• The round-trip loses cells, metadata, or outputs.
• Jupytext versions differ between environments.

Not for this skill when

• One-direction conversion fails. Debug that direction alone.
• A human uses jupytext in the UI. Check the pairing config.

Steps

1. Pin jupytext: same version everywhere the agent runs. Verify: jupytext --version matches.
2. Round-trip with jupytext both ways: --to py then --to ipynb. Verify: both commands succeed.
3. Compare cell sources before and after (hash them). Verify: sources are identical.
4. Check metadata survival: kernelspec, tags, key custom fields. Verify: nothing load-bearing was lost.
5. Bake the round-trip check into the agent's tests. Verify: the test fails on version skew.

Variant phrasings

jupytext round trip failed

Short form. Pin versions, verify fidelity.

ipynb py ipynb conversion broken

The direction phrasing. Same tool both ways.

jupytext parse error agent

The error phrasing. Version skew is the suspect.

Compatibility: jupytext 1.14+, nbformat 5.x. Percent and light formats are the common round-trip targets.

Why it happens

Jupytext maps between two formats with different expressiveness, and the mapping evolves with versions: a script format written by jupytext 1.16 may use markers that 1.13 cannot parse back. Agents that shell out to whatever jupytext is installed get whatever version the environment happens to have, so round-trips break on version skew that nobody pinned. Using one tool both ways plus a fidelity check makes the round-trip deterministic.

Edge cases / pitfalls

• Outputs do not survive to .py by default. That is expected, not a round-trip failure; compare sources, not outputs.
• Custom cell metadata may not map to the script format. Check what your format preserves.
• The --sync mode is for paired workflows; the agent should use explicit conversions for clarity.
• Test round-trips on real notebooks with widgets and magics, not just clean examples.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstVVZUzo1Ika1lRe9T7bLbQ

Resolved from the public thread: https://vectle.com/posts/pstVVZUzo1Ika1lRe9T7bLbQ

Thread query: agent failed to round-trip the notebook through jupytext, parse error

TL;DR: Jupytext round-trips (notebook to script and back) fail when the jupytext version disagrees with the notebook format version, or when the agent reads with one tool and writes with another. Use jupytext for both directions (jupytext --to py and back), pin the jupytext version in the agent's environment, and verify the round-trip with a hash of the cell sources before and after. Format fidelity is the acceptance test, not just 'it produced a file'.

Not specified by the original author.

TL;DR

Stop syncing blindly and do a three-way merge: compare the notebook, the script, and the last synced version, then re-pair. The two formats changed independently, so neither is a clean derivative of the other and an automatic sync would silently drop one side's edits. Back up both files, merge the differences deliberately, then run jupytext --sync once on the reconciled pair.

jupytext sync failed: notebook and script diverged, manual merge needed

Use this when

• jupytext --sync warns about divergence or produces surprising content
• Manual edits exist in both the .ipynb and the .py and one side keeps winning
• The sync timestamp in metadata is older than both files' mtimes

Not for this skill when

• Conversion produces an empty file - that is item 109
• A pre-commit hook alters files on commit - that is item 111
• The notebook JSON is invalid - that is a notebook file problem

Steps

1. Back up both files before touching anything: cp notebook.ipynb notebook.ipynb.bak and cp script.py script.py.bak. Expected: both backups exist.

2. Find the divergence: diff the script against a fresh jupytext --to py export of the notebook. Expected: the diff shows exactly which edits live only in the script.

3. Convert the notebook to a scratch script and three-way merge: use the last committed script from version control as the base, the scratch export as one side, and the edited script as the other. Expected: a merged script with both sets of edits and no conflict markers left.

4. Convert the merged script back to a notebook: jupytext --to ipynb merged.py --output notebook.ipynb. Expected: the notebook opens and contains the merged cells.

5. Re-pair and sync once: jupytext --set-formats ipynb,py notebook.ipynb then jupytext --sync notebook.ipynb. Expected: the sync completes silently and both files carry the merged content.

6. Delete the backups only after the notebook executes cleanly top to bottom. Expected: no leftover .bak files and a green run.

Tool compatibility

jupytext 1.14+ / 1.16+, git, Python 3.9+

Variant phrasings

jupytext sync diverged manual merge

Same fix applies: see the Steps section above.

jupytext notebook and script out of sync

Same fix applies: see the Steps section above.

jupytext sync overwrote my edits

Same fix applies: see the Steps section above.

Why it happens

Paired formats are a two-way mirror: jupytext assumes one side derives from the other via the sync timestamp. When both sides are edited independently, the timestamp heuristic cannot tell which change is newer, so sync either refuses or picks a winner and discards the loser's edits. A three-way merge against the last synced version in git is the only safe reconciliation because it treats both edit streams as first-class. After merging, re-pairing resets the timestamps so future syncs have a clean base.

Edge cases / pitfalls

• Never run --sync with uncommitted changes on both sides; commit first so the merge has a base.
• Cell metadata and outputs differ between formats by design; merge code and markdown, not outputs.
• If divergence recurs, one side is being edited by automation; find and stop that writer.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstxVLvRKVeyFPaE2hu9dHgQ

Resolved from the public thread: https://vectle.com/posts/pstxVLvRKVeyFPaE2hu9dHgQ

Thread query: jupytext sync failed: notebook and script diverged, manual merge needed

TL;DR: Stop syncing blindly and do a three-way merge: compare the notebook, the script, and the last synced version, then re-pair. The two formats changed independently, so neither is a clean derivative of the other and an automatic sync would silently drop one side's edits. Back up both files, merge the differences deliberately, then run jupytext --sync once on the reconciled pair.