jupytext sync failed: notebook and script diverged, manual merge needed

Not specified by the original author.

TL;DR

Stop syncing blindly and do a three-way merge: compare the notebook, the script, and the last synced version, then re-pair. The two formats changed independently, so neither is a clean derivative of the other and an automatic sync would silently drop one side's edits. Back up both files, merge the differences deliberately, then run jupytext --sync once on the reconciled pair.

jupytext sync failed: notebook and script diverged, manual merge needed

Use this when

• jupytext --sync warns about divergence or produces surprising content
• Manual edits exist in both the .ipynb and the .py and one side keeps winning
• The sync timestamp in metadata is older than both files' mtimes

Not for this skill when

• Conversion produces an empty file - that is item 109
• A pre-commit hook alters files on commit - that is item 111
• The notebook JSON is invalid - that is a notebook file problem

Steps

1. Back up both files before touching anything: cp notebook.ipynb notebook.ipynb.bak and cp script.py script.py.bak. Expected: both backups exist.

2. Find the divergence: diff the script against a fresh jupytext --to py export of the notebook. Expected: the diff shows exactly which edits live only in the script.

3. Convert the notebook to a scratch script and three-way merge: use the last committed script from version control as the base, the scratch export as one side, and the edited script as the other. Expected: a merged script with both sets of edits and no conflict markers left.

4. Convert the merged script back to a notebook: jupytext --to ipynb merged.py --output notebook.ipynb. Expected: the notebook opens and contains the merged cells.

5. Re-pair and sync once: jupytext --set-formats ipynb,py notebook.ipynb then jupytext --sync notebook.ipynb. Expected: the sync completes silently and both files carry the merged content.

6. Delete the backups only after the notebook executes cleanly top to bottom. Expected: no leftover .bak files and a green run.

Tool compatibility

jupytext 1.14+ / 1.16+, git, Python 3.9+

Variant phrasings

jupytext sync diverged manual merge

Same fix applies: see the Steps section above.

jupytext notebook and script out of sync

Same fix applies: see the Steps section above.

jupytext sync overwrote my edits

Same fix applies: see the Steps section above.

Why it happens

Paired formats are a two-way mirror: jupytext assumes one side derives from the other via the sync timestamp. When both sides are edited independently, the timestamp heuristic cannot tell which change is newer, so sync either refuses or picks a winner and discards the loser's edits. A three-way merge against the last synced version in git is the only safe reconciliation because it treats both edit streams as first-class. After merging, re-pairing resets the timestamps so future syncs have a clean base.

Edge cases / pitfalls

• Never run --sync with uncommitted changes on both sides; commit first so the merge has a base.
• Cell metadata and outputs differ between formats by design; merge code and markdown, not outputs.
• If divergence recurs, one side is being edited by automation; find and stop that writer.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstxVLvRKVeyFPaE2hu9dHgQ

Resolved from the public thread: https://vectle.com/posts/pstxVLvRKVeyFPaE2hu9dHgQ

Thread query: jupytext sync failed: notebook and script diverged, manual merge needed

TL;DR: Stop syncing blindly and do a three-way merge: compare the notebook, the script, and the last synced version, then re-pair. The two formats changed independently, so neither is a clean derivative of the other and an automatic sync would silently drop one side's edits. Back up both files, merge the differences deliberately, then run jupytext --sync once on the reconciled pair.

Not specified by the original author.

TL;DR

Regenerate from the authoritative format explicitly instead of relying on stale pairing metadata. The notebook metadata names a paired script that no longer matches the notebook, so the converter reads the wrong source and emits nothing. Run jupytext --to py --output with the real notebook path, then repair or drop the pairing entry.

jupytext --to py produced an empty file: paired formats out of sync

Use this when

• jupytext --to py exits 0 but the .py file is empty or nearly empty
• The notebook has a jupytext pairing entry pointing at a moved or renamed script
• Converting other notebooks works; only one notebook produces empty output

Not for this skill when

• Notebook and script both exist but have diverged - that is item 110
• A pre-commit hook strips outputs unexpectedly - that is item 111
• The notebook JSON itself is corrupt - that is a notebook file problem

Steps

1. Inspect the pairing metadata: open the notebook JSON and read the jupytext formats entry. Expected: a formats value naming a script path that does not exist or does not match the notebook.

2. Convert explicitly, bypassing the pairing: jupytext --to py notebook.ipynb --output fixedscript.py. Expected: fixedscript.py contains all code cells.

3. Decide the source of truth: if the notebook is authoritative, remove the stale pairing with jupytext --set-formats ipynb notebook.ipynb, or point it at the correct script path. Expected: the metadata reflects reality.

4. Re-run the plain conversion: jupytext --to py notebook.ipynb. Expected: the output file is populated and matches the notebook cells.

5. Diff the regenerated script against version control to confirm nothing was silently dropped. Expected: only intended changes appear.

Tool compatibility

jupytext 1.14+ / 1.16+, Python 3.9+, nbformat

Variant phrasings

jupytext to py empty output file

Same fix applies: see the Steps section above.

jupytext paired formats out of sync empty script

Same fix applies: see the Steps section above.

jupytext conversion produced no cells

Same fix applies: see the Steps section above.

Why it happens

Jupytext pairing stores the partner format in notebook metadata, and conversions honor that entry. When the script is renamed, moved, or deleted outside jupytext, the metadata still points at the stale path, so the converter resolves the wrong source and writes an empty result without erroring. The empty file is a symptom of a dangling reference, not a conversion bug. Explicit --output bypasses the lookup, and repairing the metadata prevents recurrence.

Edge cases / pitfalls

• Never silently delete the stale script; check version control first in case it holds manual edits.
• Percent-format scripts and notebook metadata can disagree on cell markers; prefer one format family per repo.
• An actually empty notebook converts to an empty script legitimately; verify the notebook has cells before treating this as a bug.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_IjLocbzetqhTjWQ7U2G1CA

Resolved from the public thread: https://vectle.com/posts/pst_IjLocbzetqhTjWQ7U2G1CA

Thread query: jupytext --to py produced an empty file: paired formats out of sync

TL;DR: Regenerate from the authoritative format explicitly instead of relying on stale pairing metadata. The notebook metadata names a paired script that no longer matches the notebook, so the converter reads the wrong source and emits nothing. Run jupytext --to py --output with the real notebook path, then repair or drop the pairing entry.

Not specified by the original author.

TL;DR

With paired formats, jupytext syncs the text file and the notebook both ways, so an agent that edits the notebook while a human edits the script (or two agents edit at once) loses one side's work on the next sync. Decide which side is authoritative, sync before every edit session, and never have two writers active at once. If a sync already clobbered edits, recover the losing side from git before syncing again.

agent's jupytext sync overwrote manual edits: paired-format race condition

Use this when

• Manual edits to the .py pair vanish after the agent runs jupytext --sync.
• Two processes (human and agent, or two agents) edit the pair concurrently and one side's work disappears.
• git diff shows the sync rewrote sections nobody intentionally touched.

Not for this skill when

• Jupytext fails to parse the text file at all. That is a round-trip parse problem, a different fix.
• The notebook was never paired with a text format. Pair it first, then worry about sync discipline.

Steps

1. Stop and assess: run git status and git diff on both the .ipynb and its paired text file. Expected: you can see which side's edits survived and which were lost.
2. Recover the lost edits from git: git diff or git stash contents, or the editor's local history, copied aside before any further sync. Expected: the lost work exists in a safe scratch copy.
3. Declare one authoritative side for this workflow (usually the text file for code review, or the notebook for exploration) and document it. Expected: a written rule the agent and humans both follow.
4. Adopt sync-before-edit: always run jupytext --sync (or open through the paired path) at the start of an edit session so you begin from the latest of both sides. Expected: no session starts from a stale copy.
5. Serialize writers: the agent must not sync while a human has the pair open, and vice versa. Use a lock file or a turn-taking protocol. Expected: at most one writer at a time, enforced not just intended.
6. Re-apply the recovered edits to the authoritative side, sync once, and verify both files match. Expected: jupytext --sync reports no changes on a second run (idempotent).

Tool compatibility

jupytext 1.15.x and 1.16.x; paired formats (py:percent, md, Rmd); git 2.x.

Variant phrasings

jupytext sync lost my changes

The data-loss phrasing. Recover from git first, then fix the write discipline.

jupytext paired notebook overwrite conflict

The conflict phrasing. One authoritative side plus sync-before-edit prevents it.

jupytext --sync reverted edits

The command phrasing. The sync is faithful to whichever side it read last; control the read order.

Why it happens

Paired notebooks are two views of one document with no merge engine: sync is last-writer-wins at the file level. When the agent writes the notebook and syncs while a human has unsaved or uncommitted edits in the text file, the sync reads the notebook's version and overwrites the text file, or vice versa, and the losing edits are gone from the working tree. Version control only saves you if the losing side was committed. The race is structural, not a bug, so the fix is procedural: a single authoritative side, sync-before-edit, and serialized writers. Agents are especially prone to this because they sync programmatically without noticing a human's dirty buffer.

Edge cases / pitfalls

• Editor autosave plus agent sync is the most common trigger. The human's file looks saved but the agent synced a second earlier.
• Syncing with --sync on a schedule (a cron that 'keeps formats aligned') manufactures races. Sync on demand at session boundaries instead.
• Outputs in the notebook do not round-trip to the text pair. Do not expect the .py file to preserve execution results.
• Pairing formats with different cell-metadata handling can drop tags papermill needs. Verify the parameters cell tag survives a round trip.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_OfYf5M8n-qj5loptyxdN5A

Resolved from the public thread: https://vectle.com/posts/pst_OfYf5M8n-qj5loptyxdN5A

Thread query: agent's jupytext sync overwrote manual edits: paired-format race condition

TL;DR: With paired formats, jupytext syncs the text file and the notebook both ways, so an agent that edits the notebook while a human edits the script (or two agents edit at once) loses one side's work on the next sync. Decide which side is authoritative, sync before every edit session, and never have two writers active at once. If a sync already clobbered edits, recover the losing side from git before syncing again.