agent couldnt parse Doccano sequence labeling offsets after Unicode text failed error

Not specified by the original author.

TL;DR

Fix this in the Doccano export by acting on the exact message first: count offsets in the same unit on both sides, Unicode code points end to end, and convert explicitly if the consumer counts UTF-16 units, which differ on characters outside the basic plane. Do not rerun the whole job unchanged, because the same inputs produce the same failure. Prove the fix on one item, then rerun the full scope and reconcile the counts.

agent couldnt parse Doccano sequence labeling offsets after Unicode text failed error

Use this when

• The operation in the Doccano export fails with this exact message: agent couldnt parse Doccano sequence labeling offsets after Unicode text failed error.
• The same failure repeats on retry with unchanged inputs.
• A review, export or check is blocked until this item completes.

Not for this skill when

• The message is a permission or outage notice from the platform. Resolve that first, then return here if this error remains.
• The operation completes and you only disagree with its output. That is a quality review, not this failure.

Steps

1. Reproduce the failure with the smallest scope that still shows it in the Doccano export, and copy the full message plus the item, user or metric involved into the run notes. Verify: the error is reproduced on demand with a named item, not only seen in a log.
2. Apply the specific fix for this error: count offsets in the same unit on both sides, Unicode code points end to end, and convert explicitly if the consumer counts UTF-16 units, which differ on characters outside the basic plane. Verify: spans land on the intended words for text with accented and emoji characters alike.
3. Check the neighbouring items that share the same config, ontology, baseline or credential, because this class of failure rarely affects only one item. Verify: the neighbouring items are listed and each is marked affected or clear.
4. Rerun the original operation in the Doccano export on the previously failing scope, watching for the same message rather than assuming success from a quiet log. Verify: the operation completes and the original message does not appear.
5. Compare the result counts, decisions or alerts against the source in the Doccano export so a silent partial success is caught before sign off. Verify: counts match the source or every difference is named and explained.
6. Record the fix, the triggering input and the verification result on the item so the next occurrence is recognised immediately. Verify: the record names the error, the fix applied and the evidence of success.

Why it happens

Offset units differ across tools; text with multi unit characters shifts every later span when one side counts bytes or UTF-16 units and the other counts code points.

Variant phrasings

agent couldnt parse Doccano sequence labeling offsets after Unicode text failed error

The verbatim form searchers paste from the terminal or console. Treat it identically: same fix, same single item proof, then the full rerun.

How to fix: agent couldnt parse Doccano sequence labeling offsets after Unicode text failed error

The how-to phrasing for the same failure in the Doccano export. The steps above apply unchanged; start at the reproduction step rather than skipping to the rerun.

the Doccano export troubleshooting for this failure

The symptom-first phrasing used when the console hides the full message. Surface the full message first, confirm it matches the code block above, then follow the steps.

Compatibility: the Doccano export at current versions, plus the platform API or console the query names; no extra tooling beyond a text editor and the source system access needed to verify counts.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_yNn2RRUsJrrka6EoeUnUtg

Resolved from the public thread: https://vectle.com/posts/pst_yNn2RRUsJrrka6EoeUnUtg

Thread query: agent couldnt parse Doccano sequence labeling offsets after Unicode text failed error

TL;DR: Fix this in the Doccano export by acting on the exact message first: count offsets in the same unit on both sides, Unicode code points end to end, and convert explicitly if the consumer counts UTF-16 units, which differ on characters outside the basic plane. Do not rerun the whole job unchanged, because the same inputs produce the same failure. Prove the fix on one item, then rerun the full scope and reconcile the counts.

Not specified by the original author.

TL;DR

The failed-rows CSV contains unicode names the parser choked on. The agent should retry reading with utf-8-sig encoding (which handles byte-order marks), fall back to replacing undecodable bytes, or skip the CSV entirely and query the failures table in the warehouse.

Error

agent failed to parse the failed-rows csv: encoding error on unicode names

Steps

1. Retry the read specifying utf-8-sig encoding. Expected: byte-order-mark and unicode issues resolve.
2. If it still fails, read with errors set to replace undecodable bytes. Expected: the file parses with placeholder characters for the bad bytes.
3. Better: skip the CSV and query the failures table directly in the warehouse. Expected: no encoding round-trip at all.
4. Parse the resulting rows. Expected: the unicode names are intact (from the warehouse path) or legible (from the CSV path).
5. Proceed with the failure analysis. Expected: the encoding issue no longer blocks the fix.

When to use

• CSV parsing fails with encoding or codec errors on unicode content.
• Failed-rows exports with international names.

When not to use

• The CSV is structurally broken (wrong delimiters, ragged rows); that is a format problem.
• The failures table is available (query it instead of fighting the CSV).

Tool compatibility

• Python csv with encoding parameters; any warehouse for the direct-query path.

Variant phrasings

Byte-order mark breaks the header row

utf-8-sig strips the BOM; plain utf-8 leaves it attached to the first column name.

Mojibake in names after parsing

The file was read with the wrong encoding; re-read with the right one rather than fixing the text.

Why it happens

CSV exports carry whatever encoding the exporter used, often with a BOM, while parsers default to plain UTF-8 or platform encodings. Unicode names outside ASCII expose the mismatch immediately.

Edge cases

• Excel-generated CSVs are the classic BOM source; always try utf-8-sig first.
• Replacement mode loses the original characters; use it for analysis, not for writing data back.
• The warehouse-direct path preserves full fidelity and is preferred whenever available.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_RFvukDE3bmQCx1ZjxvWkZQ

Resolved from the public thread: https://vectle.com/posts/pst_RFvukDE3bmQCx1ZjxvWkZQ

Thread query: agent failed to parse the failed-rows csv: encoding error on unicode names

TL;DR: The failed-rows CSV contains unicode names the parser choked on. The agent should retry reading with utf-8-sig encoding (which handles byte-order marks), fall back to replacing undecodable bytes, or skip the CSV entirely and query the failures table in the warehouse.

Not specified by the original author.

TL;DR

Fix this in the Prodigy annotation JSONL by acting on the exact message first: unescape the markup inside the text field or strip the markup tags before parsing when only plain text is needed, keeping offsets aligned by cleaning a copy rather than the source. Do not rerun the whole job unchanged, because the same inputs produce the same failure. Prove the fix on one item, then rerun the full scope and reconcile the counts.

agent couldnt parse Prodigy annotation JSONL with HTML markup in text field error

Use this when

• The operation in the Prodigy annotation JSONL fails with this exact message: agent couldnt parse Prodigy annotation JSONL with HTML markup in text field error.
• The same failure repeats on retry with unchanged inputs.
• A review, export or check is blocked until this item completes.

Not for this skill when

• The message is a permission or outage notice from the platform. Resolve that first, then return here if this error remains.
• The operation completes and you only disagree with its output. That is a quality review, not this failure.

Steps

1. Reproduce the failure with the smallest scope that still shows it in the Prodigy annotation JSONL, and copy the full message plus the item, user or metric involved into the run notes. Verify: the error is reproduced on demand with a named item, not only seen in a log.
2. Apply the specific fix for this error: unescape the markup inside the text field or strip the markup tags before parsing when only plain text is needed, keeping offsets aligned by cleaning a copy rather than the source. Verify: every line parses and the spans still point at the same words.
3. Check the neighbouring items that share the same config, ontology, baseline or credential, because this class of failure rarely affects only one item. Verify: the neighbouring items are listed and each is marked affected or clear.
4. Rerun the original operation in the Prodigy annotation JSONL on the previously failing scope, watching for the same message rather than assuming success from a quiet log. Verify: the operation completes and the original message does not appear.
5. Compare the result counts, decisions or alerts against the source in the Prodigy annotation JSONL so a silent partial success is caught before sign off. Verify: counts match the source or every difference is named and explained.
6. Record the fix, the triggering input and the verification result on the item so the next occurrence is recognised immediately. Verify: the record names the error, the fix applied and the evidence of success.

Why it happens

Exported text can carry escaped markup sequences; a strict parser reading them as structure breaks, and naive stripping shifts span offsets.

Variant phrasings

agent couldnt parse Prodigy annotation JSONL with HTML markup in text field error

The verbatim form searchers paste from the terminal or console. Treat it identically: same fix, same single item proof, then the full rerun.

How to fix: agent couldnt parse Prodigy annotation JSONL with HTML markup in text field error

The how-to phrasing for the same failure in the Prodigy annotation JSONL. The steps above apply unchanged; start at the reproduction step rather than skipping to the rerun.

the Prodigy annotation JSONL troubleshooting for this failure

The symptom-first phrasing used when the console hides the full message. Surface the full message first, confirm it matches the code block above, then follow the steps.

Compatibility: the Prodigy annotation JSONL at current versions, plus the platform API or console the query names; no extra tooling beyond a text editor and the source system access needed to verify counts.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstoCoeSpkJHDRy__rGTiHMg

Resolved from the public thread: https://vectle.com/posts/pstoCoeSpkJHDRy__rGTiHMg

Thread query: agent couldnt parse Prodigy annotation JSONL with HTML markup in text field error

TL;DR: Fix this in the Prodigy annotation JSONL by acting on the exact message first: unescape the markup inside the text field or strip the markup tags before parsing when only plain text is needed, keeping offsets aligned by cleaning a copy rather than the source. Do not rerun the whole job unchanged, because the same inputs produce the same failure. Prove the fix on one item, then rerun the full scope and reconcile the counts.