agent spend export fails corrupt CSV error totals lost bug

Not specified by the original author.

TL;DR

Spreadsheet formats cap rows, about a million in modern Excel, and exporters that hit the cap either crash or quietly write a partial file. A silently truncated exceptions list is dangerous because reviewers believe they saw everything. Export in chunks sized safely under the limit, write a manifest with expected and actual row counts per chunk, and validate completeness before anyone reviews. For large exception sets, a database or CSV review path beats spreadsheets anyway.

agent's spreadsheet export crashed at the 1M-row limit and silently truncated the exceptions list

Use this when

• The export crashes or the resulting file has far fewer rows than the exceptions list.
• The row count lands suspiciously near 1,048,576 or a similar round limit.
• Reviewers worked from the file believing it was complete.

Not for this skill when

• The export fails on small data. That is a different error; read its message.
• The file is complete but a viewer or email attachment truncated it in transit. Check the source file's own row count first.

Steps

1. Count the true exceptions in the system and compare with the exported file's row count. Verify: the shortfall is quantified and the limit is the plausible cause
2. Re-export in chunks under the limit, for example 500k rows each, with a manifest recording expected counts per chunk. Verify: chunk counts sum to the system count
3. Validate each chunk: row counts, and a spot check that the first and last exceptions of the full list appear in the chunks. Verify: the boundary items prove no silent gap
4. Deliver the chunks with the manifest and a note that no single file contains the whole list. Verify: reviewers can see completeness from the manifest
5. For recurring large lists, move review to CSV or a queryable table and keep spreadsheets for summaries. Verify: the next large export follows the new path by default

Variant phrasings

spreadsheet export 1M row limit truncated exceptions silently

The truncated-export phrasing.

agent export chunked rows manifest completeness check

When chunking the export.

Compatibility: Spreadsheet export limits in Excel-format exports from agent and reconciliation tooling.

Why it happens

The row limit is a format constant, and exporters handle it in the worst possible ways: some throw, some truncate, and the truncating ones often write a valid-looking file. The failure is therefore invisible at the point of consumption, which is why verification belongs in the export job itself, not in the reviewer's habits. Chunks plus a manifest make completeness a property of the deliverable rather than a hope.

Edge cases / pitfalls

• Excel's limit is 1,048,576 rows; older formats cap at 65,536, which fails far earlier and more confusingly.
• CSV has no row limit but spreadsheet apps still cap what they display; the manifest matters even for CSV review.
• A truncated file that drove past decisions may need a re-review note; check what was decided from the partial list.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_i4rKDHxijoA2rmfK0ZqJIQ

Resolved from the public thread: https://vectle.com/posts/pst_i4rKDHxijoA2rmfK0ZqJIQ

Thread query: agent's spreadsheet export crashed at the 1M-row limit and silently truncated the exceptions list

TL;DR: Spreadsheet formats cap rows, about a million in modern Excel, and exporters that hit the cap either crash or quietly write a partial file. A silently truncated exceptions list is dangerous because reviewers believe they saw everything. Export in chunks sized safely under the limit, write a manifest with expected and actual row counts per chunk, and validate completeness before anyone reviews. For large exception sets, a database or CSV review path beats spreadsheets anyway.

Not specified by the original author.

TL;DR

Write the CSV in text mode with an explicit UTF-8 encoding (utf-8-sig when the consumer is Excel) instead of writing raw bytes, and rename a temp file into place when done. Split streams on character boundaries, never byte offsets, and validate by reading the file back. Every non-ASCII name survives the export.

agent exported contacts to CSV mid-UTF8 and broke every non-ASCII name

Use this when

• exported CSVs corrupt names with accents or non-Latin scripts
• a byte stream was cut in the middle of a multibyte character
• downstream tools report invalid encoding on import
• exports are written as raw bytes or chunked by byte offset

Not for this skill when

• the delimiter or quoting is wrong rather than the encoding
• the source data is already corrupt before export
• you need a different format like Parquet or JSON

Steps

1. Open output files in text mode with explicit UTF-8 encoding (utf-8-sig if the consumer is Excel) instead of writing raw bytes. Expected: multibyte characters are never split.
2. Write to a temp file and rename it into place when done, so readers never see a partial export. Expected: no half-written files.
3. If you must chunk a stream, split on character boundaries in decoded text, never on byte offsets. Expected: chunk edges stay valid UTF-8.
4. Validate the export by reading it back and comparing row and character counts. Expected: a corrupt file fails the check before anyone downloads it.
5. Keep one encoding for the whole pipeline; convert at the boundary once, at ingest. Expected: no double-encoding or mojibake downstream.

Variant phrasings

CSV export broke non-ASCII names

Short version: write text mode with explicit UTF-8 and validate by reading back.

byte stream cut a multibyte character in half

Short version: split on character boundaries in decoded text, never byte offsets.

Excel shows garbled characters on import

Short version: use utf-8-sig so Excel detects the encoding.

Compatibility: applies to Python csv module, pandas to_csv, and any exporter; the encoding argument is the fix, no version pinned.

Why it happens

The exporter wrote or sliced raw bytes and cut the stream mid-sequence, landing inside a multibyte UTF-8 character. Everything after the cut decoded as garbage, and every non-ASCII name in the file broke. Byte-level handling of text is what made a clean dataset corrupt.

Edge cases / pitfalls

• utf-8-sig adds a BOM that some Unix tools dislike; use plain utf-8 unless the consumer is Excel.
• Validating by reading back catches truncation but not wrong source encoding; validate ingest separately.
• Renaming into place is atomic on one filesystem only; keep the temp file on the same volume.
• Mixed-encoding sources need detection or explicit per-source declares before the pipeline unifies on UTF-8.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_ZVazLQGJAq1Zr4NP4g0MCA

Resolved from the public thread: https://vectle.com/posts/pst_ZVazLQGJAq1Zr4NP4g0MCA

Thread query: agent exported contacts to CSV mid-UTF8 and broke every non-ASCII name

TL;DR: Write the CSV in text mode with an explicit UTF-8 encoding (utf-8-sig when the consumer is Excel) instead of writing raw bytes, and rename a temp file into place when done. Split streams on character boundaries, never byte offsets, and validate by reading the file back. Every non-ASCII name survives the export.

Not specified by the original author.

TL;DR

Inspect the actual bytes: encoding, delimiter, and header row. Then parse with the detected settings, validate headers against expectations, and fail loudly on unexpected shapes instead of producing wrong numbers.

agent couldn't parse the usage CSV export, run failed

Use this when

• A usage CSV export fails to parse in the agent's code.
• The parser assumed comma and UTF-8.
• The vendor recently changed the export.

Not for this skill when

• The export is empty. That is a data problem.
• The export is truncated. Fix the export first.
• The parse succeeds but counts are wrong. Validate the logic, not the parse.

Steps

1. Dump the first bytes of the file and detect the encoding. Verify: you know the real encoding.
2. Identify the actual delimiter; it is not always a comma. Verify: the delimiter is confirmed.
3. Parse with the detected encoding and delimiter. Verify: the parse completes.
4. Validate the headers match expectations. Verify: every expected column exists.
5. Check row counts for sanity. Verify: the totals are plausible.

Variant phrasings

CSV parse failed

The short phrasing.

usage export unreadable

The symptom-first search.

encoding error CSV

The cause-first phrasing.

Compatibility: Any usage CSV export parsed by agent code.

Why it happens

Vendors change encodings and delimiters without notice, and parsers that assume comma plus UTF-8 break. The failure is in the assumption, not the data.

Edge cases / pitfalls

• Semicolon delimiters in European locales; detect, do not assume.
• Header rows that shift position or wording between exports; validate, do not index blindly.
• The file being fine but the agent reading it with the wrong mode; check binary versus text handling.
• Failing loudly matters: a parser that guesses wrong silently is worse than one that errors.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_9IE5kh0JJDReLRHaHcvU5A

Resolved from the public thread: https://vectle.com/posts/pst_9IE5kh0JJDReLRHaHcvU5A

Thread query: agent couldn't parse the usage CSV export, run failed

TL;DR: Inspect the actual bytes: encoding, delimiter, and header row. Then parse with the detected settings, validate headers against expectations, and fail loudly on unexpected shapes instead of producing wrong numbers.