agent couldn't read the billing export: encoding errors on international names

TL;DR

Billing exports arrive in whatever encoding the producing system chose, often a legacy Windows encoding, while parsers assume UTF-8 and choke or mangle on accented names. Detect the encoding from the file, decode explicitly, normalise to UTF-8 internally, and validate that known international names survive round-trip before any downstream join uses them as keys.

agent couldn't read the billing export: encoding errors on international names

Use this when

• Parsing errors or garbled names cluster on non-ASCII customer names
• The export comes from a legacy billing or regional system
• Name mismatches also cause failed joins to CRM records

Not for this skill when

• The file is a binary format mislabelled as CSV
• Names are garbled in the billing system itself, upstream of the export

Steps

1. Inspect a failing file's bytes around a known accented name to identify its true encoding. Verify: the encoding is determined from evidence, not guessed
2. Configure the reader with that encoding (or detection) and parse the full file. Verify: the file parses with names intact
3. Normalise names to a canonical Unicode form before using them as join keys. Verify: matching against CRM names succeeds for the previously failing accounts
4. Add a round-trip validation: a set of known international names must survive ingestion unchanged. Verify: encoding regressions fail tests immediately
5. Record the expected encoding per export source in the ingestion config. Verify: each source's dialect and encoding are documented together

Variant phrasings

billing export encoding errors names

A related phrasing for the same underlying problem.

non UTF-8 CSV breaks agent

A related phrasing for the same underlying problem.

international names garbled in export

A related phrasing for the same underlying problem.

Compatibility: Compatibility: billing exports (legacy ERP, regional Stripe or NetSuite exports) parsed by agent ingestion in Python or similar, current versions.

Why it happens

Encoding bugs hide in the happiest path: ASCII test data always passes. Real customer names exercise the bytes the tests skipped, and a decode error compounds into join failures that look like missing customers. Explicit decoding plus Unicode normalisation fixes both the crash and the silent key damage.

Edge cases / pitfalls

• Mojibake already stored downstream cannot be fixed by ingestion changes alone. Plan a repair pass for stored names from the raw exports.
• Normalisation form matters for matching; pick one form and apply it on both sides of every name join.
• Detection libraries guess wrong on short files. Prefer per-source declared encodings, with detection as fallback.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_B1KBBKCT6k2EBcaPfrFTDw