agent's json parser failed on truncated update catalog

TL;DR

Catalog data is the foundation of planning, so its integrity rules must be strict: verify completeness before parsing (byte counts, checksums where the source offers them, or a streaming parse that must reach the document end token), reject incomplete catalogs outright, keep the last known-good catalog as the planning basis, and fetch large catalogs with resumable or paged methods rather than one fragile transfer. A partial catalog must never silently become the plan source, because the missing tail is indistinguishable from updates that do not exist.

agent's json parser failed on truncated update catalog

Use this when

• Catalog parsing throws unexpected-end errors on the largest syncs.
• After some syncs the plan contains noticeably fewer updates with no error recorded.
• The catalog comes from a single large file download or API response.

Not for this skill when

• The catalog parses fully but a new field breaks mapping. That is schema drift, handled by parser updates and fixtures.
• The source itself published a smaller catalog (a vendor correction). Counts alone do not prove truncation; verify against the source metadata.

Steps

1. Capture a failing catalog transfer with its expected size or checksum from the source and the received size. Verify: the shortfall is proven numerically
2. Enforce an integrity gate at ingest: size or checksum match, then a full parse that must reach the end of the document; any failure marks the catalog invalid and it is quarantined, not used. Verify: the truncated fixture is quarantined in a test
3. Change planning to read only validated catalogs, falling back to the last known-good with its age displayed in the plan header. Verify: a bad sync day still produces a plan, clearly labeled with catalog age
4. Move the fetch to a resumable or chunked method (range requests, paged API, or the vendor delta sync) sized to the path reliability. Verify: the largest catalog transfers complete intact across repeated trials
5. Add catalog metrics per sync: bytes, update count, validation result, and comparison with the previous catalog. Verify: anomalous count drops alert before planning consumes them

Variant phrasings

truncated update catalog json parser integrity check

The main phrasing.

partial catalog silently fewer updates planned danger

The silent-corruption phrasing.

last known good catalog fallback staleness note planning

The resilience phrasing.

Compatibility: Compatibility: WSUS catalog exports, vendor advisory feeds, and any large JSON catalog source; delta sync options vary by vendor.

Why it happens

Truncation is dangerous in proportion to how foundational the data is. A truncated log costs a few records; a truncated catalog quietly rewrites the agent picture of what updates exist, and every downstream decision inherits the lie. The last-known-good pattern is the standard defense for reference data URIs freshness is desirable, but a slightly stale truth beats a fresh falsehood every time, provided the staleness is visible wherever the data is used.

Edge cases / pitfalls

• Delta catalogs need chain validation (each delta applied to the right base) in addition to per-file integrity.
• If the source offers no checksum, record counts and a hash you compute at ingest so later comparisons have a baseline.
• Quarantined catalogs are diagnostic treasure; keep them with the failure record instead of deleting them in cleanup.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_3x3lPm8YwOly7DIkLKwYvw