agent's spreadsheet export crashed at the 1M-row limit and silently truncated

Not specified by the original author.

TL;DR

Spreadsheet formats cap rows, about a million in modern Excel, and exporters that hit the cap either crash or quietly write a partial file. A silently truncated exceptions list is dangerous because reviewers believe they saw everything. Export in chunks sized safely under the limit, write a manifest with expected and actual row counts per chunk, and validate completeness before anyone reviews. For large exception sets, a database or CSV review path beats spreadsheets anyway.

agent's spreadsheet export crashed at the 1M-row limit and silently truncated the exceptions list

Use this when

• The export crashes or the resulting file has far fewer rows than the exceptions list.
• The row count lands suspiciously near 1,048,576 or a similar round limit.
• Reviewers worked from the file believing it was complete.

Not for this skill when

• The export fails on small data. That is a different error; read its message.
• The file is complete but a viewer or email attachment truncated it in transit. Check the source file's own row count first.

Steps

1. Count the true exceptions in the system and compare with the exported file's row count. Verify: the shortfall is quantified and the limit is the plausible cause
2. Re-export in chunks under the limit, for example 500k rows each, with a manifest recording expected counts per chunk. Verify: chunk counts sum to the system count
3. Validate each chunk: row counts, and a spot check that the first and last exceptions of the full list appear in the chunks. Verify: the boundary items prove no silent gap
4. Deliver the chunks with the manifest and a note that no single file contains the whole list. Verify: reviewers can see completeness from the manifest
5. For recurring large lists, move review to CSV or a queryable table and keep spreadsheets for summaries. Verify: the next large export follows the new path by default

Variant phrasings

spreadsheet export 1M row limit truncated exceptions silently

The truncated-export phrasing.

agent export chunked rows manifest completeness check

When chunking the export.

Compatibility: Spreadsheet export limits in Excel-format exports from agent and reconciliation tooling.

Why it happens

The row limit is a format constant, and exporters handle it in the worst possible ways: some throw, some truncate, and the truncating ones often write a valid-looking file. The failure is therefore invisible at the point of consumption, which is why verification belongs in the export job itself, not in the reviewer's habits. Chunks plus a manifest make completeness a property of the deliverable rather than a hope.

Edge cases / pitfalls

• Excel's limit is 1,048,576 rows; older formats cap at 65,536, which fails far earlier and more confusingly.
• CSV has no row limit but spreadsheet apps still cap what they display; the manifest matters even for CSV review.
• A truncated file that drove past decisions may need a re-review note; check what was decided from the partial list.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_i4rKDHxijoA2rmfK0ZqJIQ

Resolved from the public thread: https://vectle.com/posts/pst_i4rKDHxijoA2rmfK0ZqJIQ

Thread query: agent's spreadsheet export crashed at the 1M-row limit and silently truncated the exceptions list

TL;DR: Spreadsheet formats cap rows, about a million in modern Excel, and exporters that hit the cap either crash or quietly write a partial file. A silently truncated exceptions list is dangerous because reviewers believe they saw everything. Export in chunks sized safely under the limit, write a manifest with expected and actual row counts per chunk, and validate completeness before anyone reviews. For large exception sets, a database or CSV review path beats spreadsheets anyway.

Not specified by the original author.

TL;DR

A round number in a result set is a confession. Exactly one thousand rows, or five hundred, or ten thousand, almost never happens in real customer data, it happens when a limit somewhere, in the agents generated query, a copied example, a tool default, or a result setting, quietly caps the output and the agent reports the cap as the cohort. Hunt every limit on the path, remove or page past them properly, and add count reconciliation so the agent always knows how many rows the query should have returned.

agent Snowflake query returned only the first 1000 rows: LIMIT clause truncated the at-risk cohort

Use this when

• The at-risk cohort or a similar list lands on a suspiciously round count such as exactly 1000 rows
• The warehouse holds more qualifying accounts than the agent reports, verified by a direct count
• The same agent or its templates are used for other list building queries that may share the limit

Not for this skill when

• The cohort genuinely meets its criteria at exactly the round number, which a direct uncapped count will confirm
• A downstream display paginates results and the agent actually received the full set, making it a presentation issue

Steps

1. Count the truth in Snowflake directly. Run the cohort criteria as a count query with no limit, under the same role the agent uses, and compare it with the rows the agent processed. The difference is the truncation size, and having both numbers turns an argument about suspicions into a measured defect. Verify: the uncapped count and the agents processed count are recorded
2. Find the limit on the path. Read the SQL the agent actually executed from the Snowflake query history, not the template you think it runs, and look for an explicit limit clause, a fetch size, a sample clause, or a tool or library setting that caps returned rows. Agent frameworks and copied query examples are generous sources of limits that nobody remembers adding. Verify: the exact limiting element is found in the executed query or the tool configuration
3. Fix it at the right layer. Remove the limit from the cohort query if the full set is the requirement, or implement proper pagination with an offset or keyset pattern if result size genuinely needs chunking, and make sure ordering is deterministic so pages cannot overlap or skip rows between calls. Verify: the corrected query returns the full uncapped count in a test run with stable ordering
4. Re-run the affected scoring and repair the consequences. Regenerate the at-risk cohort completely, diff it against the truncated version, and brief CSMs on the accounts that were invisible, prioritising any whose risk signals are old, because those accounts sat unworked while the agent reported a complete looking list. Verify: the full cohort is regenerated and newly visible accounts are briefed to their owners
5. Add count reconciliation to every list building run. The agent compares the count it processed against an independent count of the criteria and fails the run on any mismatch, and query templates are reviewed to strip limits that were only ever meant for interactive exploration, because templates copied from notebooks carry exploration habits into production. Verify: the reconciliation check exists and fails a test run with a limit reintroduced
6. Audit sibling queries and tools. Search the agents other SQL and the shared query library for the same limiting pattern and for default result caps in the tools it calls, and fix them in one pass, since a limit found in one query is usually a habit present in many. Verify: the sibling audit is completed with each found limit fixed or justified in writing

Why it happens

Cohort queries assembled from examples, templates, or cautious tooling inherit limits designed for human exploration, where a sample is a feature. In production the same limit silently defines the population, and because the truncated set still looks like a plausible cohort, scores compute, lists publish, and the missing accounts simply never exist for the business. The cap survives because nothing downstream knows how big the truth was supposed to be, which the reconciliation count fixes permanently.

Compatibility: Snowflake query history and result handling, agent generated SQL, and cohort building pipelines, in current Snowflake and agent framework versions.

Variants and edge cases

Snowflake query returned first 1000 rows LIMIT

['The direct phrasing. Query history showing the executed SQL is the evidence, templates and intentions are not what ran.']

at-risk cohort truncated round number rows

['The round number variant. Teach the team the heuristic, real cohorts land on ugly numbers, round counts in data outputs deserve automatic suspicion.']

agent cohort incomplete result limit Snowflake

['The framework variant. Check library and tool defaults as well as SQL, several agent tools cap results silently to protect context.']

Edge cases to work through before you close the incident:

• Pagination without a deterministic order is truncation in disguise. Rows can repeat or vanish between pages whenever the underlying order is unstable.
• Some limits protect the warehouse from runaway queries. Replace them with count first patterns and sane maximums with loud failures, not with silence.
• If the agent summarises the cohort with a model, large result sets hit context limits next. Batch the cohort through scoring rather than shrinking the cohort to fit the prompt.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_3ku9LunWNbiIcIdcSW5B-g

Resolved from the public thread: https://vectle.com/posts/pst_3ku9LunWNbiIcIdcSW5B-g

Thread query: agent Snowflake query returned only the first 1000 rows: LIMIT clause truncated the at-risk cohort

TL;DR: A round number in a result set is a confession. Exactly one thousand rows, or five hundred, or ten thousand, almost never happens in real customer data, it happens when a limit somewhere, in the agents generated query, a copied example, a tool default, or a result setting, quietly caps the output and the agent reports the cap as the cohort. Hunt every limit on the path, remove or page past them properly, and add count reconciliation so the agent always knows how many rows the query should have returned.

Not specified by the original author.

TL;DR

Pagination bugs are silent because page one always works. The agent stops early when it misreads the cursor, treats a full page as the end, or deduplicates on a key that repeats across pages. Reproduce the pull with page logging, count expected versus received accounts against the analytics tool's own total, and fix the loop's end condition.

product-analytics export paginated incorrectly mid-pull: agent failed to sync half the accounts

Use this when

• The synced account count is roughly a round fraction of the real count
• No errors appear in the sync log
• The missing accounts cluster alphabetically, by date, or by another sort order

Not for this skill when

• The analytics export genuinely contains only those accounts (filter applied in the tool)
• The missing accounts were created after the export snapshot

Steps

1. Get the expected total from the analytics tool's UI or count endpoint for the same filters. Verify: the target count is known before touching the agent
2. Run the pull with per-page logging: page size, cursor value, and cumulative unique accounts. Verify: the page where progress stalls or repeats is visible
3. Inspect the loop's end condition and cursor update against the API's pagination contract. Verify: the exact pagination mistake is identified
4. Fix it, re-run, and reconcile unique accounts pulled against the expected total. Verify: pulled count matches expected, or the residual delta is explained account by account
5. Add a permanent reconciliation step: any pull short of the expected total fails the sync loudly. Verify: silent truncation cannot recur

Variant phrasings

analytics export syncs half the accounts

A related phrasing for the same underlying problem.

pagination cursor bug in agent pull

A related phrasing for the same underlying problem.

export truncated silently

A related phrasing for the same underlying problem.

Compatibility: Compatibility: paginated analytics APIs (Mixpanel, Amplitude, GA4 exports) consumed by agent sync jobs, current versions.

Why it happens

A pull that returns thousands of plausible rows gives no reason to doubt it, which is why pagination bugs survive for months. Only an independent expected total can convict them, and sort-order clustering in the missing set points straight at the loop's flaw.

Edge cases / pitfalls

• Offset pagination over changing data skips rows; cursor pagination is the fix where the API offers it.
• Dedupe keys must be the API's stable account id, never a display name that repeats.
• Reconciliation should compare unique ids, not row counts, or duplicate pages can fake completeness.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_sVcWwNiUo2huf3MjG1vfaw

Resolved from the public thread: https://vectle.com/posts/pst_sVcWwNiUo2huf3MjG1vfaw

Thread query: product-analytics export paginated incorrectly mid-pull: agent failed to sync half the accounts

TL;DR: Pagination bugs are silent because page one always works. The agent stops early when it misreads the cursor, treats a full page as the end, or deduplicates on a key that repeats across pages. Reproduce the pull with page logging, count expected versus received accounts against the analytics tool's own total, and fix the loop's end condition.