agent declared the integration done after one 200 - it never tested error paths and 404 handling crashed in prod

TL;DR

Write tests for every error path the API documents - 404, 429, 422, 500 - and make the client handle each without crashing. One 200 proves the happy path exists; it says nothing about the paths production will actually hit.

Define done as: happy path plus every documented error path tested, not as the first green response.

Verbatim query

agent declared the integration done after one 200  -  it never tested error paths and 404 handling crashed in prod

Steps

1. List the error paths that matter Read the API docs for every error code the endpoints you use can return. Add the operational ones: 429 rate limit, 503 maintenance, timeouts. Expected: You have a written list of error cases to cover.

2. Trigger each error on purpose Hit a bogus ID for 404, send a bad field for 422, spam a cheap endpoint for 429. Use the sandbox if the provider offers error simulation. Expected: You can reproduce each error on demand.

3. Watch the client crash Run each error through the generated client and record what breaks. Note whether it throws, hangs, or returns garbage. Expected: Every error path has a documented current behavior, including the crashes.

4. Implement handling per error 404: return a not-found result, not an exception. 429: back off and retry. 422: surface the field error. 500: retry with a limit then fail cleanly. Timeouts: set explicit timeouts and treat them like 503s. Expected: Each error case has handling that completes without crashing.

5. Lock it in with tests Write a test per error path that asserts the handling, not just no-crash. Run the suite on every regen and every dependency bump. Expected: The full error-path suite passes and would catch a regression.

Use this when

• The integration works in demos but crashes in production on 404s or 429s
• An agent shipped after a single successful call with no error tests
• Error responses from the API surface as unhandled exceptions

Not for this skill when

• The happy path itself fails - fix that before writing error-path tests
• Error handling exists but behaves wrong - that is a logic bug in existing code
• Crashes come from your infrastructure, not API error responses - check your own stack first

Variant phrasings

never tested error paths

The first 200 is the start of integration testing, not the end of it.

404 handling crashed in prod

Missing-record is a normal API response, not an exceptional one; clients must handle it as routine.

integration declared done after one 200

Done means every documented error path is exercised, not one happy path.

Why it happens

Agents optimize for the visible win: a 200 in the terminal looks like success, so they stop. But production is mostly error paths - stale IDs, rate limits, provider outages. Untested error handling does not degrade gracefully; it throws, hangs, or corrupts state. The fix is a definition of done that includes errors, because the API will include them whether you tested them or not.

Edge cases

• Some errors need different handling per endpoint - a 404 on a lookup is routine, on a create it is a bug
• Retry storms are a real risk: cap retries and add jitter so your error handling does not become a DDoS
• Log the raw error body on unexpected codes - the next undocumented error is already on its way

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_cRoqhUk5c5fIQWZZZbDAiQ