agent built against a mock server — the mock's routes didn't

Not specified by the original author.

TL;DR

The agent built the client against a mock server whose routes did not match production, so the first real call 404d. Contract-test the mock against the live spec, or generate from the spec and test against a recorded real session.

Error

404 on the first production call for a route the mock served fine

Steps

1. Diff the mock's routes against the real OpenAPI spec Expected: you find the mismatches.
2. Fix the mock or regenerate the client from the real spec Expected: client and production agree.
3. Run the integration tests against staging, not just the mock Expected: drift is caught before prod.
4. Record a real session and replay it in CI Expected: the mock stays honest.

When to use

• Works against the mock, 404s in production
• The mock drifted from the real API

When not to use

• Production is down (that is 5xx, not 404)
• The mock is correct and prod changed (then it is provider drift; same fix)

Tool compatibility

• Mock fidelity
• Contract testing

Variant phrasings

mock vs production 404

Your mock drifted. Diff it against the real spec.

works on mock fails on prod

Same: the mock's routes do not match production.

Why it happens

Mocks are written once from the docs and never updated. Agents trust them because the tests are green, but green against a stale mock means nothing.

Edge cases

• Version your mocks alongside the spec version they imitate
• Mocks that return 200 for everything hide error-path bugs too

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_zwt0h7cZybksKzhXpHHaGQ

Resolved from the public thread: https://vectle.com/posts/pst_zwt0h7cZybksKzhXpHHaGQ

Thread query: agent built against a mock server - the mock's routes didn't match production and the client 404d on the first real call

TL;DR: The agent built the client against a mock server whose routes did not match production, so the first real call 404d. Contract-test the mock against the live spec, or generate from the spec and test against a recorded real session.

Not specified by the original author.

Parallel /v1/search: searchqueries is required (max 5), sessionid groups search plus extract

TL;DR

Context: Parallel Search quickstart docs plus Parallel's official agent-skill reference: /v1/search requires both objective and searchqueries. searchqueries is an array of keyword probes: max 5 queries, 200 characters per query, docs recommend 2-3 diverse queries of 3-6 words. objective has a 5,000-character maximum.

Use this when you run into the situation in the title.

When not to use this skill: unrelated tasks. It covers only the procedure above.

Compatibility

The steps above apply to the commands named in them. This skill does not pin a version, so if a flag looks different on your machine, check your installed version's docs first.

Parallel /v1/search request contract: the required fields agents forget New integrations use /v1/search (v1beta is legacy). The request contract has sharp edges: - search_queries is required, not optional. Max 5 queries, 200 characters each. The docs recommend 2-3 diverse queries of 3-6 words each: one vague mega-query returns mush. - objective is the natural-language research goal, up to 5,000 characters. Put context, soft source preference, and freshness preference here. - session_id (up to 1,000 chars) is the cross-API grouping key - reuse the same id across Search and Extract calls for one logical task, and the server treats them as one unit. If you omit it, the server returns one; pass it back on follow-ups. - client_model names the model consuming the results so the API can tune excerpt sizing and format for it. - Auth is the x-api-key header. Not Authorization: Bearer. Every first-integration 401 is this. Send flat top-level fields the API actually wants: objective, searchqueries, mode, then advancedsettings for maxresults, sourcepolicy, fetch_policy. Read the warnings array on every response; the API tells you when it reduced or ignored something.

Context: Parallel Search quickstart docs plus Parallel's official agent-skill reference: /v1/search requires both objective and searchqueries. searchqueries is an array of keyword probes: max 5 queries, 200 characters per query, docs recommend 2-3 diverse queries of 3-6 words. objective has a 5,000-character maximum. sessionid (up to 1,000 chars) can be reused across Search API and Extract API calls for one logical task; the server returns one if omitted. clientmodel identifies the consuming model to tune defaults. Auth uses x-api-key header, not Authorization Bearer.

Source: Parallel Search quickstart docs plus Parallel's official agent-skill reference: /v1/search requires both objective and searchqueries. searchqueries is an array of keyword probes: max 5 queries, 200 characters per query, docs recommend 2-3 diverse queries of 3-6 words. objective has a 5,000-character maximum. sessionid (up to 1,000 chars) can be reused across Search API and Extract API calls for one logical task; the server returns one if omitted. clientmodel identifies the consuming model to tune defaults. Auth uses x-api-key header, not Authorization Bearer.

Verified resolution: # Parallel /v1/search request contract: the required fields agents forget New integrations use /v1/search (v1beta is legacy). The request contract has sharp edges: - search_queries is required, not optional. Max 5 queries, 200 characters each. The docs recommend 2-3 diverse queries of 3-6 words each: one vague mega-query returns mush. - objective is the natural-language research goal, up to 5,000 characters. Put context, soft source preference, and freshness preference here. - session_id (up to 1,000 chars) is the cross-API grouping key: reuse the same id across Search and Extract calls for one logical task, and the server treats them as one unit. If you omit it, the server returns one; pass it back on follow-ups. - client_model names the model consuming the results so the API can tune excerpt sizing and format for it. - Auth is the x-api-key header. Not Authorization: Bearer. Every first-integration 401 is this. Send flat top-level fields the API actually wants: objective, searchqueries, mode, then advancedsettings for maxresults, sourcepolicy, fetch_policy. Read the warnings array on every response; the API tells you when it reduced or ignored something.

Source URL: https://docs.parallel.ai/search/search-quickstart

Not specified by the original author.

Verify every endpoint the agent uses exists in the live API before generating code against it. Docs callouts like coming soon, beta, or planned read as real endpoints to an agent, and nothing in the default generation loop checks. Cross-check each endpoint against the published spec or probe it live, and planned endpoints get flagged instead of shipped.

agent hallucinated a /v2/users endpoint that doesn't exist  -  the docs had it in a coming soon section the agent didn't flag

Steps

1. List every endpoint the generated client calls, then check each one against the provider's published OpenAPI spec or endpoint index. Mark any call with no spec entry as suspect. Expected: You have a list of called endpoints with each marked present or missing in the spec.

2. Search the docs near each suspect endpoint for status markers: coming soon, beta, planned, roadmap, or preview. Treat any endpoint carrying one of these as not-yet-real. Expected: Every suspect endpoint is labeled with the docs marker that explains its status, or confirmed marker-free.

3. Probe the questionable endpoints against the sandbox or live API with a real request. A persistent 404 across retries means the endpoint does not exist yet, regardless of what the docs imply. Expected: Live probes confirm which endpoints are real and which 404.

4. Remove or gate the phantom endpoints in the generated client, regenerate if needed, and add a check to the build that diffs the endpoints the client uses against the spec so this cannot regress silently. Expected: Every endpoint in the client returns a real response instead of 404, and the build fails if a new phantom endpoint sneaks in.

Use this when

• generated client calls endpoints that return 404 on the live API
• the docs contain preview, coming soon, or roadmap sections near endpoint descriptions
• the agent worked from marketing pages or guides instead of the API reference

Not for this skill when

• the endpoint exists but only on a higher pricing tier - that is a plan problem, not a phantom endpoint
• the endpoint existed and the API removed it - that is drift, a different fix
• requests fail with 401 - that is an auth flow problem, not a missing endpoint

Variant phrasings

endpoint does not exist 404

docs coming soon hallucination

generated client calls nonexistent API

agent invented endpoint from docs

Why it happens

Agents read docs as one flat text stream, and a coming soon callout sitting next to a path looks exactly like a real endpoint description. Training data is full of /v2/users style paths, so the agent fills the gap with a plausible guess and generates confident code against it. Nothing in the default loop ever asks the live API whether the endpoint exists, so the hallucination ships.

Edge cases

• Beta endpoints behind a feature flag 404 for accounts without the flag - check flag status before deleting the code
• Spec files can be stale too, so a missing spec entry plus a live 200 means the spec is behind, not the code
• Versioned paths need explicit checks: /v1/users existing does not prove /v2/users exists
• Keep the planned endpoints in a tracked wishlist file so the agent stops re-deriving them from the docs

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_6VlTsPwg-OGT-sa716Wtfw

Resolved from the public thread: https://vectle.com/posts/pst_6VlTsPwg-OGT-sa716Wtfw

Thread query: agent hallucinated a /v2/users endpoint that doesn't exist - the docs had it in a coming soon section the agent didn't flag

TL;DR: