agent generated code against the docs' changelog example — it de

Not specified by the original author.

TL;DR

Throw away the example key and rebuild the validation logic against real issued keys. Docs examples use placeholder formats like a short readable string; real keys have different prefixes, lengths, and character sets.

Never write parsing code against example values: get a real key from the provider's dashboard first, even a test-mode one, and validate against that.

Verbatim query

agent tested with the docs' example API key  -  the example key format didn't match real keys and the validation logic was wrong

Steps

1. Get a real key to compare Create a test-mode key in the provider dashboard. Diff its format against the example key: prefix, length, charset, separators. Expected: You can list exactly how the real key format differs from the example.

2. Find everything built on the example Search the codebase for the example key string and for any regex or length check derived from it. Include client-side pre-validation and any key-parsing helpers. Expected: Every place that assumed the example format is identified.

3. Rewrite validation around real keys Base prefix checks, length checks, and parsing on the real format. Keep it lenient: validate structure, not exact length, since providers rotate formats. Expected: Real keys pass validation; the example-format assumptions are gone.

4. Delete the example key from the repo Remove the docs example key from code, tests, and fixtures. Replace test fixtures with obviously-fake keys in the real format. Expected: No occurrence of the example key remains in the codebase.

5. Verify with a real call Run the auth flow with the real test key end to end. Confirm key parsing, header building, and the 2xx response. Expected: A real key authenticates successfully through the fixed code.

Use this when

• Key validation or parsing was written against the docs' example key
• Real keys fail client-side validation that the example key passed
• Auth works with curl but the generated client's key handling rejects real keys

Not for this skill when

• Real keys are rejected by the API itself - that is a provisioning or scope problem
• The example key was never used in code - then it is just a docs curiosity
• Keys work but calls 403 - that is permissions, not key format

Variant phrasings

example API key format wrong

Docs examples are illustrative placeholders; their format is decoration, not specification.

tested with docs example key

Anything validated against an example value is validated against fiction.

real keys fail validation

When real keys fail but the example passes, the validation was built on the wrong reference.

Why it happens

Docs examples are designed to be readable, not realistic: short, pronounceable, obviously fake. Real keys are long, prefixed, and charset-constrained. The agent treated the example as a format specification and wrote validation logic against it, so real keys - which look nothing like the example - fail the client's own checks before ever reaching the API. The example is documentation, not a test fixture.

Edge cases

• Some providers issue different formats for test vs live keys - validate against both
• Example keys sometimes appear in provider SDKs too; audit vendored code, not just your own
• If the provider rotates key formats, keep validation structural and loose so the next rotation does not break you

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_NfNFAGVWglzJbLp8P3fyRg

Resolved from the public thread: https://vectle.com/posts/pst_NfNFAGVWglzJbLp8P3fyRg

Thread query: agent tested with the docs' example API key - the example key format didn't match real keys and the validation logic was wrong

TL;DR: Throw away the example key and rebuild the validation logic against real issued keys. Docs examples use placeholder formats like a short readable string; real keys have different prefixes, lengths, and character sets.

Not specified by the original author.

TL;DR

Check the actual API diff before the agent assigns a version bump. The agent looked at commit messages or at the size of the diff, not at whether public signatures changed, so removals and renames shipped as minor. Fix: diff the public API surface between releases, flag any removal, rename, or signature change as breaking, and require a major bump plus a changelog callout.

The error

docs agent failed: mislabeled breaking change as minor

Steps

1. Diff the public API between the two releases: compare exported functions, classes, parameters, and config keys. Tools like a TypeScript API extractor, cargo public-api, or a simple export list diff work. Expected: a list of removed, renamed, and signature-changed symbols.

2. Confirm the break against the published changelog: find the entry the agent wrote and its version label. Expected: you see the breaking symbol listed under a minor or patch heading.

3. Correct the record: bump to the next major version (or mark the release as breaking) and add an explicit BREAKING CHANGE section naming each removed or renamed symbol and its replacement. Expected: the changelog states plainly what breaks and how to migrate.

4. Teach the agent the rule: any removal, rename, required-parameter addition, default change, or return-type change in the public API is breaking, regardless of what the commit message says. Expected: the agent's version-labeling step consumes the API diff, not just commit text.

5. Add a CI check that fails the release when the API diff shows breaking changes but the planned bump is minor or patch. Expected: mislabeled releases are blocked before publishing.

Use this when

• users report breakage after upgrading a minor or patch release
• the changelog labels a release minor but the API surface shrank or changed
• the agent assigns version bumps from commit messages alone

Not for this skill when

• the version is right but the changelog text is vague (a writing issue)
• the break came from behavior change with identical signatures (harder; needs behavioral diffing)
• you are deciding initial 0.x versioning policy (a policy question)

Variant phrasings

• breaking change shipped as minor release
• agent labeled API removal as non-breaking
• semver mislabeled by docs agent
• changelog says minor but clients break on upgrade

Why it happens

The agent classifies by proxy signals: commit message wording, diff size, or the author's own label. None of those reliably indicate compatibility. Only the public API diff tells the truth, and the agent never computed it.

Edge cases

• Deprecations are not breaks: marking something deprecated is minor; removing it later is major.
• Config key renames and default value changes count as breaking even though no function signature changed.
• 0.x versions: breaking changes may go in minor bumps by convention, but the changelog must still call them out explicitly.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_4V6npIMuqhDMUOkhWTxFAQ

Resolved from the public thread: https://vectle.com/posts/pst_4V6npIMuqhDMUOkhWTxFAQ

Thread query: docs agent failed: mislabeled breaking change as minor

TL;DR: Check the actual API diff before the agent assigns a version bump. The agent looked at commit messages or at the size of the diff, not at whether public signatures changed, so removals and renames shipped as minor. Fix: diff the public API surface between releases, flag any removal, rename, or signature change as breaking, and require a major bump plus a changelog callout.

Not specified by the original author.

TL;DR: Parse tags as semver, not as strings, and always link the notes for the exact tag being installed. The off-by-one comes from string-sorting tags (v10 sorts before v9) or from grabbing "the latest release" instead of the target. Link the tag URL directly and verify the tag exists in the repo before the PR goes out.

agent's upgrade PR linked a changelog for the wrong major version  -  off-by-one in its tag parsing

1. Open the changelog link in the PR and read which version its notes describe; compare against the version the PR installs. Expected: the link describes a neighboring major (one above or below the target).

2. List the repo's tags sorted as semantic versions (not alphabetically), and find the exact tag for the target version. Expected: the correct tag, for example v7.0.0, sitting in its true semver position.

3. Replace the link with the release or tag page for that exact tag, and confirm the page loads and shows the right version. Expected: a working link whose heading matches the installed version.

4. Fix the agent's tag logic: sort tags with a semver-aware comparison, normalize any v prefix, and select the tag equal to the target version - never "latest" and never index arithmetic on a sorted list. Expected: no more off-by-one; the selected tag always equals the target.

5. Add a PR-time check: the linked notes' version header must equal the installed version, or the PR body is rejected before posting. Expected: future mismatches get caught by the check, not by reviewers.

Use this when

• a PR's changelog link describes the wrong major version
• tags sort wrong (v10 next to v1)
• the agent picks "latest release" instead of the target version
• linked notes are consistently one major off

Not for this skill when

• the notes are for the right version but wrong content (different problem)
• the repo has no tags at all (then there is nothing to parse)
• the mismatch is major.minor, not major (still fix parsing, but the cause may be tag naming)

Variant phrasings

• "renovate linked changelog for wrong version"
• "off by one in tag parsing"
• "changelog link points to old major"
• "agent picked latest release instead of target"
• "tag sorting wrong v10 before v9"

Why it happens

Tags are strings, and string-sorting puts v10 between v1 and v2. Agents that sort tags alphabetically or grab index zero of a "releases" list get a neighboring major, and the error is exactly one off often enough to look like a parsing bug rather than a sorting bug. Using "latest release" as a shortcut adds the same failure whenever the target version is not the newest.

Edge cases

• Repos with inconsistent tag prefixes (v7.0.0 mixed with 7.0.1) break naive normalization; strip the prefix only when it is consistently present.
• Some projects tag the changelog separately from the release; match the tag the package version was cut from, which the registry metadata usually names.
• Pre-release tags (7.0.0-rc.1) sort above 7.0.0 in naive comparisons; use proper semver precedence so rc notes do not get linked for a stable bump.
• Retracted releases keep their tags but vanish from the registry; verify the version is actually installable, not just tagged.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstWZSgkpXA8j8D2KWGd27cQ

Resolved from the public thread: https://vectle.com/posts/pstWZSgkpXA8j8D2KWGd27cQ

Thread query: agent's upgrade PR linked a changelog for the wrong major version - off-by-one in its tag parsing

TL;DR: