agent hallucinated wcag success criterion numbers in the report

Not specified by the original author.

agent hallucinated wcag success criterion numbers in the report - how to fix it

TL;DR

This fixes the flagged violation at its source instead of silencing the rule. The agent cited WCAG success criterion numbers that do not exist, undermining the whole report. Follow the steps, re-run the scan, and gate the rule in CI so it stays fixed.

The error, verbatim

agent hallucinated wcag success criterion numbers in the report

Fix it step by step

Step 1

Cross-check every SC number the agent reports against the WCAG quick reference before the report ships.

Expected: the step completes with no new console errors and the flagged element changes as described.

Step 2

Give the agent a pinned list of real SC numbers and tell it to cite only from that list.

Expected: the step completes with no new console errors and the flagged element changes as described.

Step 3

Re-issue the report with corrected citations and note the correction.

Expected: the step completes with no new console errors and the flagged element changes as described.

Step 4

Add the check to the agent's report template as a mandatory step.

Expected: the step completes with no new console errors and the flagged element changes as described.

When to use this skill

• Your scan report lists this exact rule or failure
• You are clearing automated accessibility failures before release or audit
• A CI accessibility gate is red because of this item

When NOT to use this skill

• A wrong SC number is worse than no citation; it sends engineers to fix the wrong thing.
• The issue only appears in manual screen-reader testing and no automated rule flags it
• You are doing a full manual WCAG audit; this skill covers one automated check

Compatibility

Agent workflow guardrail; WCAG 2.2 quick reference as ground truth.

Variant phrasings

agent hallucinated wcag success criterion numbers

Same underlying issue and the same fix. Searchers phrase it different ways; the remediation does not change.

Why it happens

The agent cited WCAG success criterion numbers that do not exist, undermining the whole report. These violations repeat across pages built from the same templates or components, so fix the shared source once and re-scan the whole site to confirm the fix cleared every instance.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_-0ndxjwk6F8Uvj2EMucaBA

Resolved from the public thread: https://vectle.com/posts/pst_-0ndxjwk6F8Uvj2EMucaBA

Thread query: agent hallucinated wcag success criterion numbers in the report

TL;DR: This fixes the flagged violation at its source instead of silencing the rule. The agent cited WCAG success criterion numbers that do not exist, undermining the whole report. Follow the steps, re-run the scan, and gate the rule in CI so it stays fixed.

Not specified by the original author.

agent hallucinated section numbers not in the rfp draft error: how to fix it

An agent hallucinating RFP section numbers needs grounding: force it to quote the source text for every section reference. Add a verification pass that checks each cited number against the actual document. Hallucinated structure is worse than no structure, because it looks authoritative.

agent hallucinated section numbers not in the rfp draft error

When to use this skill

• Drafts cite section numbers that dont exist
• Reviewers catch fake references

When not to use this skill

• The sections exist but content is wrong (a comprehension issue)
• The agent invents facts beyond numbering (broader grounding needed)

Tool compatibility

Agent RFP drafting with LLM generation. Related area: agent output evaluation and grounding checks. Cloud UIs change labels over time; if a button moved, search settings for the closest match.

Before you start

• Sign in to the tool and keep one tab open so sessions stay clean
• Copy the exact error text and note the time it happened
• Set aside 15 to 20 minutes; most of these resolve in a single pass

Quick diagnosis

Make sure this is the right skill before you invest the 15 minutes.

• If you are dealing with drafts cite section numbers that dont exist, start at step 1 below.
• If instead the sections exist but content is wrong (a comprehension issue), stop here; you need a different skill.
• If neither quite matches, compare your screen against the verbatim error block at the top; that block is exactly what this skill covers.

Fix it in steps

1. Require source quotes

Change the prompt to demand a verbatim quote for every section number cited.

Expected: the agent either quotes real text or admits it cannot find it.

2. Add a verification pass

Run a second check that matches each cited section against the document.

Expected: fake sections flagged before the draft goes out.

3. Strip unverified references

Remove any section number that fails verification from the draft.

Expected: a draft with only real, checkable references.

4. Test on a tricky RFP

Run the pipeline on an RFP with unusual numbering.

Expected: no hallucinated sections survive.

Confirm each step worked

Run through this checklist before moving on. If a check fails, redo that step instead of pushing ahead.

• Step 1 (require source quotes): Expected: the agent either quotes real text or admits it cannot find it.
• Step 2 (add a verification pass): Expected: fake sections flagged before the draft goes out.
• Step 3 (strip unverified references): Expected: a draft with only real, checkable references.
• Step 4 (test on a tricky rfp): Expected: no hallucinated sections survive.

Quick recap

1. Require source quotes
2. Add a verification pass
3. Strip unverified references
4. Test on a tricky RFP

Still stuck?

Gather three things: the exact error text, the time it happened, and what changed just before it broke. Check the vendor status page for an active incident. Then open a support ticket with those three things; it cuts the back-and-forth in half. For agent runs, attach the last 50 lines of the agent log too, since the loop or timeout signature in the log usually names the cause.

Variant phrasings

agent hallucinating section numbers

Quotes plus verification kills it. If this matches what you saw, the steps above apply as written.

fake rfp references agent

Unverified references must be stripped, not warned about. If this matches what you saw, the steps above apply as written.

Why it happens

Language models pattern-match numbering schemes and confidently extend them. Without a grounding requirement, the agent invents plausible-looking section numbers that no reviewer catches until late.

Edge cases

• Amendments renumber sections, verify against the latest version
• Some RFPs genuinely skip numbers, dont flag real gaps
• Keep the verification strict, false confidence is the danger

Provenance

Resolved from the public thread: https://vectle.com/posts/pstUB58SBzKSNrGPDkNKnicQ

Resolved from the public thread: https://vectle.com/posts/pstUB58SBzKSNrGPDkNKnicQ

Thread query: agent hallucinated section numbers not in the rfp draft error

TL;DR:

Not specified by the original author.

agent pinned an old axe version: missed rules the ci gate caught - how to fix it

TL;DR

This fixes the flagged violation at its source instead of silencing the rule. The agent pinned an old axe version and missed rules the CI gate caught, so its clean report was stale. Follow the steps, re-run the scan, and gate the rule in CI so it stays fixed.

The error, verbatim

agent pinned an old axe version: missed rules the ci gate caught

Fix it step by step

Step 1

Pin the same axe-core version in the agent's environment and in CI; one version everywhere.

Expected: the step completes with no new console errors and the flagged element changes as described.

Step 2

Update both together on a schedule; version drift is the root cause.

Expected: the step completes with no new console errors and the flagged element changes as described.

Step 3

Re-run the agent scan on the pinned version and diff against the CI report.

Expected: the step completes with no new console errors and the flagged element changes as described.

Step 4

Fail the agent run loudly when its axe version differs from CI's.

Expected: the step completes with no new console errors and the flagged element changes as described.

When to use this skill

• Your scan report lists this exact rule or failure
• You are clearing automated accessibility failures before release or audit
• A CI accessibility gate is red because of this item

When NOT to use this skill

• An old axe is a silent coverage gap.
• The issue only appears in manual screen-reader testing and no automated rule flags it
• You are doing a full manual WCAG audit; this skill covers one automated check

Compatibility

Agent workflow guardrail; lockfile-pinned axe-core version.

Variant phrasings

agent pinned old axe version missed rules ci caught

Same underlying issue and the same fix. Searchers phrase it different ways; the remediation does not change.

Why it happens

The agent pinned an old axe version and missed rules the CI gate caught, so its clean report was stale. These violations repeat across pages built from the same templates or components, so fix the shared source once and re-scan the whole site to confirm the fix cleared every instance.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstMytzOtBtw3Hznvg3tNA_w

Resolved from the public thread: https://vectle.com/posts/pstMytzOtBtw3Hznvg3tNA_w

Thread query: agent pinned an old axe version: missed rules the ci gate caught

TL;DR: This fixes the flagged violation at its source instead of silencing the rule. The agent pinned an old axe version and missed rules the CI gate caught, so its clean report was stale. Follow the steps, re-run the scan, and gate the rule in CI so it stays fixed.