audit agent unable to parse cypress-axe json output error

audit agent unable to parse cypress-axe json output error - how to fix it

TL;DR

Parse the JSON defensively: cypress-axe output varies by version (violations arrays, callback shapes, terminal tables), so the agent should try JSON.parse, fall back to the callback object, and validate the schema before reading fields. One line of why: the agent breaks when the output shape drifts a minor version, and defensive parsing survives the drift.

The error, verbatim

AuditAgentError: unable to parse cypress-axe JSON output
    detail: Unexpected token 'A' at position 0 (output started with "AssertionError")
    cypress-axe: 1.5.0

Fix it step by step

Step 1: Reproduce the parse failure

node agent/run-audit.js --spec cypress/e2e/a11y.cy.js | rg -i 'parse|json' | head -5

Expected: The parse error repeats when the spec output is not pure JSON.

Step 2: Inspect the raw output

npx cypress run --spec cypress/e2e/a11y.cy.js | head -20

Expected: Shows the output starts with terminal text (AssertionError), not JSON.

Step 3: Add defensive parsing

rg -n 'JSON.parse|violations' agent/reporter.js | head -10

Expected: Add: try JSON first, fall back to structured callback data, validate schema.

Step 4: Re-run the audit

node agent/run-audit.js --spec cypress/e2e/a11y.cy.js | tail -4

Expected: Agent parses whatever shape the output takes and reports violations.

Step 5: Add a regression probe

node agent/run-audit.js --smoke | tail -3

Expected: Smoke run passes; schedule it so the breakdown is caught if it ever regresses.

When to use this skill

• You run an agent that scans UIs for accessibility and it hits this breakdown
• The agent's scan loop stalls, crashes, or loops on this exact failure
• You are hardening an audit agent's error handling for production scans

When NOT to use this skill

• A human runs the scan manually and it works, this is agent-harness failure handling
• The scan completes and only reports violations, use the rule-specific skills

Compatibility

Audit agent consuming cypress-axe output. Prefer the violation callback object over stdout parsing. Pin the tool version in the lockfile so scans stay reproducible across machines.

Variant phrasings

agent cypress-axe output parse error

Same failure, same defensive parsing.

cypress-axe json schema changed

Practitioner phrasing for version drift.

the breakdown hits other routes too

Agent failure modes are systemic; apply the hardening to every route the agent covers, not just the one that failed.

Why it happens

Cypress-axe output is not a stable API: versions change the callback payload, the terminal reporter prints tables instead of JSON, and failures prepend AssertionError text to stdout. Agents that JSON.parse stdout break on all three. The robust pattern is preferring the structured violation callback (an object, not text), validating the schema version field when present, and only falling back to stdout parsing with clear error handling. Pin the cypress-axe version too, so drift is a deliberate upgrade. Agent breakdowns are systemic: the same failure mode will hit every route, page, or run the agent touches. Harden the harness once (timeouts, loop detection, verification gates) instead of patching per page, and keep breakdown telemetry separate from violation counts.

Edge cases

• Pin cypress-axe in the agent's dependencies, unpinned minor bumps cause these breaks.
• The violation callback object is the stable interface, stdout is the fragile one.
• Log the raw output on parse failure, the agent needs the evidence to adapt.
• Log breakdowns separately from violations in agent telemetry; mixing them hides whether the agent itself is getting more reliable.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_5wl4abqoWN51Q9n-uh0Itw