agent failed to fit finding notes in context limit and dropped critical vulnerabilities

TL;DR

Finding notes do not belong in context: the agent must append each confirmed finding to a findings ledger on disk immediately, with severity, evidence, and tool output reference. Context holds a compact digest only. If the window fills, the ledger is complete and the agent restarts from the digest. A finding that exists only in context does not exist.

agent failed to fit finding notes in context limit and dropped critical vulnerabilities

Use this when

• A finished run's notes are missing vulnerabilities visible in the raw tool output.
• The agent summarized "top findings" and the critical ones are not among them.
• You need a findings record that survives context exhaustion.

Not for this skill when

• The tools found nothing. Then the ledger is correctly empty.
• The agent never confirmed the findings. Unconfirmed tool output is not a finding yet.
• You are testing outside your authorized scope. Stop.

Steps

1. Define the ledger format: one JSON record per finding with title, severity, affected target, evidence excerpt, and the tool output file it came from. Verify: the format is written down before the next run.
2. Make ledger writes immediate: the agent appends the record the moment a finding is confirmed, before moving to the next target. Verify: a mid-run kill leaves every confirmed finding on disk.
3. Keep a digest in context: counts by severity plus one-line titles, rewritten as findings land. Verify: the digest fits in a fraction of the context window.
4. On context exhaustion, restart from the digest and the ledger, not from memory. Verify: the resumed agent's digest matches the ledger exactly.
5. At report time, build from the ledger, and diff the ledger against raw tool outputs as a final completeness check. Verify: no tool-reported issue is missing from the ledger without a written reason.

Variant phrasings

agent lost vulnerability findings context overflow

Common phrasing.

persist agent findings to disk immediately

Workflow phrasing.

critical findings dropped from agent notes

Symptom-level search.

Compatibility: any agent-driven assessment. The ledger pattern is tool-agnostic.

Why it happens

Models compress context under pressure, and compression drops detail that looks redundant, which is exactly what a long list of findings looks like. Critical vulnerabilities get summarized away because the agent was never told that findings are the one thing that must not be compressed.

Edge cases / pitfalls

• Severity assigned at discovery time is provisional. The ledger records it, but the tester reviews severity before the report.
• Duplicate findings from multiple tools should be merged at report time, not at write time. The ledger keeps each tool's record; merging is a separate step.
• Evidence excerpts must not include credentials or session tokens. Scrub before writing.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_VuVa7C2m5AqV-h8nGDrMpQ