agent context window overflow on full masscan output caused the scan summary to fail

TL;DR

Full masscan output is far too large for any context window, so the fix is to never put it there: stream-parse the output file into a compact host-to-ports summary on disk, and give the agent only the summary. Keep the raw output on disk for drill-down. Summarization reads files; it does not ingest them.

agent context window overflow on full masscan output caused the scan summary to fail

Use this when

• Masscan finished and wrote its output file, but the agent's summary step failed or produced a truncated summary.
• The agent tried to read the raw output file into context.
• You run internet-scale or large-subnet scans under agent control.

Not for this skill when

• Masscan itself failed or wrote no output. That is a scan issue.
• The output is small enough to fit comfortably. Then just read it.
• You are scanning ranges outside your authorized engagement. Stop.

Steps

1. Confirm the size mismatch: check the output file size against the agent's context budget. Verify: the file is orders of magnitude larger than what the agent tried to read.
2. Write a streaming parser that reads the masscan output line by line and emits a compact summary: one line per host with its open ports. Verify: the summary file is small enough to fit in context with room to spare.
3. Have the agent summarize from the compact file only. Verify: the summary covers every host in the compact file (count check).
4. Keep the raw output on disk and teach the agent a drill-down pattern: grep the raw file for a specific host when it needs detail. Verify: the agent can answer a detail question about one host without reading the whole file.
5. Add the pattern to the agent's scan playbook: raw output stays on disk, summaries go to context. Verify: the next masscan run follows the pattern without being told.

Variant phrasings

masscan output too large for agent context

Common phrasing.

summarize large scan output without loading it

Workflow phrasing.

agent chokes on full port scan results

Symptom-level search.

Compatibility: masscan text output. The stream-and-summarize pattern works for any large scanner output.

Why it happens

Agents default to reading files whole, and masscan's whole output for a large range is enormous. The summary step then fails on the context limit, and the agent either truncates silently (losing hosts) or errors out (losing everything).

Edge cases / pitfalls

• Binary masscan output must be converted to text first; do not try to stream-parse the binary format directly.
• The compact summary should keep port states, not just port numbers. A ports-only list without states misleads the next stage.
• If the summary is still too large, aggregate further: count hosts per open port, and list only the interesting outliers in full.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_HfAvc-qmhzWMdIRiu8SVIg