agent's API key for the test dashboard leaked into the test log: rotation

TL;DR

A credential in a test log is compromised the moment the log is stored or shared: rotate it immediately, then fix the leak. Scrub the secret from stored logs, find every place the test printed it, and add a guardrail: secrets are referenced, never printed, and log output is scanned for secret patterns before upload. Rotation first, forensics second.

Error

agent's API key for the test dashboard leaked into the test log: rotation

The test printed its dashboard credential in plain text; the log is now in CI artifacts where anyone with read access can see it.

Steps

1. Rotate the credential immediately in the dashboard or provider; do not wait for the investigation. Expected output: the leaked value is dead.
2. Find the leak: locate where the test or agent printed the secret (debug logging, error dump, echoed config). Expected output: the exact line that exposed it.
3. Scrub stored logs: redact the secret from CI artifacts and any copies; if the log cannot be scrubbed, restrict access and note the exposure window. Expected output: no retrievable copy of the live secret.
4. Fix the code: remove the print, use a secret reference from the secret manager, and never interpolate secrets into log strings. Expected output: a test that authenticates without ever holding the secret in a printable form.
5. Add the guardrail: scan test log output for secret patterns before artifact upload, and fail the run if a live secret pattern appears. Expected output: future leaks caught before the log is stored.

When to use

• Any real credential appears in test output or logs.
• An agent's debugging printed config that included secrets.
• You want a standing check that logs are secret-free.

When not to use

• The leaked value is a fake or example credential (scrub it anyway, but rotation is unnecessary).
• The secret was only in memory and never written to a log (verify, then move on).
• You are designing the secret flow fresh (then build it secret-free from the start; this skill is the incident response).

Variant phrasings

Agent echoed the environment to debug a failure

Env dumps are the most common leak vector. Ban full-env logging; log names only.

Error object serialized with headers included

HTTP client errors can carry auth headers. Sanitize error serialization.

Secret in a failed assertion diff

Assertion output on config objects can print secrets. Redact before asserting or compare hashes.

Why it happens

Tests need credentials to run, and debugging prints everything. The agent, chasing a failure, dumps config, env, or error objects that happen to contain the secret. Logs feel ephemeral but are stored, searchable, and broadly readable. The leak is usually one debug line the agent added and forgot.

Edge cases

• Rotation may break other consumers of the same credential; inventory users of the key before rotating, then rotate anyway.
• Scrubbing cannot reach logs already exported to external systems; treat those as exposed and monitor.
• Pattern-based log scanning false-positives on random-looking test data; tune patterns and allowlist known-safe values.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstdWEw5zhWQAhRtVu1maFPw