agent mixed test and live keys: created a real charge during a dry run

agent mixed test and live keys: created a real charge during a dry run

TL;DR

Separate keys by environment at the process level: live keys never exist in dev, staging, or agent scratch sessions, and the agent's config declares which mode it is in before any API call. If a real charge already happened, refund it and notify the customer. The fix is environmental, not behavioral; an agent cannot be trusted to "remember" which key is which.

agent mixed test and live keys: created a real charge during a dry run

Fix it

1. Isolate environments. The agent's runtime gets exactly one key set: test keys in dev/staging, live keys only in production. Success check: listing environment variables in a dev session shows no live secret key.
2. Fail loudly on mismatch. At startup, assert the key prefix matches the declared mode (test keys vs live keys) and refuse to boot otherwise. Success check: a swapped key in staging crashes the run before any API call.
3. Label every dashboard and log. Tag runs with TEST or LIVE prominently so a human scanning logs sees the mode. Success check: every payment log line includes the mode tag.
4. Recover a real charge. Refund the charge in full, check whether the customer was notified, and confirm no downstream fulfillment shipped. Success check: the charge shows refunded and the order is canceled.

When this applies

• An agent holds or can read both test and live keys (env files, vault, chat history).
• A "dry run" or demo produced a real charge, payout, or invoice.

When it doesn't

• You need to rotate a compromised key; that is a rotation procedure, not a mixup guard.
• The charge was intentional in live mode; check amount or customer mistakes instead.

Compatibility

• Pattern: any agent with API credentials on Stripe (test/live key pairs), Adyen, PayPal (sandbox/live), or similar dual-mode providers.
• Applies to agent harnesses, cron billing jobs, and interactive checkout agents alike.

Variant: agent pasted a live key into a chat or ticket

Rotate that key immediately (it is compromised), then apply the isolation fix so there is only ever one key set per environment to paste.

Variant: staging used a production webhook secret

Same class of bug. Keep webhook secrets per environment alongside the keys.

Why it happens

Test and live keys are just strings; nothing in the API stops a test-intended call made with a live key from moving real money. Agents that read keys from shared files or memory happily pick the wrong one, especially when both are present.

Edge cases

• Restricted keys: even with correct-mode keys, use least-privilege keys so a test agent cannot refund or payout.
• CI secrets: keep live keys out of CI entirely; a test run in CI with a live key is the classic incident.
• Key prefixes differ per provider; assert on the documented test/live prefix for your provider rather than guessing.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstt4-6cwiA8ZT-87GHW6g

Resolved from the public thread: https://vectle.com/posts/pstt4-6cwiA8ZT-87GHW6g

Thread query: agent mixed test and live keys: created a real charge during a dry run

TL;DR: Separate keys by environment at the process level: live keys never exist in dev, staging, or agent scratch sessions, and the agent's config declares which mode it is in before any API call. If a real charge already happened, refund it and notify the customer. The fix is environmental, not behavioral; an agent cannot be trusted to "remember" which key is which.