Greenhouse 401 "Invalid API key" after the Harvest key rotated and the agent's stored credential went stale

Greenhouse 401 "Invalid API key" after the Harvest key rotated

TL;DR

The Harvest API key was rotated and the agent is still sending the old one, so every request 401s. Fix: replace the stored credential with the new key, confirm auth with a cheap read call, and add rotation handling so the next rotation does not break the agent. Stale stored credentials are the cause, not the API.

Error

401 Unauthorized
"Invalid API key"

Steps

1. Verify it is the key and not the code: make a manual GET to /v1/jobs with the currently stored key (Harvest uses HTTP Basic auth with the key as the username and a blank password). Expected: 401 confirms the stored key is dead.

2. Get the new key from whoever rotated it (Greenhouse admin: Configure, then Dev Center, then API Credential Management). Expected: a fresh Harvest API key.

3. Update the secret where the agent reads it (your secret manager or environment store, not hardcoded), then restart or reload the agent so it picks up the new value. Expected: the agent reads the new key on its next call.

4. Confirm with a cheap read: GET /v1/jobs. Expected: 200 and a job list; auth is working.

5. Add rotation resilience: a key-version label, a runbook entry for rotations, and an alert on the first 401 so a human is looped in before the queue piles up. Expected: the next rotation is a planned swap, not an outage.

Use this when

• all Greenhouse Harvest calls start 401ing at once
• a key rotation just happened
• the agent stores the key in config or a vault
• staging works but production fails (different keys per environment)

Not for this skill when

• only some calls 401 (check per-endpoint permissions, not the key)
• the error is 403 on-behalf-of (that is the permissions problem)
• the key works manually but the agent still 401s (the agent is reading a different stored value)

Variant phrasings

• "Greenhouse Harvest 401 after rotation"
• "invalid API key Greenhouse agent"
• "Harvest API key expired"
• "agent stored credential stale Greenhouse"

Why it happens

API keys are long-lived secrets until someone rotates them, and agents typically read the key once from a stored location. Nothing notifies the agent of the rotation, so it keeps presenting the old key. The API cannot distinguish "old key" from "wrong key", hence the generic 401.

Edge cases

• Test vs production keys are separate; rotating one does not rotate the other, so check which environment broke.
• Basic auth needs the key as the username with a blank password; a malformed auth header 401s even with a good key.
• If keys rotate on a schedule, automate the swap instead of documenting it.
• After replacing the key, watch for queued jobs that failed during the outage; replay them.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst0vNJXXuUoupibSy-iFcag