onboarding agent session token expired mid user provisioning run causing sync failure

Fix onboarding agent failing when its session token expires mid provisioning run

TL;DR

The agent's own session token expiring mid-run kills the sync because nothing refreshes it. Add token refresh to the agent's API client so it renews before expiry, and make long runs resume from a checkpoint. The provisioning logic is fine; the agent's credential hygiene is not.

The error

Onboarding agent failed
Session token expired mid user provisioning run. Sync incomplete; 25 users unprocessed.

Use this skill

Agent? Search Vectle for this error:

curl -s "https://vectle.com/api/v1/search?q=onboarding agent session token expired mid user provisioning run causing sync failure"

Fix it

Step 1: Checkpoint the run state

Persist the last processed user before doing anything else.

Expected: A restart resumes instead of redoing work.

Step 2: Add proactive token refresh

Refresh the session token when it is near expiry, not after it dies.

Expected: Long runs never see an expired token.

Step 3: Handle expiry on 401 too

On an auth 401 mid-run, refresh once and retry the failed call.

Expected: Transient expiries recover without failing the run.

Step 4: Resume from the checkpoint

Restart the agent and let it continue from the saved position.

Expected: All users process exactly once.

Step 5: Alert on repeated refresh failures

If refresh itself fails, stop and page a human instead of looping.

Expected: Credential problems surface instead of silently stalling.

When this applies

• Agents die mid-run on expired session tokens
• Long provisioning runs fail partway with auth errors
• You are building long-running provisioning agents

When it doesn't

• The token is invalid from the start (check the initial auth)
• The API rejects valid tokens (check scopes)
• Runs fail for non-auth reasons (different problem)

Compatibility

Session-based APIs generally. Any long-running agent framework.

Variant phrasings

agent session expired mid run

Same failure. Proactive refresh plus checkpoints is the fix.

provisioning agent token timeout

Token lifetimes shorter than the run guarantee this. Refresh or shorten runs.

agent auth expired long running job

Long jobs must treat credentials as renewable resources, not constants.

Why it happens

Session tokens have lifetimes, and provisioning runs can outlast them. Agents that authenticate once at startup and never refresh are guaranteed to die on long runs. The failure looks like an API problem but it is the agent's credential lifecycle.

Edge cases

• Refresh tokens can expire too; handle the full chain
• Concurrent agents sharing one session can invalidate each other's tokens
• Log token age in run diagnostics so expiry is visible before it kills a run

If it still fails

• Reproduce with a minimal run: one user, one file, one step.
• Read the agent's full trace, not just the final error; the failure is usually upstream.
• Check the underlying API or tool directly, outside the agent, to separate agent bugs from service bugs.
• Reduce concurrency to one and see if the failure persists; races hide as flakes.
• If the run is business-critical, add a human checkpoint before the destructive steps.

Prevention

• Checkpoint long runs so any failure resumes instead of restarting.
• Cap and back off every retry loop; unbounded retries are outages waiting to happen.
• Validate inputs at each pipeline stage; fail fast with clear errors.
• Log enough context per step that a timeout is diagnosable without rerunning.
• Give destructive steps a human checkpoint or a dry-run mode.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst__KBLX7WTPa9tu6cAp4rPwA