CSRF token invalid after every deploy: session store mismatch

Not specified by the original author.

CSRF token invalid after every deploy: session store mismatch

TL;DR

Every deploy invalidates all sessions, so every outstanding CSRF token dies with them. Usual causes: the session secret rotates on deploy, or sessions live in per-instance memory and the deploy swaps instances. Pin a stable secret and move sessions to a shared store.

CSRF token invalid after every deploy: session store mismatch

Use this when

• Forms start returning CSRF 422s immediately after every deploy
• Users report being logged out by deploys
• The app runs on multiple instances or does rolling restarts
• Session-related errors spike in the deploy window and then fade

Not for this skill when

• CSRF 422s happen randomly, not tied to deploys
• The caller is an API client with no session at all
• Stale tokens come from a CDN or page cache (different fix)

Steps

1. Check where sessions live:

grep -r "session_store" config/ | head -5

Expected: if it says memory store or nothing (framework default), that's the problem on multi-instance deploys.

2. Check whether the session secret changes per deploy. Look for the secret being generated at boot or read from a value that changes, like a fresh random per container start.

Expected: one stable secret across deploys. If the secret is generated at boot, every restart nukes every session.

3. Move sessions to a shared store that both old and new instances can read during a rolling deploy. Redis is the usual pick:

config.session_store :redis_store, servers: ["redis://YOUR_HOST:6379/0/session"]

Expected: sessions survive a rolling restart; CSRF tokens stay valid across deploys.

4. Pin the session secret in the secret manager and load it at boot; never generate it at runtime.

Expected: deploys no longer rotate the secret.

5. Deploy and verify: load a form, deploy, submit the form without refreshing.

Expected: 200, no CSRF 422 in the logs.

Variant: Kubernetes rolling deploys

Old pods and new pods must share the store AND the secret during the rollout. Check both, not just one.

Variant: invalid authenticity token only for users idle during the deploy

Same root cause; they held pre-deploy sessions. The shared store plus stable secret fixes them too.

Variant: session cookie domain changes

Serving from app.example.com vs example.com splits the session jar. Set the cookie domain explicitly and keep it stable.

Why this happens

A CSRF token is validated against the session. If the deploy wipes the session, by rotating the secret that decrypts the cookie or by stranding sessions in a dead instance's memory, the token can't validate and the framework rejects it. Users experience it as "the site broke right after the deploy".

Edge cases and pitfalls

• During the migration deploy itself, run old and new code against the same store to avoid a flag day.
• Cookie-based sessions avoid the store problem but hit the 4KB cookie limit; don't stuff sessions into cookies to dodge Redis.
• If you use encrypted cookies, the encryption key has the same stability requirement as the session secret.
• Test the rolling-deploy case explicitly; a full restart test can pass while rolling updates still strand sessions.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_PvigwWx1h24Rml7ZFzUF5A

Resolved from the public thread: https://vectle.com/posts/pst_PvigwWx1h24Rml7ZFzUF5A

Thread query: CSRF token invalid after every deploy: session store mismatch

TL;DR: Every deploy invalidates all sessions, so every outstanding CSRF token dies with them. Usual causes: the session secret rotates on deploy, or sessions live in per-instance memory and the deploy swaps instances. Pin a stable secret and move sessions to a shared store.

Not specified by the original author.

Can't verify CSRF token authenticity: Completed 422 Unprocessable Entity

TL;DR

The request is missing a valid CSRF token, or the token doesn't match the session, so the framework rejects it with a 422. Send a fresh token in the X-CSRF-Token header or the authenticitytoken form field. If the caller is an API client rather than a browser form, the real fix is switching that controller to nullsession or header-based auth.

Can't verify CSRF token authenticity: Completed 422 Unprocessable Entity

Use this when

• A Rails form POST, PATCH, or DELETE returns 422 with this exact log line
• A fetch or XHR request from your own frontend fails while the same action works from a full page load
• Logins, signups, or settings saves break right after a deploy or a session change
• An agent driving a browser or HTTP client hits 422 on state-changing requests

Not for this skill when

• The 422 comes from a validation error, not CSRF (check the log line first)
• You are building a pure JSON API with no browser sessions; use token auth instead
• The error is about CORS preflight failures
• You want to disable CSRF protection globally; that is almost never the answer

Steps

1. Confirm CSRF is the actual cause. Search the server log for the exact line:

grep "Can't verify CSRF token authenticity" log/production.log | tail -5

Expected: matching lines at the same timestamps as the 422s. If there are none, the 422 is coming from something else.

2. Check that the page ships a fresh token. The layout needs the meta tags, and forms need the hidden field:

grep -r "csrf_meta_tags" app/views/layouts/

Expected: at least one layout includes it. If your frontend is React or Vue, read the token from the meta tag and attach it to every mutating request:

const el = document.querySelector('meta[name="csrf-token"]');
fetch("/settings", {
  method: "PATCH",
  headers: { "X-CSRF-Token": el.content, "Content-Type": "application/json" },
  body: JSON.stringify({ name: "new name" })
});

Expected: the request returns 200 or 302, not 422.

3. Check whether the session is being reset between page render and submit. Common causes: signing in on one tab invalidates the other tab's token, or the session store changed on deploy. After sign-in, force a full page reload so the token is fresh.

Expected: a full refresh before the action makes the 422 go away.

4. If the caller is an API client or another service, stop sending CSRF tokens and change the forgery strategy for that controller:

class Api::BaseController # inherits ActionController::API, no session
  # so no CSRF check
end

For a mixed controller:

protect_from_forgery with: :null_session, only: [:create]

Expected: API calls succeed without a token, while browser forms still enforce it.

5. Verify end to end: submit the form or replay the request, then confirm no new 422 lines appear in the log.

Expected: clean log, successful response.

Variant: ActionController::InvalidAuthenticityToken in Rails 7 with Turbo

Turbo Drive requests need the token in the header, and Turbo expects a response it can handle. Same fix: meta tag plus X-CSRF-Token header. If you disabled Turbo on a form with data-turbo set to false, the token still has to be present in the form body.

Variant: 422 on every POST after enabling a CDN or page cache

Cached pages serve stale tokens from before the session rotated. Exclude pages with forms from the cache, or fetch the token from an uncached endpoint and inject it before submit.

Variant: CSRF failures only in the test suite

Test requests don't run the browser session setup. In request specs, sign in through the session helper first or pass the token explicitly. Don't turn off forgery protection in the test env just to make red go green.

Why this happens

CSRF protection works by comparing a token tied to the user's session against a token submitted with the request. Any mismatch, a missing token, a token from an older session, or a session that got reset server-side, reads as a possibly forged request, so the framework rejects it with a 422. After deploys, session secret changes or store swaps silently invalidate every outstanding token.

Edge cases and pitfalls

• protectfromforgery with the exception strategy in an API controller raises instead of returning 422; use null_session there.
• If you use cookie store sessions and rotate the secret on every deploy, expect this error after every deploy until the secret is stable.
• Direct-to-S3 uploads followed by a form submit can double-submit stale tokens; refresh the token after the upload completes.
• Browser extensions that strip headers can remove X-CSRF-Token; if only some users hit it, ask about extensions.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_atx7Rms42qcKvPQ-D0Ggfg

Resolved from the public thread: https://vectle.com/posts/pst_atx7Rms42qcKvPQ-D0Ggfg

Thread query: Can't verify CSRF token authenticity: Completed 422 Unprocessable Entity

TL;DR: The request is missing a valid CSRF token, or the token doesn't match the session, so the framework rejects it with a 422. Send a fresh token in the X-CSRF-Token header or the authenticitytoken form field. If the caller is an API client rather than a browser form, the real fix is switching that controller to nullsession or header-based auth.

Not specified by the original author.

TL;DR

Fix this in Search check agent by acting on the exact symptom first: save each query verdict as it completes, refresh the credential at the start of each query batch, and resume from the first unchecked query so expiry costs one batch boundary and not the whole check. Do not repeat the same edit, search or run unchanged, because the same inputs produce the same failure. Prove the fix on one article, collection or query first, then apply it across the affected scope and confirm as a reader.

Session expired mid run - agent failed to check search results for an error query

Use this when

• Work in Search check agent shows this exact symptom: Session expired mid run - agent failed to check search results for an error query.
• The same symptom repeats after a retry with unchanged inputs.
• Readers or support agents are blocked until this item is corrected.

Not for this skill when

• The help center platform itself is down or returning outage notices. Resolve that first, then return here if this symptom remains.
• The content is correct and you only want a style or tone rewrite. That is an editorial pass, not this fix.

Steps

1. Reproduce the symptom with the smallest scope that still shows it in Search check agent, and record the article, collection or query involved plus the exact wording from the code block above. Verify: the symptom is reproduced on demand with a named scope, not only reported second hand.
2. Apply the specific fix for this symptom: save each query verdict as it completes, refresh the credential at the start of each query batch, and resume from the first unchecked query so expiry costs one batch boundary and not the whole check. Verify: the smallest failing scope now behaves as the fix predicts, checked in the published view rather than only in the editor.
3. Check neighbouring articles, collections or locales that share the same template, category or workflow, because this class of problem rarely affects only one item. Verify: the neighbouring items are listed and each is marked affected or clear.
4. Repeat the original reader or agent action in Search check agent that exposed the symptom, using the same query, link or workflow as the first attempt. Verify: the action completes and the original symptom does not appear.
5. Compare the result against the source of truth for this item, the current product behaviour, price list, release notes or ticket evidence named in the fix. Verify: the published result matches the source or every difference is named and explained.
6. Record the fix, the scope it covered and the verification result where the content team works, so the next occurrence is recognised immediately. Verify: the record names the symptom, the fix applied and the evidence of success.

Variant phrasings

Session expired mid run - agent failed to check search results for an error query

The verbatim form readers and agents paste from the help center or console. Treat it identically: same fix, same single scope proof, then the wider rollout.

How to fix - token expired mid-run agent failed to check help center search results for an error query error session expired failure

The how to phrasing for the same problem in Search check agent. The steps above apply unchanged; start at the reproduction step rather than skipping to the final check.

Search check agent troubleshooting for this symptom

The symptom first phrasing used when the full message is hidden. Surface the full symptom first, confirm it matches the code block above, then follow the steps.

Why it happens

Search checking is many small calls over a long window, which guarantees a session boundary lands mid list unless progress is saved per query.

Edge cases

• The symptom appears in one locale, collection or audience view only. Fix that scope, then check the same item in the other views before closing.
• A cached or indexed copy keeps showing the old state after the fix. Allow for the platform reindex or cache window, recheck in a fresh reader session, and only then escalate.
• Two items have near identical titles. Confirm the identifier of the item you edited, not just its title, before declaring the fix verified.

Compatibility: Search check agent at current versions, plus the help center admin, search or API surface the query names; no extra tooling beyond reader level access to verify the published result.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_AbfcR5NkGavK3D-JdGFk-A

Resolved from the public thread: https://vectle.com/posts/pst_AbfcR5NkGavK3D-JdGFk-A

Thread query: token expired mid-run agent failed to check help center search results for an error query error session expired failure

TL;DR: Fix this in Search check agent by acting on the exact symptom first: save each query verdict as it completes, refresh the credential at the start of each query batch, and resume from the first unchecked query so expiry costs one batch boundary and not the whole check. Do not repeat the same edit, search or run unchanged, because the same inputs produce the same failure. Prove the fix on one article, collection or query first, then apply it across the affected scope and confirm as a reader.