Opaque API token non expiring nullable expiration explicit opt in revocation validation legacy token migration

TL;DR: never let a bare null expiration mean never expires. Store expiration as an explicit policy on the token record (fixed term, never expires, or legacy unknown) next to the nullable expiry time, and allow the never-expires policy only when a recorded opt-in with an owner, a time and a reason exists. Validation fails closed on any inconsistent row: a fixed-term token with no expiry time, or a never-expires token with no opt-in record, is rejected. For legacy tokens created before expiration existed, do not guess: classify each row, give the unknown ones a dated grace deadline to opt in or lapse, and after the deadline they fail closed too. Revocation stays a per-request store check, which is what makes a token with no expiry safe enough to offer at all.

Verbatim problem (the search that created the source thread):

Opaque API token non expiring nullable expiration explicit opt in revocation validation legacy token migration security

Symptoms this fixes: adding a never-expires checkbox quietly makes every token immortal because old rows and omitted fields are also null; an update request that omits expiration, or sends it as null, silently clears a real expiry; nobody can later tell which non-expiring tokens were a deliberate choice and which are accidents; a migration that treats every legacy null as never expires turns years of forgotten tokens into permanent credentials.

Steps

1. Model expiration as a policy, not just a nullable timestamp

Add four fields to the token record alongside the existing hash, scopes and revoked time: an expiration policy field holding one of three values (fixedterm, neverexpires, legacyunknown), a nullable expiry time used only by fixedterm, an opt-in owner and time, and an opt-in reason text. The rule the whole design hangs on: the expiry time being null is never, by itself, what makes a token non-expiring. Only the policy value does, and the policy can only say neverexpires when the opt-in fields are filled. Put a database check constraint or a creation-time assertion behind that rule so no code path, import script or manual edit can produce a neverexpires row with a missing opt-in.

Expected output: every token row states, in its own fields, whether it expires, when, and who chose otherwise. Success check: a query for rows where the policy says never_expires and the opt-in time is null returns zero rows.

2. Default creation to a finite lifetime, and make opt-in a separate explicit act

The creation request takes a lifetime in days and defaults it (for example to 90 days) when the client omits it. Omission must never produce null. Non-expiring is requested with a separate boolean field named neverexpires plus a mandatory reason string, not by sending the expiry as null, because in most JSON stacks an explicit null and an omitted field are hard to tell apart and clients send null by accident all the time. When neverexpires is set, the server ignores any lifetime value, writes the policy as never_expires, stamps the opt-in owner from the authenticated session (never from the request body) and stores the reason. Consider requiring a second factor here that finite tokens do not need: an admin approval for organization tokens, or at least a confirmation step that shows the words never expires. GitHub ships exactly this shape in production: fine-grained tokens may have no expiration, while an organization can cap lifetimes and require administrator approval.

Expected output: a token created by a lazy client gets the default finite expiry; a non-expiring token exists only where someone typed a reason. Success check: create a token sending no expiration fields at all and confirm it has a real expiry time and the fixed_term policy.

3. Guard the update path so null cannot clear an expiry by accident

Partial updates are the quiet leak. An update that changes the name or scopes and happens to serialize the expiry field as null must not clear it. Treat a null expiry in an update as leave unchanged; the only way to move a fixedterm token to neverexpires is the same explicit opt-in act from step 2, and the only way back is setting a new finite expiry, which should also clear the opt-in fields so the record never claims both. Log every policy change as an audit event with the actor, the old policy and the new one.

Expected output: routine edits never change a token lifetime. Success check: rename a fixed-term token with a client that sends nulls for untouched fields and confirm its expiry is unchanged afterwards.

4. Validate in a fixed order, and fail closed on inconsistency

On every request, after the hash matches, check in this order and reject on the first hit: the revoked time is set; the policy is fixedterm and the expiry time is null (inconsistent row, reject and alert, do not treat as non-expiring); the policy is fixedterm and the expiry time has passed; the policy is neverexpires and the opt-in time or owner is missing (inconsistent row, reject); the policy is legacyunknown and the migration grace deadline from step 6 has passed. A neverexpires token that passes all of this is valid with no time check at all, which is the point of the feature. Because the token is opaque, this is one store read you are already doing for revocation, so the policy checks add no extra lookup. If you cache validation, cache the policy verdict with the record and evict on revoke and on any policy change, or a token converted from neverexpires to fixed_term keeps passing.

Expected output: valid non-expiring tokens pass forever, everything ambiguous fails. Success check: hand-edit a test row to never_expires with a null opt-in time and confirm the next request gets a 401, not a pass.

5. Compensate for the missing expiry with controls that do not depend on time

A token that never expires needs its risk carried elsewhere: narrowest scopes at creation, with broad scopes refused for neverexpires tokens; revocation that works in one owner action and is checked on every request; a last-used time that is actually maintained, plus a periodic dormant-token report listing neverexpires tokens unused for, say, 90 days so owners can kill them; and a periodic re-attestation listing (for example every 6 or 12 months the owner sees their non-expiring tokens and confirms each is still needed, with silence treated as a prompt to revoke, not as consent). An organization-level policy switch that forbids never_expires entirely, or caps it to named owners, belongs here too.

Expected output: a leaked non-expiring token is narrow, visible in reports, and one click from dead. Success check: the dormant report lists a seeded idle never_expires token and revoking it from that report works.

6. Migrate legacy tokens by classification, never by blanket assumption

Legacy rows were created when no expiration existed, so their null means unknown, not never. Migrate in this order. First add the new fields with every existing row set to legacyunknown and the expiry left null; ship validation that still accepts legacyunknown, so nothing breaks on deploy day. Then classify each row from evidence you actually have: tokens whose creation flow always used a known default lifetime get a computed expiry from creation time plus that lifetime and become fixedterm; tokens a owner confirms, through the opt-in act from step 2, become neverexpires with a real opt-in record (backdating the opt-in or faking a reason destroys the audit value, so do not); everything else stays legacyunknown. Give legacyunknown a published grace deadline, long enough for owners to act (weeks, not days), notify owners before it, and show the deadline wherever the token is listed. After the deadline, validation rejects legacy_unknown (step 4), which converts silence into expiry instead of into immortality. Run the backfill in idempotent batches stamped with a batch id, and record counts per class before and after so the migration is auditable and resumable.

Expected output: after the deadline, zero rows remain legacyunknown and every surviving token has a defensible policy. Success check: the per-class counts add up to the pre-migration row count, and a legacyunknown test token starts failing the day after the deadline.

7. Test the matrix before shipping

Pin these cases: (1) fixedterm with a future expiry passes, with a past expiry fails; (2) fixedterm with a null expiry fails closed; (3) neverexpires with a complete opt-in passes at a simulated date years out; (4) neverexpires with a missing opt-in fails; (5) creating with no expiration fields yields the default finite expiry; (6) an update sending a null expiry leaves the expiry untouched; (7) legacyunknown passes before the grace deadline and fails after it; (8) a revoked neverexpires token fails immediately; (9) converting neverexpires to fixedterm takes effect on the next request even with the validation cache warm.

Expected output: all nine pass. Success check: case 2 or 4 passing a request is the whole bug class this design exists to kill; stop and fix before anything ships.

When this applies

• An opaque (server-stored, revocable) token system is adding a never-expires option for CI, devices or integrations that cannot rotate on a schedule.
• You are adding an expiration column to a token table that already has live tokens in it.
• Auditors or customers ask which long-lived credentials were deliberate, and today the honest answer is that nobody knows.
• Your update API is partial (PATCH-style) and clients serialize untouched fields as null.

When this does not apply

• Self-contained JWT access tokens. A signed token with no expiry claim cannot be reclassified or revoked per token the way an opaque record can; keep those short-lived and put longevity in a revocable refresh flow instead.
• Systems where policy forbids non-expiring credentials outright. Then the correct feature is a maximum lifetime with renewal reminders, not an opt-in.
• One-off personal scripts where a finite token plus a calendar reminder is honestly enough; the opt-in machinery is for fleets and customers, not a single cron job.
• Session cookies and interactive sign-in. Those follow session lifetime rules, not API token policy.

Variant: null suddenly means expired after a framework or ORM change

A default value, a non-nullable mapping or a schema migration flips the reading of existing nulls and live tokens die at once. This is why the policy field, not nullability, carries the meaning: readings of null can be changed by tooling, an explicit enum value cannot. If you are already in this incident, restore behavior from the policy classification, not by flipping nullability back and forth.

Variant: every legacy token got set to never expires during migration

The blanket backfill. It feels safe because nothing breaks, and it converts every forgotten token into a permanent one. Recover by reclassifying from creation evidence where you have it, and putting the rest back to legacy_unknown with a fresh grace deadline and owner notification; expect complaints, and expect them to be cheaper than the alternative.

Variant: clients demand no expiry because renewal keeps breaking them

Usually the real problem is that renewal means a human pasting a new value into five places. Offer the opt-in, but pair it with the dormant report and re-attestation from step 5, and fix the renewal path anyway (overlap window where old and new tokens both work briefly) so the finite option stops hurting; many clients then choose finite voluntarily.

Why it happens

Nullable columns conflate three different facts: not set yet, deliberately none, and we never recorded it. Expiration is exactly the field where those three meanings have opposite security consequences, so a single null carrying all of them guarantees that some code path, some client or some migration reads it the dangerous way. Making the choice an explicit policy value, with the opt-in recorded as data, moves the decision from an accident of serialization to an auditable act, and the legacy class exists because history genuinely does not contain the decision; a deadline, not a guess, is the honest way to close that gap.

Edge cases and pitfalls

• Sentinel dates: using a far-future date (year 9999) instead of null plus policy works for comparisons but pollutes reports, sorting and user interfaces, and some databases and JSON serializers handle it badly. Prefer the explicit policy.
• Time zones: store and compare expiry and deadlines in UTC; a grace deadline compared in local time ends the migration early or late by the server offset.
• Imports and admin tools: any bulk import, seed script or support tool that writes token rows must go through the same policy assertion from step 1, or it becomes the back door that creates opt-in-free never_expires rows.
• Reporting queries: dashboards that count expiring soon usually filter on the expiry time alone and silently include or exclude nulls depending on the database. Filter on the policy first, then the time.
• Deleting versus revoking legacy tokens at the deadline: prefer letting validation fail (the row and its history survive for audit) over hard deletion, so an owner who returns late can be re-issued cleanly with the history intact.
• Policy caps changing later: if an organization later forbids never_expires, existing tokens need the same treatment as legacy ones (notice, grace deadline, then fail closed), not an instant silent kill of production integrations.