# Recover an expired event page cursor from a proven replay boundary A procedure for replacing an expired opaque continuation token by replaying from a durable event boundary without lowering the applied watermark or skipping unprocessed events, including nonunique seek keys. Exact reference: {"kind":"skill_version","skill_id":"skl_2USZBW5LX84tfuPYoKUMaw","version_id":"skv_iNzNUaoe484bng2a1DP1FA"} Applicability: [{"constraint":"expired-opaque-continuation-token-with-stable-replay-seek","technology":"paginated-event-streams","version_scheme":"unknown"}] # Recover an expired event page cursor from a proven replay boundary ## When to use Use this procedure when a provider permanently rejects an opaque continuation token after a timeout or retry, while the consumer has a durable event checkpoint. The provider must offer a fresh seek or restart operation that can enumerate a complete range of retained events in a stable order. This procedure assumes immutable event identities and an append-only order in which an event cannot later appear behind a finalized applied boundary. Do not infer token expiry from a generic fetch failure. Check the provider's documented error meaning first; authorization changes, query-scope changes, transient failures, and retention loss require separate handling. ## Preserve the two progress coordinates Keep the provider token, the greatest fully durable applied event position, and a checkpoint generation separately. The provider token is an opaque fetch handle. The applied position is a claim about completed event work. If the provider's seek key is not unique, such as a timestamp shared by several events, that key alone is not a fully applied position until the whole group is durably complete. Keep a stable tie breaker as part of the position, or hold the applied boundary before the unfinished group. Token expiry invalidates the handle; it does not undo durable event outcomes or license a larger applied position. Retain receipts or sink idempotency keys for events that a recovery fetch may replay. The effect and its receipt must have a durable ordering that makes a crash followed by replay safe. ## Choose a recovery anchor 1. Read the durable checkpoint and record its token, applied position, generation, query scope, and any provider snapshot or finality marker. 2. Choose an anchor at or before the applied position from which the provider guarantees complete traversal. A seek strictly after the applied position is safe only when the source contract proves that the applied prefix is complete and no event can arrive behind it. For a nonunique seek key, prove that every event sharing the boundary key is included in the applied prefix before seeking past it. Otherwise seek inclusively from before the entire group and keep the returned successor pending until the group is complete. An inclusive seek at an earlier retained position gives room to verify overlap with receipts. 3. If the provider cannot seek, restart from a retained origin only if the resulting traversal is complete through the unprocessed range. Check retention before assuming that an old origin is still available. 4. If no retained anchor and traversal contract can prove coverage, record an unresolved gap and stop automatic cursor advancement. A newly issued token alone is not a coverage proof. A bounded lookback does not fix unbounded late backfills. If the source can publish an event behind the anchor or applied position, require a provider finality rule that excludes that event before treating the replay as complete. ## Reconcile and promote 1. Start a recovery attempt tied to the old checkpoint generation and query scope. Fetch from the chosen anchor. Keep the returned successor token pending. 2. Verify that the new traversal belongs to the same stream and satisfies the provider's ordering and completeness contract from the anchor. For events at or below the applied position, verify durable receipts and any payload identity needed by the contract. For new events, make effects and replay protection durable in source order. Stop at the first unresolved event or unexplained coverage gap. 3. After every event in the fetched page has a durable terminal outcome and coverage to its successor is proven, atomically replace the expired token with that page's exact successor using a compare-and-swap on the old token and generation. Advance the applied position only through newly completed work. The applied position never moves backward even when the recovery anchor does. 4. If the comparison loses, reread the winning checkpoint. Do not install a successor from the losing recovery attempt. Continue later pages with the same page-completion rule. 5. Treat an empty recovery page as a barrier only if the provider documents that its successor preserves complete coverage from the anchor. An empty response by itself does not prove that the missing interval contains no events. ## Reasoned example A checkpoint holds expired token C8, applied position 105, and generation 12. The provider documents a complete inclusive seek at position 103 and returns positions 103, 104, 105, and 106 followed by token N1. Receipts confirm 103 through 105. The consumer durably handles 106, then compares the checkpoint with C8 and generation 12 before storing N1 and applied position 106. If 106 fails or another worker changes the checkpoint first, N1 remains unpromoted. This is a reasoned example, not an executed test. A separate boundary example uses a provider that seeks by timestamp. Events A and B both have time 09:00, but only A has a durable receipt. Recording 09:00 as a fully applied position and seeking strictly after it would skip B. Keep the applied boundary before 09:00, or, if the provider supports it, use a documented stable composite seek position that distinguishes A from B. A complete inclusive replay of the 09:00 group can reconcile A's receipt and durably handle B; move beyond 09:00 only after the provider's ordering and completeness or finality guarantee closes that group. If the provider cannot enumerate the whole group, leave a gap unresolved. This is a reasoned example, not an executed test. ## Limits and validation The procedure cannot recover a missing interval from a provider that offers only an expired token and no complete retained replay path. It also cannot make a high-water mark safe against unrestricted backfills. Validate a real implementation against the provider contract with token expiry before and after event commits, retention-edge restarts, overlap mismatches, empty pages, concurrent recovery attempts, and nonunique seek keys whose ties cross page boundaries. An inclusive timestamp seek is insufficient if the provider omits or reorders members of the tie group without a stable traversal or finality guarantee. No tests were executed in preparing this guidance. ## Supporting basis and limitations Reasoned from the selected base version's completeness and durable-effect invariants and the maintenance conversation's timestamp-tie example. No provider-contract, implementation, or fault-injection tests were executed. ## Change and rationale Clarifies when a nonunique timestamp is an unsafe applied boundary, adds a concrete two-event replay example, and extends validation to ties across pages. The existing recovery procedure requires a complete applied prefix, but a seek key shared by multiple events can conceal an unfinished member. The added boundary rule and example make the coverage proof explicit while preserving the cursor promotion and replay safeguards.