# Separate Page Traversal from the Committed Event Watermark Reconcile retries in paginated event ingestion by keeping transport pagination state separate from a monotonic watermark, with an explicit boundary for gaps among eligible event positions. Exact reference: {"kind":"skill_version","skill_id":"skl_qn4C34ErNDtLpeFtP6A_7g","version_id":"skv_O9x5gGWnIdAsaOmnLw75SA"} Applicability: [{"constraint":"ordered stream consumption with retryable page fetches, durable side effects, and eligibility filters evaluated against a stable source boundary","technology":"paginated event ingestion","version_scheme":"unknown"}] # Separate page traversal from the committed event watermark ## When to use Use this procedure when a consumer reads an ordered event stream through paginated API responses and a retry, timeout, or crash can occur after a page is fetched but before every event in that page is durably handled. ## Core invariant Maintain two different positions: - The **traversal cursor** is the API token used to fetch another page during the current attempt. - The **committed watermark** is the greatest event position for which every earlier eligible event in the same ordered stream has a durable outcome. A page token proves where fetching reached. It does not prove where processing committed. Never copy a returned next-page token into the durable watermark. ## Procedure 1. Define one stable total order for eligible events. Prefer an immutable sequence; otherwise use the complete ordering tuple plus a unique immutable event identifier. 2. Bind the consumer state to the stream scope, filters, ordering definition, and snapshot or upper bound. A cursor from a different binding cannot reconcile the same progress. 3. Persist the committed watermark independently from any traversal cursor. Treat it as monotonic: a retry or stale worker may leave it unchanged or advance it, but may never lower it. 4. Fetch a page, normalize it into the stable order, and discard or deduplicate events at or before the committed watermark. Overlap is expected replay, not evidence that progress regressed. 5. Handle remaining events in order. Couple each event's durable effect with its deduplication or completion record in one transaction when possible. If the effect is external, use a transactional outbox or an equivalent durable handoff before treating the event as committed. 6. Advance the watermark only through the greatest gap-free prefix of eligible events whose events all have durable outcomes. Gap-free refers to eligibility within the bound stream, not to consecutive numeric positions. Use a compare-and-set, lease generation, or ownership epoch so a stale attempt cannot advance shared progress. 7. If event B fails after event A commits, keep the watermark at A even if later events were fetched. Resume from a replay position that can return B, then tolerate duplicates through the durable completion records. 8. Use the page's continuation token only to continue the live traversal after the current page's required prefix is committed. If that token must be persisted for an API with opaque positions, persist it as a replay anchor associated with the prior committed watermark, not as proof that the page completed. 9. On an empty, repeated, or stale page, do not infer processing progress from the token alone. Continue only when the API contract proves the token moves within the same bound; otherwise restart from the durable replay anchor or fail visibly. ## Reasoned example A page contains positions 41, 42, and 43 and returns a token for the following page. The effect for 41 commits, the effect for 42 fails, and 43 is not handled. The durable watermark becomes 41, while the returned token remains only transport state. A retry replays from after 41, deduplicates any repeated 41, and attempts 42 again. Persisting the next-page token as the watermark would skip 42 and 43. This example is reasoned, not an executed test. ## Eligibility boundary example Suppose a stable snapshot contains positions 41, 42, and 43, but the bound filter makes only 41 and 43 eligible. If durable outcomes exist for both eligible events and the snapshot proves that 42 is ineligible, the eligible prefix can be complete through 43 even though the numeric sequence has a hole. Numeric continuity is not required. The same advancement is unsafe when position 42 can be inserted behind the watermark, when its ordering fields can change, or when mutable filter membership can make it eligible after the decision. In those cases, the consumer has not proved a complete eligible prefix through 43. It needs an insertion-stable boundary, versioned snapshot, or replay window before advancing. A different page token or an apparently empty page does not supply that proof. This boundary example is reasoned, not an executed test. ## Important limits A gap-free prefix means the ordered prefix of eligible events, not necessarily consecutive integer values. If the source can insert events behind the watermark, mutate ordering fields, or change filter membership without a stable snapshot, a monotonic consumer watermark alone cannot guarantee no skips. Obtain an insertion-stable source boundary, versioned reads, or an explicit replay window. This procedure provides at-least-once delivery unless the effect and checkpoint share an atomic transaction. Exactly-once claims require stronger end-to-end idempotency and atomicity guarantees. ## Validation scenarios Test the target implementation with failures before the first effect, between an effect and its completion record, between completion and watermark advancement, after the final event but before requesting the next page, and with two attempts using different ownership generations. Also test a stable snapshot where a numeric position is provably ineligible and a mutable-membership case where the same position becomes eligible later. Verify that duplicates can occur, omissions cannot, the watermark never decreases, and a numeric gap is accepted only when the bound stream proves it contains no missing eligible event. No tests were executed for this guidance. ## Supporting basis and limitations Reasoned analysis recorded in the maintenance conversation through sequence 2. The discussion separates this eligibility-boundary clarification from an independent recovery proposal and states that no implementation was inspected and no tests were executed. ## Change and rationale Sharpens the gap-free eligible-prefix boundary with a concrete stable-snapshot example and a contrasting mutable-membership case, while preserving the existing traversal, durability, retry, and validation guidance. The base states that a gap-free prefix concerns eligible events rather than consecutive integers, but the boundary remains easy to misread. A concrete example distinguishes a harmless numeric hole proven ineligible within a stable binding from an unsafe hole whose membership can change after the watermark advances.