Close a timestamp group before advancing a timestamp-only event cursor

When to use

Use this when a paginated event API can resume by timestamp but cannot seek by a unique event key. Several events may share one timestamp, including across page boundaries, and a retry may replay earlier pages. This procedure addresses the boundary between one timestamp group and the next. It assumes stable event identities and pages in nondecreasing timestamp order, but that ordering alone is not a completeness guarantee.

Check the source contract first

A timestamp group is safe to close only if all of these hold:

• Pagination can traverse every event at one timestamp, even when that group exceeds one page. The source must provide a stable snapshot, a continuation with equivalent completeness, or another proven traversal contract. Nondecreasing timestamps on individual pages do not establish this.
• Seeing a later timestamp proves that no earlier timestamp can arrive in a future fetch, or the provider supplies an explicit finality watermark with that meaning. A snapshot of current contents does not prove future finality if backfills are allowed.
• Event timestamps and stable identities do not change after publication.
• Replayed effects are safe through an event receipt, a sink idempotency key, or an atomic effect and receipt.

If the API cannot page reliably through a tied group, or can later insert events behind the proposed watermark without a bounded finality rule, a timestamp-only checkpoint cannot guarantee complete ingestion. Obtain a stable composite seek key or a source contract that supplies both complete traversal and finality before claiming that guarantee.

Boundary example: ordered pages can still skip a tied event

Suppose the page size is two. Events A, B, and C all have time 10:01, and D has time 10:02. On the first offset request, the source orders them A, B, C, D, so page one returns A and B. Before the next request, the same unchanged event set is presented in the tie order C, A, B, D. Offset two now returns B and D. Both pages are nondecreasing by timestamp; every identity and timestamp stayed fixed; and the consumer saw a later time. Nevertheless C was never returned. Advancing a strict timestamp checkpoint through 10:01 would permanently skip C.

This is a traversal failure, separate from the question of whether 10:01 is final against future backfills. Receipts can suppress B's duplicate effect but cannot reveal the missing C. A stable snapshot, stable total ordering over an unchanged set, or a provider continuation that guarantees complete enumeration can address this specific failure. A deterministic tie break by itself does not make offset pages safe if the source can insert or remove events during traversal. Seeing D is evidence of a group boundary only after the traversal contract and finality contract are established.

This is a reasoned counterexample, not an executed test.

Procedure

1. Store a durable closedthroughtimestamp: the latest timestamp whose entire group has terminal durable outcomes. Query strictly after this value. Store a checkpoint generation for compare-and-swap updates.
2. Treat all events at the next timestamp as an open group. Traverse its pages under the source's complete traversal contract. Record each event's durable outcome under its stable identity before relying on it for replay. A retry may restart from the last closed timestamp and use receipts to recognize work already done.
3. Keep the open timestamp out of the durable cursor while any event in its group is pending, failed, or known only from memory. A page boundary is not proof that the group is complete.
4. After traversing a complete, stable page sequence, use the first strictly later timestamp or an explicit source finality watermark as evidence that the open group is complete only when the source contract gives that observation its claimed meaning. Confirm that every event in that group has a durable terminal outcome. Then compare-and-swap closedthroughtimestamp and the checkpoint generation. A losing retry rereads the checkpoint.
5. Begin the next attempt strictly after the newly closed timestamp. Events at the later timestamp remain an open group until that group obtains its own completion proof.

If the stream becomes quiet while the newest group has no finality proof, leave that group open and replay it on later polls. This costs repeat reads but preserves the boundary.

Retry example

Suppose the checkpoint is closed through time 10:00. One page contains events A and B at 10:01; the next contains C at 10:01 and D at 10:02. A worker durably handles A and B, then crashes. On retry it queries after 10:00, recognizes A and B by their receipts, and handles C. Seeing D proves that the 10:01 group has been fully traversed only under the source ordering, completeness, and finality contracts above. Once C's outcome is durable, the worker may close through 10:01. The next query after 10:01 still includes D.

This is a reasoned example, not an executed test.

Limits and validation

A timestamp-only cursor cannot distinguish two events with the same timestamp by itself. Receipts prevent duplicate effects; a complete and final timestamp group prevents skips. Validate a real implementation with a tie group larger than one page, tie order changing between offset requests even when the event set is unchanged, a crash before and after each receipt, a crash before the checkpoint update, concurrent stale retries, and a late event at the proposed closed timestamp. The late-event case must either be ruled out by the provider contract or show that the checkpoint remains open. No implementation or tests were executed for this guidance.