agent's reconciliation job timed out at the 55-minute container limit with 20k lines unmatched

TL;DR

The container limit will not move, so the job must become resumable: stage the work as load, exact match, rule match, fuzzy match, and report, persisting the match state after each stage and each chunk within a stage. Run the job as a chain of invocations that each start from the last saved state and stop cleanly before the limit. With checkpointing, the 55-minute ceiling becomes a scheduling detail instead of a wall, and 20k lines progress every run instead of restarting.

agent's reconciliation job timed out at the 55-minute container limit with 20k lines unmatched

Use this when

• The job is killed at a fixed platform limit, here 55 minutes, with work visibly incomplete, here 20k lines unmatched.
• Each rerun starts the matching from scratch and reaches a similar point before dying.
• Match results are held in memory or only written at the end.

Not for this skill when

• Profiling shows one stage consuming nearly all the time through an obvious inefficiency, such as a nested loop. Fix the algorithm first; checkpointing a bad algorithm institutionalises it.
• The platform allows a higher limit or a batch tier on request. If a simple limit raise exists, take it, and still add checkpoints for safety.

Steps

1. Profile the job stages to see where the 55 minutes go and what state exists at kill time. Verify: stage timings and the lost-work point are measured
2. Persist match outcomes incrementally: every chunk of lines writes its matches and remaining exceptions before processing continues. Verify: killing the job mid-stage preserves all completed chunks
3. Split the job into stage invocations with saved state passed between them, each sized to finish comfortably inside the limit. Verify: each stage completes standalone from saved state
4. Add a driver that chains stages and resumes the chain after any kill, tracking overall progress by lines matched and remaining. Verify: the chain survives a simulated kill and continues
5. Run the full 20k-line backlog through the chain and report throughput per invocation. Verify: all lines reach matched or exception state across the chained runs

Variant phrasings

agent job container timeout 55 minutes resumable stages

The platform-limit phrasing.

reconciliation job killed time limit checkpoint state

When making the job resumable.

Compatibility: Container and serverless time limits for agent batch jobs, including 55-minute class limits on hosted runners.

Why it happens

Hosted platforms impose hard execution ceilings for their own economics, and batch reconciliation volumes grow past them quietly. A job designed as one long transaction with end-only persistence has a maximum feasible size set by the ceiling, and crossing it produces total failure rather than partial progress. Checkpointed stages invert the relationship: the ceiling bounds one invocation, not the job, and progress becomes cumulative across invocations.

Edge cases / pitfalls

• Stop starting new chunks near the ceiling; a stage should check elapsed time and wind down with state saved, not be killed mid-write.
• Persist state in durable storage outside the container, not on its local disk, or the checkpoint dies with the container.
• Throughput work still matters: doubling lines per minute halves the invocations. Profile after checkpointing, with the pressure off.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstFNFA2DO1pu4pkW7BKfudA