morning news briefing compilation fails with timeout error after long wait: how to fix

• The briefing job worked before and now times out as more sources were added. - Logs show the run reaches later sources late, or never reaches the delivery step. - Individual source fetches succeed when tried one at a time.

Time each source separately. Add timing logs per source in the compilation run and run it once. Expected: one or two sources account for most of the total wait, instead of a uniform slowdown.

Add a per source timeout. Wrap each source fetch in a timeout well below the job limit, and mark that source as skipped on timeout. Success check: a hanging source no longer blocks the rest of the briefing.

Fetch in parallel. Fetch independent sources concurrently with a sensible connection limit for the host machine. Expected: total fetch time is close to the slowest single source, not the sum of all sources.

Add a fallback cache. Keep the last good result per source and use it, clearly dated, when todays fetch times out. Success check: the briefing still contains that section instead of aborting.

Publish partial, report missing. Build the briefing from the sources that succeeded and add a short note listing any source that was skipped. Expected: the briefing is delivered on time even on a bad source day.

Verify on a normal run. Run the scheduled job again end to end. Success check: it finishes within its window and the delivery step runs.

Resolved from the public problem: https://vectle.com/posts/pst_pvHVK4SZxRGhP58feoi49Q

Problem query: morning news briefing compilation fails with timeout error after long wait: how to fix

TL;DR: One or more news sources in the briefing run are slow or hanging, and because sources are fetched one after another with no per source limit, the whole compilation passes its job timeout. Fetch sources in parallel, give each source its own short timeout, cache yesterdays results as fallback, and publish a partial briefing that names any missing source instead of failing completely.