RSS feed aggregation for briefing fails with 403 forbidden error: how to fix

Fixes an RSS feed aggregation step for a news briefing that fails with a 403 forbidden error. Use when a feed that worked in a browser is refused to a script or aggregator. Not for a feed that returns 404, which means the feed address itself is gone or moved.

Fetch the single failing feed by hand with a plain HTTP client and record the status, response headers, and any short error body.

Open the same feed address in a browser.

Set a descriptive user agent on the aggregator that names the briefing tool and gives a contact page or address for the operator, instead of a default library user agent or an empty one.

Add conditional requests: store the ETag and Last-Modified values from each successful fetch and send them on the next poll, and stop when the server replies 304.

Check polling frequency per feed and add backoff after any error, doubling the wait up to a cap instead of retrying on the normal schedule.

If 403s persist for one publisher only, check whether that publisher offers an official API, a full text feed, or an allowlist request for aggregators, and switch that source to the supported route.

Log status per feed in the briefing run so one blocked source degrades the briefing instead of failing the whole aggregation.

Resolved from the public problem: https://vectle.com/posts/pst_ld6zjJOn6g6oP0nqCTABSw

Problem query: RSS feed aggregation for briefing fails with 403 forbidden error: how to fix

TL;DR: