agent parsed RSS dates with the wrong timezone and reposted week-old items as new
Teaches an agent to parse feed dates as timezone-aware values normalized to UTC, and to dedupe on item identity rather than dates. Use when a feed reader reposts old items as new because of timezone mistakes. Not for feed discovery, content extraction, or publishing cadence.
TL;DR
Parse every date into a timezone-aware value and normalize to UTC before any comparison or storage; when the feed omits the zone, apply an explicit documented default. Dedupe on the item guid, not on the date, and skip anything older than a UTC high-water mark. Old items stop re-entering the queue.
agent parsed RSS dates with the wrong timezone and reposted week-old items as newUse this when
- a feed reader reposts old items as new
- items show the wrong published time by several hours
- feeds come from sources in different timezones
- the pipeline compares dates to decide what is new
Not for this skill when
- the feed items themselves are duplicates at the source
- you need to discover new feeds or parse article content
- publishing cadence or scheduling is the problem
Steps
- Parse every date into a timezone-aware datetime; if the feed omits the zone, apply the feed's declared zone or UTC explicitly and document the choice. Expected: no naive datetimes anywhere in the pipeline.
- Normalize all timestamps to UTC before comparing or storing. Expected: age is computed correctly regardless of source zone.
- Dedupe on the item guid (falling back to link) instead of on the parsed date. Expected: reposts of the same item are caught even if the date shifts.
- Compare item time against a stored high-water mark in UTC and skip anything older. Expected: old items never re-enter the queue.
- Log the raw date string alongside the parsed value for the first items of a new feed. Expected: misparsed feeds are visible immediately.
Variant phrasings
feed reader reposted week-old items as new
Short version: parse timezone-aware, normalize to UTC, dedupe on guid.
RSS dates parsed with the wrong timezone
Short version: never leave a datetime naive; default the missing zone explicitly.
old items keep re-entering the queue
Short version: keep a UTC high-water mark and skip anything older.
Compatibility: works with feedparser, Python datetime with zoneinfo, or any date library that supports IANA zones; no version pinned.
Why it happens
The parser read the date string and produced a naive datetime, which the pipeline interpreted in the server's local zone. A feed publishing in UTC looked hours newer than it was, so the is-it-new check passed for week-old items. One missing timezone turned every comparison into a wrong answer.
Edge cases / pitfalls
- Some feeds mix zones across items; normalize per item, never assume one zone per feed.
- Daylight saving shifts move naive comparisons by an hour twice a year; aware datetimes avoid this entirely.
- Guids are sometimes missing or reused; the link fallback covers most of the rest, and a content hash covers the remainder.
- Document the default zone choice per feed; a silent default becomes a mystery bug six months later.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_FDiKiKoetRyfLRlMj3SIVw