data agent's regex parsing dates it should not: use a date parser

TL;DR

Replace the regex with a real date parser and declare the expected format per source. Regexes match shape, not meaning: 03/04/2026 is valid month-first and day-first, and no pattern can tell which was meant.

Symptom

Dates come out wrong with no errors: month and day swapped, two-digit years misread, or non-dates matched as dates. The pipeline looks healthy; the data is quietly wrong.

Steps

1. Replace the regex with a real date parser with explicit ordering settings:

from dateutil import parser
parsed = parser.parse(raw, dayfirst=False, yearfirst=False)

Expected output: actual calendar validation (month 13 raises) instead of shape matching.

2. Declare the expected format per source and reject strings that do not match instead of guessing:

from datetime import datetime
parsed = datetime.strptime(raw, "%Y-%m-%d")

Expected output: unexpected formats raise immediately at the source that produced them.

3. Add a sanity check: the parsed date must fall within a plausible range for the data (not year 1900, not next century).

Expected output: misparses that slip through parsing still get caught before landing.

4. Log the raw string alongside the parsed value so misparses are auditable later.

Expected output: when a bad date is found downstream, you can trace it to the exact input.

When to use

• Date extraction silently produces wrong dates
• Formats vary across sources the agent reads
• You need validation, not just extraction

When not to use

• Fixed-format timestamps you control end to end (strptime with one format is already correct)
• Dates are already parsed upstream (do not re-parse)
• You need to match date-like substrings inside free text for highlighting (thats search, not parsing)

Variant phrasings

agent swapped day and month on european dates

Set dayfirst=True for that source explicitly; never rely on a global guess.

two-digit years parsed as 19xx instead of 20xx

Declare the century rule per source; parsers guess, and they guess inconsistently.

Why it happens

A regex checks that characters look date-shaped; it cannot know whether 03/04 means March 4th or April 3rd, and it happily matches things like 99/99/9999. Real parsers validate against the calendar and let you pin the interpretation per source.

Edge cases

• Ambiguous formats need a per-source rule, never a global default; US and EU sources in one pipeline need different settings.
• Timezone-naive vs timezone-aware datetimes cause comparison bugs downstream; normalize at parse time.
• dateutil's fuzzy parsing can surprise on messy strings; prefer strict strptime formats wherever the source format is known.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_zjwu0LMabLYfCyaXDQKqnA