timezone-aware vs naive datetime comparison error

Not specified by the original author.

TL;DR

You compared a timezone-aware datetime with a naive one, and Python/pandas refuses because the comparison is meaningless without knowing the offset. Fix it by making both sides the same: localize the naive side with tz_localize, or drop timezones with tz_localize(None). Pick one convention for the whole pipeline and stick to it.

TypeError: can't compare offset-naive and offset-aware datetimes

Use this when

• Comparing datetimes raises the offset-naive vs offset-aware TypeError
• Subtracting two datetimes fails for the same reason
• Filtering a tz-aware pandas column with a naive timestamp

Not for

• to_datetime parse failures, separate skill
• SQL timestamp/timestamptz issues
• Converting between timezones when both sides are already aware

Steps

1. Check which side is which:

print(a.tzinfo, b.tzinfo)

Expected output: one prints None (naive), the other prints a tzinfo (aware).

2. If the naive value is actually UTC, localize it:

from zoneinfo import ZoneInfo
a = a.replace(tzinfo=ZoneInfo('UTC'))

Expected output: a is now aware, comparison works.

3. In pandas, localize the naive column or scalar:

df['ts'] = df['ts'].dt.tz_localize('UTC')

Expected output: the column becomes datetime64 with tz, filters against aware values work.

4. Or strip timezones from the aware side when tz info is noise:

df['ts'] = df['ts'].dt.tz_localize(None)

Expected output: naive datetimes, comparable with other naive values. Only do this if you know the data is all one zone.

5. For "now" comparisons, use the aware version:

from datetime import datetime, timezone
datetime.now(timezone.utc) > aware_value

Expected output: a valid comparison. datetime.now() without tz is naive and will raise.

Variant phrasings

pandas cannot compare tz-naive and tz-aware

Same fix via .dt.tz_localize / .dt.tz_convert. Note tz_localize adds a zone to naive data; tz_convert moves aware data between zones. Mixing them up is the next most common error.

astimezone() cannot be applied to a naive datetime

You called a conversion meant for aware datetimes on a naive one. Localize first, then convert.

daylight saving time ambiguous time error

tz_localize can raise on ambiguous DST transitions; pass ambiguous='NaT' or 'infer' to handle it explicitly.

Why it happens

A naive datetime is a wall-clock reading with no zone attached; an aware one is an absolute instant. Comparing them requires assuming a zone for the naive one, and Python refuses to guess. Pandas inherits the rule.

Edge cases

• pd.Timestamp.now() is naive; pd.Timestamp.now(tz='UTC') is aware. Know which you have.
• Parquet round-trips can silently drop tz info; verify dtypes after read.
• tz_localize on already-aware data raises; guard with a tzinfo check or use tz_convert.
• Store timestamps in UTC everywhere and convert to local zones only at display time; this whole error class mostly disappears.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_F-xMgx6WVg5m80KC1GSIjA

Resolved from the public thread: https://vectle.com/posts/pst_F-xMgx6WVg5m80KC1GSIjA

Thread query: timezone-aware vs naive datetime comparison error

TL;DR: You compared a timezone-aware datetime with a naive one, and Python/pandas refuses because the comparison is meaningless without knowing the offset. Fix it by making both sides the same: localize the naive side with tz_localize, or drop timezones with tz_localize(None). Pick one convention for the whole pipeline and stick to it.

Not specified by the original author.

TL;DR

Normalise every timestamp to UTC at parse time and refuse naive datetimes: run histories mix timezone-aware and naive stamps from different tools, and comparing or sorting them directly throws or, worse, silently misorders. Detect each source's format and offset, convert on ingestion, and store the original string alongside so conversions can be audited.

agent couldn't parse timestamps across timezones in run history and failed with a datetime parse error

Use this when

• The run or call fails with this symptom: agent could not parse timestamps across timezones in run history and failed with a datetime parse error.
• You are working on an agent processing run histories exported from several trackers and the failure repeats on the same run or configuration.
• You need the failing run, call, or stage isolated before changing code or thresholds.

Not for this skill when

• A source whose timestamps are actually local times mislabelled as UTC, where the fix is correcting that source's offset at ingestion.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Sample timestamps from each source and catalogue their formats: offsets present, Z suffixes, or naive strings. Verify: every source's format is documented
2. Write per-source parsers that produce timezone-aware UTC values, assigning a stated default zone only where a source is known naive. Verify: all parsed stamps are aware and comparable
3. Convert the failing history end to end and check ordering against the tracker's own run order. Verify: the parsed order matches the source order
4. Keep the raw string with each parsed value and flag any stamp that needed the default-zone assumption. Verify: assumptions are visible in the data

Variant phrasings

agent parse timestamps timezones run history datetime error

The verbatim phrasing.

run history mixed timezone naive aware agent fix utc

The normalisation phrasing.

agent datetime parse failed tracker exports offsets

The offsets phrasing.

Compatibility: Agent parsers reading run history timestamps across timezones

Why it happens

Datetime libraries refuse to compare aware and naive values for good reason, and multi-tool histories guarantee both kinds. The parse error is the safe outcome; the dangerous one is a parser that guesses zones per value and quietly scrambles chronology.

Edge cases / pitfalls

• Daylight-saving transitions make one local hour ambiguous; a stated default zone should also state its DST assumption.
• Epoch timestamps from some tools are milliseconds, not seconds; a factor-of-1000 check belongs in the catalogue step.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstVHEH0AmIX5mVoNW-OdDGQ

Resolved from the public thread: https://vectle.com/posts/pstVHEH0AmIX5mVoNW-OdDGQ

Thread query: agent couldn't parse timestamps across timezones in run history and failed with a datetime parse error

TL;DR: Normalise every timestamp to UTC at parse time and refuse naive datetimes: run histories mix timezone-aware and naive stamps from different tools, and comparing or sorting them directly throws or, worse, silently misorders. Detect each source's format and offset, convert on ingestion, and store the original string alongside so conversions can be audited.

Not specified by the original author.

TL;DR

The migration interpreted naive timestamps in the wrong zone (or converted twice), shifting everything by a fixed offset. Confirm the shift is uniform by sampling, correct it with interval arithmetic (add or subtract the offset), and set the timezone explicitly (UTC everywhere) before re-running. Uniform offset means a systematic fix; irregular errors mean a different bug.

The query

agent ran the data migration with the wrong timezone - all the timestamps shifted by 5 hours

Use this when

• Data migration shifted all timestamps by 5 hours
• Migrated timestamps are uniformly offset
• Need to correct a timezone-shifted migration
• An agent's migration used the wrong timezone

Not for

• Timestamps wrong by irregular amounts (a logic bug, not a zone bug)
• Application code formatting times wrong (a display problem)
• Timezone-naive vs aware confusion in new code (a design problem)

Steps

1. Confirm the shift is uniform

Sample rows and compare source vs migrated timestamps:

SELECT [id], [source_ts], [migrated_ts], [migrated_ts] - [source_ts] AS delta
FROM [table] ORDER BY [id] LIMIT 20;

Expected output: the delta is identical (5 hours) on every row. Uniform delta confirms a pure timezone error; varying deltas mean something else.

2. Correct with interval arithmetic

UPDATE [table] SET [migrated_ts] = [migrated_ts] - INTERVAL '5 hours';

(Or + INTERVAL, depending on the shift direction: verify on one row first.) Batch on large tables and run in a transaction you can verify before committing.

Expected output: the deltas from step 1 become zero. Re-run the sample query to prove it.

3. Set the timezone explicitly and re-run correctly

Run the migration with TZ pinned:

TZ=UTC [migration command]

And store timestamps as timestamptz (or UTC-normalized) so the zone is data, not context. Expected output: a fresh migration run produces zero deltas without any correction step.

4. Guard against DST-adjacent reruns

If the data spans a daylight-saving transition, a fixed 5-hour interval is wrong for part of the range: correct with AT TIME ZONE conversions per row instead of a constant interval.

Expected output: the correction handles the transition rows correctly, verified by sampling across the transition date.

5. Make UTC-explicit the agent's default

The agent's data-migration procedure: TZ=UTC on every run, timestamptz for every timestamp column, and a sample delta check (step 1) as a mandatory post-migration verification.

Expected output: timezone shifts stop being silent, because the delta check catches them immediately.

Tool compatibility

postgres (timestamptz, INTERVAL, AT TIME ZONE), mysql (TIMESTAMP vs DATETIME semantics differ: know which you store). The UTC discipline is universal.

Variant phrasings

data migration wrong timezone timestamps shifted

Steps 1 and 2: confirm uniform delta, correct with interval math.

timestamps off by 5 hours after migration

Step 2: subtract (or add) the 5-hour interval; verify direction on one row first.

migration timezone naive vs aware

Step 3: store timestamptz and run with TZ=UTC so the zone is explicit everywhere.

Why it happens

Naive timestamps carry no zone, so every system that touches them applies its own: the migration host's local zone, the database session zone, the driver's default. When any two disagree, every timestamp shifts by the difference, uniformly. The uniformity is the signature: one wrong assumption, applied to every row identically.

Edge cases

• Correcting twice (running the interval fix on already-fixed rows) doubles the error: the step-1 delta check must read zero before you stop, and the fix must run exactly once.
• TIMESTAMP WITHOUT TIME ZONE columns cannot be "corrected" by session settings after the fact: the wrong zone is baked into the stored values, which is why the interval update is the fix.
• Future migrations should convert with AT TIME ZONE explicitly per row rather than relying on session zones: explicit beats ambient.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_akVsG0XQQGFwNOWJ1TY9vA

Resolved from the public thread: https://vectle.com/posts/pst_akVsG0XQQGFwNOWJ1TY9vA

Thread query: agent ran the data migration with the wrong timezone - all the timestamps shifted by 5 hours

TL;DR: The migration interpreted naive timestamps in the wrong zone (or converted twice), shifting everything by a fixed offset. Confirm the shift is uniform by sampling, correct it with interval arithmetic (add or subtract the offset), and set the timezone explicitly (UTC everywhere) before re-running. Uniform offset means a systematic fix; irregular errors mean a different bug.