agent can't parse dagster event log timestamps, parsing failed
Fixes agent parsers choking on Dagster event log timestamps: the timestamp formats in play and how to normalize them. Use when timestamp parsing fails on Dagster events. Not for missing events.
TL;DR
Dagster event timestamps come in multiple representations: epoch floats in some API responses, ISO strings in others, and timezone handling that varies by client. Normalize everything to UTC epoch at the ingestion boundary and parse defensively. A parser that assumes one format breaks on the first event that uses the other.
agent can't parse dagster event log timestamps, parsing failedUse this when
- An agent's parser fails on Dagster event timestamps.
- Some events parse and others do not.
- Timezone-naive and timezone-aware values mix.
Not for this skill when
- Events are missing entirely. That is a query problem.
- The timestamps parse but look wrong. That is a timezone bug.
Steps
- Capture a failing timestamp value verbatim. Note the exact string or number. Verify: the failing format is in hand
- Enumerate the formats Dagster emits in your path: epoch float, ISO string, with and without timezone. Note the set. Verify: the format zoo is mapped
- Write a parser that tries each format and normalizes to UTC epoch. Note the function. Verify: one parser handles all formats
- Add the failing samples as regression tests. Note the tests. Verify: the formats are pinned
- Verify the parser handles a mixed event stream. Note the clean parse. Verify: the fix is verified
Variant phrasings
dagster event timestamp format
The format-shaped search.
dagster graphql timestamp parsing
The API-shaped phrasing.
Compatibility: Dagster 1.x event logs via GraphQL and Python APIs. Formats are stable but mixed across endpoints.
Why it happens
Dagster's event system grew organically: storage writes epoch floats, GraphQL serializes ISO strings, and clients add their own timezone assumptions. No single format won, so consumers see several. A parser written against one endpoint's format breaks the moment the agent queries a different endpoint or Dagster changes a serializer. Normalizing at the boundary, one function, every timestamp in, UTC epoch out, contains the mess in a single place.
Edge cases / pitfalls
- Epoch floats lose precision for very old or far-future dates in float32 contexts. Use float64 or decimal.
- Timezone-naive ISO strings are ambiguous. Assume UTC only if you have verified the emitter; otherwise treat as an error.
- Sorting events by timestamp across formats without normalizing first produces wrong orders. Normalize before comparing.
- Dagster version upgrades can change a serializer. Re-run the format regression tests after upgrades.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_hhwUOWQoOoeFRjUqnoWb-Q