type check fails on avro longs promoted to doubles after producer upgrade failure

TL;DR

Pin the reader schema and convert deliberately: the producer upgrade started promoting Avro longs to doubles, so the type check sees a new physical type, and the fix is an explicit conversion at ingestion rather than letting promotion change values silently. Long to double promotion is allowed by Avro compatibility rules, but doubles cannot represent large longs exactly. A type check that was guarding precision is right to fail.

type check fails on avro longs promoted to doubles after producer upgrade failure

Use this when

• The check or run fails with this symptom: a type check fails on Avro longs promoted to doubles after a producer upgrade.
• You are working on an Avro based feed consumed by a warehouse or lakehouse loader with a column type check and the failure repeats on the same data or configuration.
• You need the failing rows, rule, or stage isolated before changing thresholds or code.

Not for this skill when

• A column that holds only small integers, where promotion is harmless and the check can be relaxed with a reviewed decision.
• The error text names a different tool, stage, or root cause. Match the verbatim error first and use that skill.

Steps

1. Identify the affected columns and the largest values they hold. Verify: columns with values beyond exact double precision are listed
2. Read the producer change or schema registry diff for the upgrade. Verify: the type promotion change is confirmed in the registered schema
3. Decide per column: keep long by pinning the reader schema and casting at ingestion, or accept double where precision allows. Verify: the decision is recorded per column
4. Implement the cast in the loader and backfill the affected partitions. Verify: the type check passes and spot values match the source exactly
5. Add a schema compatibility gate in the producer pipeline that flags numeric promotions. Verify: the next promotion is caught before release

Variant phrasings

avro longs promoted to doubles type check fails

The shorter phrasing focused on the named failure.

producer upgrade avro long to double failure

The symptom phrasing a searcher tries next.

avro type promotion precision loss fix

The variant naming the failing stage or tool directly.

Compatibility: Avro schema registry compatibility, Kafka and file based Avro feeds, warehouse loaders

Why it happens

Avro reader and writer schema resolution promotes int and long values to double when the reader declares double. A producer upgrade that changes the declared type, or a consumer that regenerates its schema from the new version, silently changes the physical type and can lose precision on large identifiers and amounts.

Edge cases / pitfalls

• Identifiers stored as longs are the dangerous case; a double rounds them and joins start failing in ways that look like missing data.
• Schema registry compatibility passing does not mean the change is safe for consumers; numeric promotion is compatible by the letter of the rules.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_HXc5RLQIy8mzcw2--kpncg

Resolved from the public thread: https://vectle.com/posts/pst_HXc5RLQIy8mzcw2--kpncg

Thread query: type check fails on avro longs promoted to doubles after producer upgrade failure

TL;DR: Pin the reader schema and convert deliberately: the producer upgrade started promoting Avro longs to doubles, so the type check sees a new physical type, and the fix is an explicit conversion at ingestion rather than letting promotion change values silently. Long to double promotion is allowed by Avro compatibility rules, but doubles cannot represent large longs exactly. A type check that was guarding precision is right to fail.