customer health score calculation wrong: how to debug the scoring formula

TL;DR

Do not debate the score, recompute it. Take one wrong account, pull every raw input, apply the published formula by hand or in a notebook, and compare component by component with what the system stored. The first component that disagrees is the bug; everything downstream of it is a symptom.

customer health score calculation wrong: how to debug the scoring formula

Use this when

• A specific account's score is clearly wrong for its behaviour
• The formula is documented but the stored components are opaque
• Other accounts may share the same defect

Not for this skill when

• The model itself is being replaced, making component debugging moot
• Inputs are missing entirely, which is a pipeline problem first

Steps

1. Export the account's stored score with every component value and the run timestamp. Verify: the system's claimed components are captured as-is
2. Pull the raw inputs for the same window: usage, tickets, logins, billing status. Verify: raw inputs are assembled independent of the scoring pipeline
3. Recompute each component and the total by hand from the raw inputs. Verify: your recomputed components form the reference truth
4. Diff stored versus recomputed components and trace the first mismatch to its transform or source. Verify: the faulty transform, stale input, or wrong window is named
5. Fix it, re-score the affected population, and publish a short note on which accounts changed and why. Verify: corrected scores ship with an audit trail, not silently

Variant phrasings

health score wrong for one account

A related phrasing for the same underlying problem.

recompute health score manually

A related phrasing for the same underlying problem.

scoring formula component mismatch

A related phrasing for the same underlying problem.

Compatibility: Compatibility: warehouse-based health scoring (dbt models or scheduled SQL) writing to CRM or CS platforms, current versions.

Why it happens

Scoring pipelines stack transforms until nobody can say which layer lied. A single-account hand computation collapses the stack: with the formula fixed and inputs raw, each stored component either matches arithmetic or exposes its layer as the defect.

Edge cases / pitfalls

• Window mismatches (calendar month versus trailing 30 days) are the most common silent error. Align windows before comparing values.
• A component can be right but weighted from an old config version. Record the config version each run used.
• Hand recomputation must use production raw data, not the dashboard's rounded display values.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_MDu0rOb7IO7vvWyyvhzIWw

Resolved from the public thread: https://vectle.com/posts/pst_MDu0rOb7IO7vvWyyvhzIWw

Thread query: customer health score calculation wrong: how to debug the scoring formula

TL;DR: Do not debate the score, recompute it. Take one wrong account, pull every raw input, apply the published formula by hand or in a notebook, and compare component by component with what the system stored. The first component that disagrees is the bug; everything downstream of it is a symptom.