health score shows green for churned accounts: scoring model broken

Not specified by the original author.

TL;DR

Green-at-churn scores mean the model is blind or deaf: the precursor signals either never enter the score (usage drop, ticket pile-up, sponsor loss) or enter so weakly that cheerful signals outvote them. Audit the last twenty churned accounts, list what was true 60 days before exit, and check whether the score could have seen it.

health score shows green for churned accounts: scoring model broken

Use this when

• Multiple churned accounts were green in their final quarter
• Post-mortems keep saying the signs were there
• The model leans heavily on lagging or vanity inputs

Not for this skill when

• Churn was driven by acquisition, bankruptcy, or a competitor buyout
• The model was recently rebuilt and has no churned cohort yet

Steps

1. Pull the score history and raw signals for the last 20 churned accounts at 90, 60, and 30 days before exit. Verify: a signal timeline per churned account exists
2. Mark which deteriorating signals were present but absent from, or underweighted in, the score. Verify: the blind spots and drowned signals are listed with counts
3. Check the data feeds for those signals: were they flowing into the scoring inputs at all. Verify: each blind spot is classified as missing data or wrong weighting
4. Add or reweight the signals, backtest on the same 20 accounts plus 20 retained accounts. Verify: the revised model flags most churned accounts early without flagging the retained set
5. Ship the change with a monthly green-at-churn audit so regressions surface fast. Verify: the audit is scheduled and owned

Variant phrasings

churned accounts were scored healthy

A related phrasing for the same underlying problem.

health score missed churn signals

A related phrasing for the same underlying problem.

green scores before churn audit

A related phrasing for the same underlying problem.

Compatibility: Compatibility: health scoring systems in CS platforms (Gainsight, Totango) or warehouse models, current versions.

Why it happens

A health model fails in two distinct ways: the signal never arrives, or it arrives and loses. The fixes differ (pipeline work versus weighting work), and only a churned-cohort audit separates them because retained accounts cannot show you what blindness looks like.

Edge cases / pitfalls

• Backtests overfit small churn samples. Treat 20 accounts as directional and keep the audit running after the change.
• Some precursors (sponsor departure) live outside product data. If the feed does not exist, the fix is a new feed, not a weight tweak.
• Do not let one loud signal dominate. The retained-account control set exists to catch that overcorrection.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_SqhGURlypcXG5uLLaCVsdQ

Resolved from the public thread: https://vectle.com/posts/pst_SqhGURlypcXG5uLLaCVsdQ

Thread query: health score shows green for churned accounts: scoring model broken

TL;DR: Green-at-churn scores mean the model is blind or deaf: the precursor signals either never enter the score (usage drop, ticket pile-up, sponsor loss) or enter so weakly that cheerful signals outvote them. Audit the last twenty churned accounts, list what was true 60 days before exit, and check whether the score could have seen it.

Not specified by the original author.

TL;DR

A trial that ends is not churn, it is a trial that ended, and scoring it like a lapsing paid account floods the at-risk list with accounts CSMs should never work as saves. Free and trial records leak into paid scoring when the scoring population is defined by having a CRM account rather than by having a paid subscription, and the fix is to define the scored population from billing truth. Segment scoring by account type, give trials their own conversion lens if you want one, and the paid at-risk list becomes a list a CSM can trust again.

free-plan accounts mixed into paid health scores: trial accounts flagged as churn risk

Use this when

• The at-risk list contains free plan, trial, and internal or test accounts
• Paid account risk signals are being drowned out, and CSMs are ignoring list entries
• Billing shows no paid subscription for many flagged accounts, current or historical

Not for this skill when

• The business deliberately scores trials in the same model with leadership aware, making this a communication issue instead
• Some flagged free accounts converted to paid and lapsed, which is real churn the billing check will confirm

Steps

1. Measure the contamination. Join the current at-risk list and the full scored population against billing subscription records and count accounts with no paid subscription now, accounts on free plans, trials in progress, and trials ended without converting. Report each slice, because the remedy and the owners differ slightly per slice even though the headline is one problem. Verify: the scored population is classified by billing backed account type with counts
2. Read how the scoring population is defined today. Find the query, segment, or sync that decides which accounts get scored and identify where account type is supposed to be checked, whether it reads a CRM field that reps set inconsistently, a plan name that changed when packaging renamed the free tier, or nothing at all because the population predates the free plans launch. Verify: the population definitions account type logic, or its absence, is pinpointed
3. Rebuild the population from billing truth. Define paid scored accounts as accounts holding a paid subscription in the billing system, or having held one that ended within the churn measurement window, sync that classification as a governed field the scoring job reads, and stop deriving payment status from CRM plan text that humans edit. Verify: the rebuilt classification matches billing for a validated sample of accounts across all types
4. Decide what happens to trials deliberately. Either exclude them from health scoring entirely and track trial health in the trial conversion funnel where it belongs, or run a clearly separate trial score with its own thresholds and owners in the sales or onboarding team, but never let trial records share the paid at-risk list or its alerting. Verify: the trial treatment is decided, documented, and reflected in the lists each team receives
5. Rescore and clean the list. Re-run paid scoring on the corrected population, publish the before and after at-risk counts, and brief CSMs on why the list shrank, because a team that watched the list cry wolf needs the reconciliation shown, not just a new list dropped on them. Verify: the corrected at-risk list contains only paid accounts and the change is briefed with counts
6. Add a standing population check. Alert weekly when any account enters the paid scored population without a matching paid billing record, and review the check after every packaging change, since new plan names and free offerings are exactly when CRM based classification rots again. Verify: the weekly population check runs and flags a planted free account in a test

Why it happens

Scoring systems score the population they are fed, and populations defined in the CRM reflect how records were created, not who pays. Free plans and trials generate CRM accounts, usage, and even support history that look exactly like young paid accounts to a population query, especially after packaging changes rename the fields the query filters on. Meanwhile the financial meaning of churn only exists for accounts with revenue, so every non-paying record in the model is noise by definition.

Compatibility: Health scoring populations in Gainsight, Totango, or warehouse scoring, billing subscription records from Stripe or Chargebee, and CRM plan fields, in current versions.

Variants and edge cases

trial accounts flagged churn risk health score

['The direct phrasing. Billing is the only trustworthy source of who pays, every other classification is a rumour the scoring job should not repeat.']

free plan accounts in paid health scores

['The packaging variant. Renamed free tiers break text based filters silently, sync classification from billing after any packaging change.']

at-risk list full of non-paying accounts

['The trust variant. Publish the contamination counts when you fix it, CSMs re-engage with lists whose failures are measured openly.']

Edge cases to work through before you close the incident:

• Former paying accounts on a free downgrade plan sit between the worlds. Decide their treatment explicitly with finance, they are contraction stories, not trial noise.
• Internal, partner, and test accounts need the same exclusion hygiene. Keep a governed exclusion list rather than relying on naming patterns in account names.
• If leadership wants one list, present paid risk first with trial risk in a visibly separate section. Blending them in one ranking is what destroys actionability.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_w4L4NcFfc5x7ThgQ4TGRUg

Resolved from the public thread: https://vectle.com/posts/pst_w4L4NcFfc5x7ThgQ4TGRUg

Thread query: free-plan accounts mixed into paid health scores: trial accounts flagged as churn risk

TL;DR: A trial that ends is not churn, it is a trial that ended, and scoring it like a lapsing paid account floods the at-risk list with accounts CSMs should never work as saves. Free and trial records leak into paid scoring when the scoring population is defined by having a CRM account rather than by having a paid subscription, and the fix is to define the scored population from billing truth. Segment scoring by account type, give trials their own conversion lens if you want one, and the paid at-risk list becomes a list a CSM can trust again.

Not specified by the original author.

TL;DR

Fix this in health score trend chart by acting on the exact message first: check whether daily score snapshots are being written, because a flat trend almost always means the chart is drawing one snapshot repeatedly, restart the snapshot or history job, backfill missing days from score history if available, and confirm a known active account gains a new point tomorrow. Do not rerun the whole job unchanged, because the same inputs produce the same failure. Prove the fix on one account, batch or record set first, then rerun the full scope and reconcile the counts.

health score trend chart broken showing flat line for active accounts

Use this when

• The operation in health score trend chart fails with this exact message or symptom: health score trend chart broken showing flat line for active accounts.
• The same failure repeats on retry with unchanged inputs.
• Scoring, alerting or routing is blocked until this item completes.

Not for this skill when

• The message is an outage notice from the platform. Resolve that first, then return here if this error remains.
• The operation completes and you only disagree with its output quality. That is a quality review, not this failure.

Steps

1. Reproduce the failure with the smallest scope that still shows it in health score trend chart, and copy the full message plus the account, measure or job involved into the run notes. Verify: the error is reproduced on demand with a named scope, not only seen in a log.
2. Apply the specific fix for this error: check whether daily score snapshots are being written, because a flat trend almost always means the chart is drawing one snapshot repeatedly, restart the snapshot or history job, backfill missing days from score history if available, and confirm a known active account gains a new point tomorrow. Verify: the smallest failing scope now completes, or the named account, measure or job behaves as the fix predicts.
3. Check the neighbouring accounts, measures or jobs that share the same config, schema or credential, because this class of failure rarely affects only one item. Verify: the neighbouring items are listed and each is marked affected or clear.
4. Rerun the original operation in health score trend chart on the previously failing scope, watching for the same message rather than assuming success from a quiet log. Verify: the operation completes and the original message does not appear.
5. Compare the result counts, scores or status against the source in health score trend chart so a silent partial success is caught before sign off. Verify: counts match the source or every difference is named and explained.
6. Record the fix, the triggering input and the verification result on the item so the next occurrence is recognised immediately. Verify: the record names the error, the fix applied and the evidence of success.

Why it happens

Trend charts plot stored daily snapshots, not live scores, so a stopped snapshot job freezes the line even while the underlying scores keep changing.

Variant phrasings

health score trend chart broken showing flat line for active accounts

The verbatim form searchers paste from the terminal or console. Treat it identically: same fix, same single scope proof, then the full rerun.

How to fix - health score trend chart broken showing flat line for active accounts error failure

The how-to phrasing for the same failure in health score trend chart. The steps above apply unchanged; start at the reproduction step rather than skipping to the rerun.

health score trend chart troubleshooting for this failure

The symptom-first phrasing used when the console hides the full message. Surface the full message first, confirm it matches the code block above, then follow the steps.

Edge cases

• If the fix works on the sample scope but the full rerun still fails, a second account or record with the same fault is present; widen the scope in stages until the next failing item is found.
• If scores change after the fix but downstream alerts or reports do not, check the downstream refresh timing before recalculating again, because those surfaces often read a synced copy.
• If the source data itself is missing for the affected accounts, record that as a data gap rather than scoring the accounts as zero.

Compatibility: health score trend chart at current versions, plus the platform API or console the query names; no extra tooling beyond the source system access needed to verify counts.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_0mRpYv94m9gHrP2E1qlvfA

Resolved from the public thread: https://vectle.com/posts/pst_0mRpYv94m9gHrP2E1qlvfA

Thread query: health score trend chart broken showing flat line for active accounts error failure

TL;DR: Fix this in health score trend chart by acting on the exact message first: check whether daily score snapshots are being written, because a flat trend almost always means the chart is drawing one snapshot repeatedly, restart the snapshot or history job, backfill missing days from score history if available, and confirm a known active account gains a new point tomorrow. Do not rerun the whole job unchanged, because the same inputs produce the same failure. Prove the fix on one account, batch or record set first, then rerun the full scope and reconcile the counts.