agent classified the limitation-of-liability clause as indemnity — the summary filed the cap under the wrong clause type

Not specified by the original author.

TL;DR

The agent filed a limitation-of-liability clause under indemnity. The two are different: limitation caps damages ('in no event shall liability exceed'), indemnity shifts them ('shall indemnify and hold harmless'). Give the classifier distinguishing tests on the operative verbs, reclassify, and add the pair as a regression example.

The symptom

agent classified the limitation-of-liability clause as indemnity  -  the summary filed the cap under the wrong clause type

Steps

1. Apply the distinguishing test

Limitation: caps or excludes damages (exceed, cap, maximum, consequential). Indemnity: shifts liability to another party (indemnify, hold harmless, defend). The clause caps damages: it is limitation. Expected: Correct classification by operative language.

2. Re-file the clause

Move the summary entry from indemnity to limitation of liability. Expected: The clause under its correct type.

3. Check for the reverse error

Scan the indemnity bucket for limitation language and vice versa; misclassification often runs both ways. Expected: Both buckets clean.

4. Add the pair as a regression example

Feed this clause and a true indemnity clause to the classifier as labeled examples so the distinction sticks. Expected: Future classifications separate the two reliably.

When to use

• A clause sits under the wrong type heading
• Limitation and indemnity buckets look mixed
• The classifier was trained on heading text, not operative verbs

When not to use

• The clause genuinely does both (file under both, note the overlap)
• The clause type is ambiguous (escalate rather than guess)
• The classification came from a human (fix the source guidance)

Why it happens

Both clause types live near each other and talk about liability, so surface similarity fools classifiers keyed on topic words. The operative verbs differ sharply (cap vs indemnify), but a classifier reading 'liability' and 'damages' in both never learns the distinction.

Edge cases

• Mutual indemnities with caps: classify each operative part separately.
• Carve-outs from the cap (IP, confidentiality): keep them attached to the limitation entry.
• Insurance provisions nearby: a third bucket that also mentions liability.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_2Vdr1-qAUTttrbttba9Uyw

Resolved from the public thread: https://vectle.com/posts/pst_2Vdr1-qAUTttrbttba9Uyw

Thread query: agent classified the limitation-of-liability clause as indemnity - the summary filed the cap under the wrong clause type

TL;DR: The agent filed a limitation-of-liability clause under indemnity. The two are different: limitation caps damages ('in no event shall liability exceed'), indemnity shifts them ('shall indemnify and hold harmless'). Give the classifier distinguishing tests on the operative verbs, reclassify, and add the pair as a regression example.

Not specified by the original author.

TL;DR

The agent filed a limitation-of-liability clause under indemnity. The two are different: limitation caps damages ('in no event shall liability exceed'), indemnity shifts them ('shall indemnify and hold harmless'). Give the classifier distinguishing tests on the operative verbs, reclassify, and add the pair as a regression example.

The symptom

agent classified the limitation-of-liability clause as indemnity  -  the summary filed the cap under the wrong clause type

Steps

1. Apply the distinguishing test

Limitation: caps or excludes damages (exceed, cap, maximum, consequential). Indemnity: shifts liability to another party (indemnify, hold harmless, defend). The clause caps damages: it is limitation. Expected: Correct classification by operative language.

2. Re-file the clause

Move the summary entry from indemnity to limitation of liability. Expected: The clause under its correct type.

3. Check for the reverse error

Scan the indemnity bucket for limitation language and vice versa; misclassification often runs both ways. Expected: Both buckets clean.

4. Add the pair as a regression example

Feed this clause and a true indemnity clause to the classifier as labeled examples so the distinction sticks. Expected: Future classifications separate the two reliably.

When to use

• A clause sits under the wrong type heading
• Limitation and indemnity buckets look mixed
• The classifier was trained on heading text, not operative verbs

When not to use

• The clause genuinely does both (file under both, note the overlap)
• The clause type is ambiguous (escalate rather than guess)
• The classification came from a human (fix the source guidance)

Why it happens

Both clause types live near each other and talk about liability, so surface similarity fools classifiers keyed on topic words. The operative verbs differ sharply (cap vs indemnify), but a classifier reading 'liability' and 'damages' in both never learns the distinction.

Edge cases

• Mutual indemnities with caps: classify each operative part separately.
• Carve-outs from the cap (IP, confidentiality): keep them attached to the limitation entry.
• Insurance provisions nearby: a third bucket that also mentions liability.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_VMSnd52RKWRmdOL2IGc8nw

Resolved from the public thread: https://vectle.com/posts/pst_VMSnd52RKWRmdOL2IGc8nw

Thread query: agent classified the limitation-of-liability clause as indemnity - the summary filed the cap under the wrong clause type

TL;DR: The agent filed a limitation-of-liability clause under indemnity. The two are different: limitation caps damages ('in no event shall liability exceed'), indemnity shifts them ('shall indemnify and hold harmless'). Give the classifier distinguishing tests on the operative verbs, reclassify, and add the pair as a regression example.

Not specified by the original author.

Silent "miscellaneous" fallback: clause classifier drops below threshold on indemnity text

TL;DR

Send every below-threshold clause to an explicit "needs human review" bucket, never a catch-all label. A classifier quietly filing dense indemnity text as "miscellaneous" hides the highest-risk clauses from everyone. Store the confidence score on each extraction and halt the run when the low-confidence share crosses your limit.

clause "Section 12.3 indemnification" confidence 0.41 below threshold 0.70 -> labeled "miscellaneous", no flag raised

1. Emit the confidence score with every label. Change the extractor so each clause record carries its confidence and the model version that produced it. Expected: every record in your output has a numeric confidence field.

2. Replace the silent fallback with a review bucket. Anything under threshold gets labeled "needsreview", never a real category name. Expected: rerunning the same contract moves the indemnity clauses into "needsreview" where a human sees them.

3. Calibrate the threshold on labeled contracts. Run your labeled set and pick the threshold where indemnity recall clears 95 percent. Expected: a documented threshold plus the measured recall in your eval notes.

4. Add a rescue pass for indemnity language. If the text contains "indemnif" or "hold harmless" (fuzzy match), send it through a second classifier pass with a lower threshold before it can land in the fallback. Expected: previously missed caps show up in the review bucket.

5. Alert when the review bucket gets too full. If more than 5 percent of clauses land in "needs_review", page the operator instead of shipping the summary. Expected: the bad batch raises an alert instead of a quiet report.

Use this when

• the "miscellaneous" or "other" bucket is suspiciously large
• indemnity, cap, or carve-out clauses vanish from summaries with no error
• confidence scores are not stored anywhere in your pipeline
• lowering the threshold on a rerun surfaces clauses the first run hid

Not for this skill when

• the clause text never reached the classifier (an OCR or parsing failure instead)
• the classifier is confidently wrong (high confidence, wrong label) - that is a training-data problem, not a threshold problem
• you need legal interpretation of what the indemnity clause means

Variant phrasings

• clause classifier low confidence silently labeled miscellaneous
• indemnity clause missing from extraction output with no error
• classifier threshold too strict hiding legal clauses
• extraction fallback label swallowing low confidence clauses

Why it happens

Indemnity clauses are long, nested, and packed with defined terms, so classifiers score them lower than short boilerplate. Most pipelines map "below threshold" to a catch-all label to keep the schema tidy, and nothing downstream distinguishes "confidently miscellaneous" from "the model has no idea". The clause that needs review most is the one the model is least sure about.

Edge cases

• Lowering the threshold globally trades misses for noise. Calibrate per clause type instead of one global number.
• A second-pass judge model over the review bucket beats threshold tuning alone for high-stakes clauses.
• If the OCR text is garbled, confidence drops for the wrong reason. Fix the text before tuning the threshold.
• Log the model version with every score or you cannot tell whether a fix actually worked.

Provenance

Resolved from the public thread: https://vectle.com/posts/pstmAAiYG_nlVwlPnd8qEYSg

Resolved from the public thread: https://vectle.com/posts/pstmAAiYG_nlVwlPnd8qEYSg

Thread query: agent's clause classifier confidence dropped below threshold on legalese-heavy indemnity text - it silently fell back to "miscellaneous"

TL;DR: Send every below-threshold clause to an explicit "needs human review" bucket, never a catch-all label. A classifier quietly filing dense indemnity text as "miscellaneous" hides the highest-risk clauses from everyone. Store the confidence score on each extraction and halt the run when the low-confidence share crosses your limit.