# SEC EDGAR HTML filing has malformed tables that break parsing

## TL;DR
Old EDGAR HTML filings have malformed tables because they were hand-built or converted from word processors decades ago, with unclosed tags and nested tables that strict parsers choke on. Parse with a lenient HTML parser that recovers from bad markup, and prefer table-aware extraction over raw tag walking. Validate the extracted tables against the filing's own totals before the numbers reach a briefing.

## The error
```text
pandas.errors / lxml.etree.XMLSyntaxError
malformed table in EDGAR HTML filing; parser raised on unclosed tags / nested tables
```

## When this helps
- strict parsers raise on old EDGAR HTML filings
- table extraction returns junk from legacy filings
- building a pipeline over pre-2010 filings
- validating extracted statement tables

## When it doesn't
- the filing is XBRL-era; parse the instance XML instead of the HTML
- tables are fine but numbers mismatch; that is an amendment problem
- you need footnotes; those live outside tables and need text parsing

## Works with
python 3.8+ with pandas, lxml, html5lib. Malformed markup is common in pre-2010 EDGAR HTML.

## Steps
### 1. Parse with a recovering HTML parser instead of a strict one
```python
import pandas as pd
tables = pd.read_html("filing.html", flavor="lxml")
print("lxml tables:", len(tables))
if not tables:
    tables = pd.read_html("filing.html", flavor="html5lib")
    print("html5lib tables:", len(tables))
```
Expected: A table count. lxml is fast and lenient; html5lib follows the browser parsing algorithm and survives the worst legacy markup.

### 2. Inspect the recovered table shapes
```python
import pandas as pd
tables = pd.read_html("filing.html", flavor="html5lib")
for i, t in enumerate(tables[:8]):
    print(i, t.shape)
```
Expected: Shapes per table. Financial statement tables are large and rectangular; tiny or single-column tables are usually layout tables to skip.

### 3. Select the statement tables by content, not position
```python
import pandas as pd
tables = pd.read_html("filing.html", flavor="html5lib")
for i, t in enumerate(tables):
    head = " ".join(str(c) for c in t.iloc[0].tolist()[:6])
    if "revenue" in head.lower() or "assets" in head.lower():
        print("candidate statement table", i, t.shape)
```
Expected: The index of the real financial tables. Position-based selection breaks across filings; content matching survives.

### 4. Validate against the filing's reported totals
```python
import pandas as pd
tables = pd.read_html("filing.html", flavor="html5lib")
for i, t in enumerate(tables):
    txt = t.to_string().lower()
    if "total revenue" in txt:
        print("table", i, "carries total revenue; cross-check the figure")
        break
```
Expected: A located total to cross-check. Malformed markup can drop rows silently, so a totals check catches incomplete extraction.

## Other ways people phrase this
### edgar html malformed table lxml error
lxml's XML parser is strict; its HTML parser or html5lib recovers. Use the right parser for the job.

### pandas read_html fails sec filing
Try the other flavor before concluding the filing has no tables. html5lib is the most forgiving.

### nested tables edgar filing parse
Legacy filings nest layout tables inside statement tables. Content-based selection skips the layout ones.

## Why it happens
Early EDGAR HTML was produced by converters and hand editing with no validation, leaving unclosed tags, misnested tables, and presentational markup. Strict XML parsers reject this; browser-grade HTML parsers recover it the way a browser would. The filing renders fine because browsers are lenient; your parser must be too.

## Edge cases
- html5lib is slow on 200-page filings; use lxml first and fall back only on failure.
- Some filings embed the statements as images inside HTML; then you need OCR, not a better parser.
- Table headers spanning multiple rows need manual repair after extraction.
- Inline XBRL filings have cleaner markup; prefer the instance XML when it exists.

## Provenance

Resolved from the public thread: https://vectle.com/posts/pst_deXzYX8Fg138l8BVjcpjog