xml.etree.ElementTree.ParseError invalid token in sitemap.xml
Fixes ElementTree ParseError invalid token on sitemap.xml: finding illegal characters in raw bytes and fixing or pre-cleaning them. Use when a downloaded sitemap fails strict XML parsing. Not for schema validation issues or sitemaps you generate yourself (fix the generator).
TL;DR
The sitemap contains characters that are illegal in raw XML, almost always a bare & in a URL or a stray control character. Fetch the raw bytes, find the offending character, and either fix the source sitemap or pre-clean the text before parsing.
xml.etree.ElementTree.ParseError invalid token in sitemap.xmlUse this when
- ElementTree fails on a downloaded sitemap.xml.
- The error names "invalid token" with a line and column.
- A sitemap that used to parse now fails.
Not for this skill when
- The XML is well-formed but the schema is wrong. That is a validation problem, not a parse problem.
- You control the sitemap generator. Fix the generator instead of cleaning output.
Steps
- Save the raw response bytes to a file before parsing. Verify: the file size is nonzero.
- Open the file at the line and column from the error message. Verify: you can see the exact offending character.
- Look for bare
&in URLs (it must be written in its escaped form), unescaped quotes in attributes, or non-UTF8 bytes. Verify: you can point at the bad character. - If you own the source, fix the generator to escape URLs properly. Verify: a fresh download parses cleanly.
- If you do not own the source, replace the bad characters in a pre-pass, then parse the cleaned text with ElementTree. Verify: parsing succeeds and the URL count looks right.
Variant phrasings
sitemap xml parse error python
General search for the same failure.
elementtree invalid token
Parser-level search.
sitemap contains invalid xml character
Symptom-level search.
Compatibility: Python 3.x xml.etree.ElementTree. The strict parser rejects invalid tokens with no recovery mode; lxml offers a recover option if you can add the dependency.
Why it happens
Sitemaps are machine-generated, and generators sometimes concatenate raw URLs containing & or control characters into XML without escaping. XML parsers are strict by design: one illegal character anywhere fails the whole document, and the error points at the first offense only.
Edge cases / pitfalls
- A byte-order mark at the start of the file can confuse naive decoders. Decode as utf-8-sig before parsing.
- After fixing the first invalid token, parse again. There is often a second one further down.
- Pre-cleaning with string replacement is a workaround, not a fix. Log the bad URLs and report them to the sitemap owner.
- Huge sitemaps (50k URLs) should be parsed with iterparse, not fromstring, to bound memory.
Provenance
Resolved from the public thread: https://vectle.com/posts/pst_th0v5wWXiBVCZ7qHpRP1ig