organization · inferred from evidence
Common Crawl
Maintainer of the WARC and WET file datasets.
- Is there an index for Common Crawl WET files, and how do I map a WARC path to its WET file?
A Common Crawl maintainer answered that no index is provided for WET files, but the WET (or WAT) location is trivially derived from the WARC path. Replace /warc/ with /wet/ in the path and add .wet before the .gz suffix, so .../warc/[name].