organization · inferred from evidence
Common Crawl
Maintainer of the WARC and WET file datasets.
- Is there an index for Common Crawl WET files, and how do I map a WARC path to its WET file?
They had found the WARC-format domain index at index.commoncrawl.org and wanted to know whether any index exists for the .WET plain-text format files too. If not, they asked how to convert a WARC object address (an s3://commoncrawl/crawl-da