Common Crawl Foundation

non-profit

Verified

https://commoncrawl.org

AI & ML interests

Crawled data and metadata

Recent Activity

malteos updated a bucket 20 days ago

commoncrawl/commoncrawl

lfoppiano new activity 20 days ago

commoncrawl/statistics:Update data (CC-MAIN-2026-43)

malteos updated a dataset about 1 month ago

commoncrawl/commonlid-cache_commonlid_nano

View all activity

updated a bucket 20 days ago

commoncrawl/commoncrawl

in commoncrawl/statistics 20 days ago

Update data (CC-MAIN-2026-43)

#4 opened 20 days ago by

updated a dataset about 1 month ago

commoncrawl/commonlid-cache_commonlid_nano

Viewer • Updated Oct 7 • 1.51k • 64 • 2

updated a bucket about 2 months ago

commoncrawl/test-bucket

in commoncrawl/commonlid-results about 2 months ago

Add py3langid results on commonlid_nano

#9 opened about 2 months ago by

Refresh results from results

#6 opened 2 months ago by

updated a dataset about 2 months ago

commoncrawl/CommonLID

Viewer • Updated Sep 22 • 373k • 844 • 58

in commoncrawl/statistics about 2 months ago

Update data (CC-MAIN-2026-39)

#3 opened about 2 months ago by

authored 3 papers about 2 months ago

Table Understanding and (Multimodal) LLMs: A Cross-Domain Case Study on Scientific vs. Non-Scientific Data

Paper • 2507.00152 • Published Jun 30, 2025 • 1

SciLaD: A Large-Scale, Transparent, Reproducible Dataset for Natural Scientific Language Processing

Paper • 2512.11192 • Published Dec 12, 2025 • 1

CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data

Paper • 2601.18026 • Published Jan 25

updated a bucket about 2 months ago

commoncrawl/warc2zip-examples

in commoncrawl/commonlid-results about 2 months ago

Add GoogleTranslate-v3 results on commonlid_nano

#8 opened about 2 months ago by

Rename GoogleTranslate to GoogleTranslate-v2

#7 opened about 2 months ago by

updated 2 datasets about 2 months ago

commoncrawl/host-index-testing-v2

Viewer • Updated Sep 15 • 9B • 4.67k • 1

commoncrawl/statistics

Viewer • Updated 20 days ago • 658k • 948 • 31

in commoncrawl/host-index-testing-v2 about 2 months ago

Update README.md

#1 opened about 2 months ago by

updated 2 Spaces 2 months ago

Web Graph Embeddings

What is actually inside llms.txt

Analyze llms.txt files for content insights

in commoncrawl/commonlid-results 2 months ago

Adding results from GoogleTranslate for commonlid_nano

#5 opened 2 months ago by