Common Crawl

@common_crawl@robot.villas

Common Crawl's crawl-archive releases, web graphs, research, tutorials, and announcements about its free repository of web crawl data.

Source: https://www.commoncrawl.org/blog/rss.xml

Homepage: https://www.commoncrawl.org

Posts

100

Posts

Host- and Domain-Level Web Graphs July, August, and September 2026 #DataScience #WebData #BigData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386439
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386439
Don't have an account? Find a server at joinmastodon.org
Sep 22, 2026
September 2026 Crawl Archive Now Available #Data #Web #Archive
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386447
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386447
Don't have an account? Find a server at joinmastodon.org
Sep 19, 2026
Getting Started with Common Crawl Data on Hugging Face #DataScience #BigData #OpenSource
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386456
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386456
Don't have an account? Find a server at joinmastodon.org
Sep 16, 2026
CommonsDB, ISCC, Spaghetti, and Meatballs #DataScience #WebCrawling #OpenData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386465
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386465
Don't have an account? Find a server at joinmastodon.org
Sep 15, 2026
Web Graph Embeddings: An Experimental Dataset Release #DataScience #WebGraph #OpenData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386476
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386476
Don't have an account? Find a server at joinmastodon.org
Sep 7, 2026
A Content Analysis of llms.txt Files from the July 2026 Crawl Archive #DataScience #WebCrawling #AI
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386482
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386482
Don't have an account? Find a server at joinmastodon.org
Aug 31, 2026
Host- and Domain-Level Web Graphs June, July, and August 2026 #WebData #BigData #OpenData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386491
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386491
Don't have an account? Find a server at joinmastodon.org
Aug 25, 2026
August 2026 Crawl Archive Now Available #Data #WebArchive #OpenData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386502
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386502
Don't have an account? Find a server at joinmastodon.org
Aug 24, 2026
Common Crawl Foundation at ACL 2026 #Data #Web #Research
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386511
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386511
Don't have an account? Find a server at joinmastodon.org
Aug 20, 2026
Announcing the First Stable Release of CC-Downloader #Data #OpenSource #Web
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386521
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386521
Don't have an account? Find a server at joinmastodon.org
Aug 10, 2026
Notes from HTTP Workshop Basel and IETF 126 Vienna #Web #Networking #OpenData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386532
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386532
Don't have an account? Find a server at joinmastodon.org
Aug 3, 2026
July 2026 Crawl Archive Now Available #Data #WebArchive #OpenData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386551
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386551
Don't have an account? Find a server at joinmastodon.org
Jul 28, 2026
Host- and Domain-Level Web Graphs May, June, and July 2026 #WebData #OpenData #Research
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386542
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386542
Don't have an account? Find a server at joinmastodon.org
Jul 28, 2026
Common Crawl Joins Project Tapestry #OpenData #WebArchive #Research
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386561
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386561
Don't have an account? Find a server at joinmastodon.org
Jul 27, 2026
Measuring Crawled Coverage of a Website in Common Crawl #WebData #DataScience #OpenSource
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386572
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386572
Don't have an account? Find a server at joinmastodon.org
Jul 20, 2026
Is one vantage point enough? IPv6 across the top million web hosts #IPv6 #Web #Network
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386582
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386582
Don't have an account? Find a server at joinmastodon.org
Jul 17, 2026
Turning 30,000 Arabic Domains Into a Better Crawl #WebCrawling #DataScience #OpenData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386596
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386596
Don't have an account? Find a server at joinmastodon.org
Jul 3, 2026
Common Crawl Foundation at LREC 2026 #Data #Web #Research
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386605
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386605
Don't have an account? Find a server at joinmastodon.org
Jun 30, 2026
13th Web-as-Corpus Workshop @ EMNLP 2026 #Data #Research #Web
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386613
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386613
Don't have an account? Find a server at joinmastodon.org
Jun 29, 2026
Host- and Domain-Level Web Graphs April, May, and June 2026 #WebData #OpenData #Research
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386624
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386624
Don't have an account? Find a server at joinmastodon.org
Jun 25, 2026
June 2026 Crawl Archive Now Available #Data #Web #Archive
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386636
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386636
Don't have an account? Find a server at joinmastodon.org
Jun 22, 2026
CommonLID Update: New Tools, Growing Impact #Data #Web #OpenData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386645
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386645
Don't have an account? Find a server at joinmastodon.org
Jun 16, 2026
Common Crawl Foundation at IIPC-WAC 2026 #WebArchive #Data #OpenData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386655
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386655
Don't have an account? Find a server at joinmastodon.org
Jun 10, 2026
The Columnar Index Is Now the URL Index #Data #Web #OpenData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386668
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386668
Don't have an account? Find a server at joinmastodon.org
Jun 3, 2026
Introducing the AI Visibility Audit #AI #WebData #Research
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386679
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386679
Don't have an account? Find a server at joinmastodon.org
Jun 1, 2026
Host- and Domain-Level Web Graphs March, April, and May 2026 #Data #Web #Archive
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386684
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386684
Don't have an account? Find a server at joinmastodon.org
May 29, 2026
May 2026 Crawl Archive Now Available #Data #WebArchive #OpenData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386696
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386696
Don't have an account? Find a server at joinmastodon.org
May 25, 2026
April 2026 Crawl Archive Now Available in a Hugging Face Storage Bucket #Data #WebArchive #OpenData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386707
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386707
Don't have an account? Find a server at joinmastodon.org
May 20, 2026
You can now build directly on Common Crawl from the browser #Data #Web #OpenData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386719
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386719
Don't have an account? Find a server at joinmastodon.org
May 6, 2026
Host- and Domain-Level Web Graphs February, March, and April 2026 #Data #Web #Archive
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386729
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386729
Don't have an account? Find a server at joinmastodon.org
Apr 30, 2026
April 2026 Crawl Archive Now Available #Data #WebArchive #OpenData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386737
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386737
Don't have an account? Find a server at joinmastodon.org
Apr 28, 2026
April 2026 Common Crawl Newsletter #Data #Web #OpenSource
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386748
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386748
Don't have an account? Find a server at joinmastodon.org
Apr 6, 2026
Announcing a Change to Common Crawl Dataset Size Reporting #Data #Web #OpenData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386759
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386759
Don't have an account? Find a server at joinmastodon.org
Apr 1, 2026
Host- and Domain-Level Web Graphs January, February, and March 2026 #WebData #OpenData #Research
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386770
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386770
Don't have an account? Find a server at joinmastodon.org
Mar 24, 2026
March 2026 Crawl Archive Now Available #Data #WebArchive #OpenData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386780
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386780
Don't have an account? Find a server at joinmastodon.org
Mar 19, 2026
IPv6 Adoption Across the Top 100K Web Hosts #IPv6 #Web #Data
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386792
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386792
Don't have an account? Find a server at joinmastodon.org
Mar 16, 2026
Web Graph Statistics Gets a Proper Upgrade #Data #Web #OpenData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386798
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386798
Don't have an account? Find a server at joinmastodon.org
Mar 6, 2026
Measuring Web Accessibility from Crawl Archives #Web #Accessibility #Data
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386807
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386807
Don't have an account? Find a server at joinmastodon.org
Mar 2, 2026
Announcing the Whirlwind Tour of Common Crawl's Datasets Using Java #Data #Web #Java
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386822
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386822
Don't have an account? Find a server at joinmastodon.org
Feb 26, 2026
Host- and Domain-Level Web Graphs December 2025 and January/February 2026 #DataScience #WebCrawling #BigData
Boost from your server
Enter the server your account is on. The post opens there so you can boost it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386832
Don't have an account? Find a server at joinmastodon.org
Favorite from your server
Enter the server your account is on. The post opens there so you can favorite it.
Your server
or
Or paste this post's address into your app's search:https://robot.villas/users/common_crawl/posts/22386832
Don't have an account? Find a server at joinmastodon.org
Feb 24, 2026