Studies
Numbers we measured ourselves, with the method and the data. Use them, quote them, or run the script again and tell us if you get something different.
- 14 November 2026
Which AI crawlers do large websites block?
robots.txt on 60 well-known sites: who blocks training, who blocks search, who blocks nothing.
Open - 14 November 2026
How many large websites have an llms.txt?
llms.txt on 60 well-known sites: who has one, and whether it follows the format.
Open - 14 November 2026
What do Google's AI Overviews cite?
356 pages Google's AI Overviews cited for 48 buyer questions, sorted by type.
Open
What these are, and are not
Small checks of public files on well-known sites, run with the same code as our free tools. Each page gives the date, how it was done, what was left out, and the data for every site so you can check it.
They are not surveys of the web. Each covers 60 sites we chose, and says so. We do not have a customer dataset yet, and when we do, results from real sites will be published here the same way: with the sample, the method and the cases that did not go well.