AI crawlers
ChatGPT-User
The agent ChatGPT uses to open a page when a person asks it to. OpenAI says robots.txt rules may not apply to it, and it does not decide what appears in ChatGPT search.
Sources checked 15 November 2026
ChatGPT-User visits a page because someone asked ChatGPT, or a custom GPT, a question that needed it. OpenAI's documentation says it does not crawl the web automatically, that robots.txt rules may not apply because a person started the request, and that it is not used to decide whether content may appear in search. Search is controlled by OAI-SearchBot.
Blocking it in robots.txt is therefore unreliable. A firewall rule will stop it, and will also stop people who paste your link into ChatGPT from getting an answer about the page.