OAI-SearchBot

OpenAI's crawler documentation says sites that opt out of OAI-SearchBot will not be shown in ChatGPT search answers, though they can still appear as navigational links. It also says a robots.txt change takes about 24 hours to register.

It is separate from GPTBot, which collects training data, and OpenAI says each setting is independent. You can allow OAI-SearchBot and block GPTBot, which is what most businesses asking the question want.

8 of the 56 large sites we checked on 14 November 2026 block it, all of them large publishers and platforms that chose to. If your robots.txt gives OAI-SearchBot a group of its own, it ignores your rules under User-agent: *, so repeat any private paths in its group.