OAI-SearchBot

OAI-SearchBot is the OpenAI crawler that finds and reads pages so they can appear as sources in ChatGPT search answers; blocking it in robots.txt can keep your pages out of those answers.

Also searched as  OAI SearchBot · ChatGPT search crawler · OpenAI search bot

Updated

What it means for a Dubai business

If you want your own pages to be among the links ChatGPT shows when it searches the web, OAI-SearchBot has to be able to read them. OpenAI’s own guidance says sites that block it may not be shown in ChatGPT search answers, though they can still appear as a plain link in some cases.

For most Dubai businesses that sell to buyers who ask ChatGPT, there is no reason to block it. It reads public pages only.

Example

A real estate brokerage in Dubai Marina writes an area guide answering what service charges to expect in the Marina. With OAI-SearchBot allowed, ChatGPT search can read that page and cite it when a buyer asks the same question.

Common mistake

Checking robots.txt and forgetting the firewall. A CDN or security plugin can refuse the bot at the server even when robots.txt allows it. If your pages never appear as sources, ask whoever runs your hosting whether AI bots are being challenged or blocked.

Related terms

Guides for your field

From the journal

Every term, A to Z