HTTP clients commonly send a user-agent string, and they choose its contents. The header is optional. A scraper can call itself GPTBot as easily as OpenAI's crawler can. To know whether a request really came from the operator it names, you need evidence the client can't simply type.
A user agent is a claim
Bot names like GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot, Googlebot, bingbot and Applebot are useful. They tell you what the client says it is, and honest crawlers use them consistently. But they cost nothing to fake. Treat a recognized name as a claim until something else supports it.
Check 1: the operator's published networks
Several major operators publish the IP ranges their crawlers use, as machine-readable JSON files:
- OpenAI publishes separate ranges for GPTBot, OAI-SearchBot and ChatGPT-User.
- Perplexity publishes ranges for PerplexityBot and Perplexity-User.
- Google publishes ranges for its common crawlers and its user-triggered fetchers.
- Microsoft Bing publishes ranges for bingbot.
- Apple publishes ranges for Applebot.
If a request claims to be GPTBot and its source address falls inside OpenAI's published GPTBot ranges, the claim has real support. Fetch these lists fresh. They change, and an old copy can wrongly reject a genuine crawler.
Check 2: reverse-and-forward DNS
Google, Bing and Apple also document a DNS-based check:
- Look up the hostname for the source IP (a reverse DNS lookup).
- Confirm the hostname matches the operator's documented domains with an exact domain or dot-delimited subdomain boundary. Google documents
googlebot.comandgoogle.comfor different crawler types; Bing usessearch.msn.comand Apple usesapplebot.apple.com. A lookalike such asevilgooglebot.comdoes not matchgooglebot.com. - Look up that hostname's IP addresses (a forward DNS lookup) and confirm the original IP is among them.
Both directions matter. Whoever controls an IP address can make its reverse record say almost anything, but they can't make the operator's own domain resolve back to their address.
Where verification goes wrong
- Proxies and CDNs. Behind a CDN or load balancer, the address your server sees may belong to the proxy. You need the original client address from a header only your trusted proxy can set. Otherwise you are verifying the wrong machine.
- A mismatch isn't proof of spoofing. A failed check can come from proxy configuration, an outdated list or a product that uses different infrastructure. Label it "not verified", not "fake".
- Shared cloud ranges prove nothing. An address in a large cloud provider's range identifies the network provider, not the customer or operator sending the request.
- The network isn't the purpose. A verified OpenAI address supports the operator. It doesn't tell you which model is involved, why the page was fetched or whether a person asked for it.
- User-triggered fetches are different.
ChatGPT-UserandPerplexity-Userusually fetch a page because someone asked an assistant about it. They aren't training crawlers, so report them separately.
What these checks can't see
AI agents that drive an ordinary browser usually send a normal browser user agent and come from ordinary networks. Name matching and network checks won't identify them. Unless other signals catch them, they can look like human visitors, so treat any human count as an estimate.
How PulseAI Analytics handles it
PulseAI collectors match requests against a reviewed catalog of 227 agent and crawler names. For 18 supported names from OpenAI, Perplexity, Google, Bing and Apple, the collector asks the PulseAI server to check the original client address against the official networks, with reverse-and-forward DNS where the operator supports it. A successful check issues a receipt tied to that specific request. Only then is it counted as a verified operator. ChatGPT's browser agent signs its requests, so those can be checked cryptographically. Everything else stays a claim, is flagged as suspected automation, or is counted as estimated human. See the verification setup or explore the demo.