Paste your robots.txt. This reads it the way Google's parser actually does, and
tells you two different things for each of 18 crawlers: whether it is allowed,
and whether you ever decided that — or inherited it from a wildcard, or from a
rule you wrote about a different crawler entirely.
It runs entirely in your browser. Nothing is uploaded, stored, or sent anywhere — view source and check. It is verified cell-for-cell against Protego, the RFC 9309 parser behind our 25,000-domain study, so it cannot quietly disagree with the numbers we published.
| Crawler | Verdict at / | Where the rule comes from |
|---|
Verdict at / means whether the crawler may fetch your site root.
A crawler can be allowed at / and still blocked from individual paths — this tool
reports the site-wide posture, which is the one almost nobody has checked.
A robots.txt is one deploy away from a rule nobody meant to ship. In our scan of the top 25,000 domains, 93.8% of sites blocking GPTBot by name had never written any rule for OpenAI's search crawler — not a decision, a default nobody revisited. Driftwitness watches files like this one and tells you the day they change.