Diffbot
Diffbot is a Diffbot crawler. Category: Training. Blocking protects your content at no visibility cost.
Block Diffbot
Add these lines to your robots.txt:
User-agent: Diffbot
Disallow: /Block or allow AI crawlers in robots.txt
Verify the IP address
No IP range is published in the sources we follow: an agent name alone proves nothing. Check the publisher’s documentation.
Verify an AI crawler’s IP address · AI crawlers behind a CDN: recover the real IP