This repository contains up-to-date lists of IP addresses of known bots and crawlers, useful for whitelisting or filtering network traffic. They can also be used as blacklists.
Do you have any questions or want to receive notifications about important changes or new features in my repositories? Join my Discord server! If you don't use Discord, you can also open an issue on GitHub.
The project is released under the MIT license — you can do whatever you want with it.
If you like this repository, leave a star ⭐. Thank you!
Lists are updated every 6 hours.
| Service | Sources | Downloads |
|---|---|---|
| AhrefsBot | api.ahrefs.com | TXT • CSV • JSON |
| BetterStack | uptime.betterstack.com | TXT • CSV • JSON |
| BingBot | www.bing.com | TXT • CSV • JSON |
| Bunny CDN | IPv4 & IPv6 | TXT • CSV • JSON |
| Censys | RIPEstat & RADB & ARIN | TXT • CSV • JSON |
| Cloudflare | IPv4 & IPv6 | TXT • CSV • JSON |
| DuckDuckBot | duckduckgo.com | TXT • CSV • JSON |
| FacebookBot | RIPEstat & RADB & ARIN | TXT • CSV • JSON |
| Google Special Crawlers | developers.google.com | TXT • CSV • JSON |
| Google User-Triggered Fetchers | Fetchers & Google | TXT • CSV • JSON |
| GoogleBot | developers.google.com | TXT • CSV • JSON |
| Kagi | Custom lists | TXT • CSV • JSON |
| Modat Scanner | scanner.modat.io | TXT • CSV • JSON |
| NASK PL | RIPEstat & RADB | TXT • CSV • JSON |
| OpenAI | GitHub | TXT • CSV • JSON |
| Palo Alto Networks | RADB & ARIN | TXT • CSV • JSON |
| PingdomBot | IPv4 & IPv6 | TXT • CSV • JSON |
| Pulsetic | IPv4 & IPv6 | TXT • CSV • JSON |
| RSS API | rssapi.net | TXT • CSV • JSON |
| Semrush | RIPEstat & RADB | TXT • CSV • JSON |
| Shodan | Custom lists | TXT • CSV • JSON |
| Stripe | stripe.com | TXT • CSV • JSON |
| TelegramBot | core.telegram.org | TXT • CSV • JSON |
| UptimeRobot | uptimerobot.com | TXT • CSV • JSON |
| WebPageTest Bot | www.webpagetest.org | TXT • CSV • JSON |
| YandexBot | yandex.com | TXT • CSV • JSON |
https://raw.githubusercontent.com/sefinek/known-bots-ip-whitelist/main/lists/all-safe-ips.txt
curl -fsS --max-redirs 0 -o all-safe-ips.txt https://raw.githubusercontent.com/sefinek/known-bots-ip-whitelist/main/lists/all-safe-ips.txthttps://raw.githubusercontent.com/sefinek/known-bots-ip-whitelist/main/lists/all-safe-ips.csv
curl -fsS --max-redirs 0 -o all-safe-ips.csv https://raw.githubusercontent.com/sefinek/known-bots-ip-whitelist/main/lists/all-safe-ips.csvhttps://raw.githubusercontent.com/sefinek/known-bots-ip-whitelist/main/lists/all-safe-ips.json
curl -fsS --max-redirs 0 -o all-safe-ips.json https://raw.githubusercontent.com/sefinek/known-bots-ip-whitelist/main/lists/all-safe-ips.json
