Parallel.ai
ShapBot
A specialized AI crawler used to discover and index websites for Parallel's structured web info APIs.
Purpose: Search indexing for AI APIs
📊 Popularity & Traffic
Used to populate knowledge for developer-facing web data APIs.
🤖 User Agent Strings
Use these patterns to identify ShapBot in your server logs or configure your robots.txt file.
ShapBot
Respects robots.txtBackend indexing bot for AI APIs
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ShapBot/0.1.0🌐 IP Ranges
Source: Official Parallel.ai JSON file
Official source fileIdentified IP Ranges8 Ranges
34.60.130.210/32Subnet with 1 addresses
34.136.80.156/32Subnet with 1 addresses
104.198.36.41/32Subnet with 1 addresses
104.154.193.69/32Subnet with 1 addresses
34.135.143.163/32Subnet with 1 addresses
34.135.191.216/32Subnet with 1 addresses
How to read CIDR notation:
The/28 suffix indicates a block of 16 IP addresses. For example,.112/28 covers all addresses from .112 up to .127. Adding these to your firewall will block the entire range used by ShapBot.📝 Robots.txt Configuration
Add the following to your robots.txt file to block ShapBot:
User-agent: ShapBot
Disallow: /💡 Important Notes
- Focuses on vertical-specific data discovery (tech, documentation)
- Provides clear identification to avoid being mistaken for malicious scrapers
- Used by Google Vertex AI for grounding with Parallel web search
- Small footprint focused on high-value data discovery
Beyond blocking crawlers
See what AI is saying about your brand
Understanding crawlers is step one. With Aiso, you can see the actual conversations happening about your brand inside ChatGPT, Claude, and Perplexity.