Rate Limits
ScrapeBadger uses tier-based rate limiting to ensure fair usage and reliable performance for all users.
How Rate Limiting Works
Tier-Based Limits
Your rate limit is determined by your account tier. All API keys inherit the same limit from your account.
Sliding Window
We use a sliding window algorithm for accurate rate limiting. Limits are calculated per minute with smooth request distribution.
Real-Time Headers
Every response includes rate limit headers so you can monitor usage and implement smart retry logic.
Rate Limits by Tier
Default tier for new accounts
- 60 requests per minute
- All endpoints accessible
- Standard support
For small-scale applications
- 300 requests per minute
- All endpoints accessible
- Priority support
For production applications
- 1,000 requests per minute
- All endpoints accessible
- Priority support
For high-volume applications
- 5,000 requests per minute
- All endpoints accessible
- Dedicated support
Rate Limit Headers
Every API response includes these headers to help you track your rate limit usage:
| Header | Description | Example |
|---|---|---|
| X-RateLimit-Limit | Maximum number of requests allowed per minute for your tier | 300 |
| X-RateLimit-Remaining | Number of requests remaining in the current window | 287 |
| X-RateLimit-Reset | Unix timestamp when the rate limit window resets | 1703123456 |
| Retry-After | Seconds until you can retry (only on 429 responses) | 45 |
Rate Limit Exceeded Response
When you exceed your rate limit, the API returns a 429 Too Many Requests response:
{
"detail": "Rate limit exceeded",
"limit": 300,
"remaining": 0,
"reset_at": 1703123456,
"tier": "basic"
}Handling Rate Limits
Here's how to implement proper rate limit handling in your application:
# Check rate limit headers in response curl -i -H "x-api-key: sb_live_xxx" \ https://api.scrapebadger.com/v1/twitter/tweets/123 # Response headers: # X-RateLimit-Limit: 300 # X-RateLimit-Remaining: 299 # X-RateLimit-Reset: 1703123456
Best Practices
Monitor rate limit headers
Check X-RateLimit-Remaining on each response to proactively slow down before hitting limits.
Implement exponential backoff
When rate limited, wait the Retry-After time and consider increasing wait time on repeated limits.
Use request queuing
Queue requests and process them at a controlled rate to stay within your limits.
Cache responses
Cache API responses where appropriate to reduce the number of requests you need to make.
Avoid Rate Limit Abuse
Repeatedly hitting rate limits or attempting to bypass them may result in temporary or permanent restrictions on your account.
- • Don't create multiple accounts to circumvent limits
- • Don't use rotating API keys to bypass rate limiting
- • Contact support if you need higher limits
Frequently Asked Questions
How are rate limits applied?
Rate limits are applied per API key using a sliding window algorithm. Each API key has its own separate rate limit counter based on your account tier.
What happens when I hit the rate limit?
When you exceed your rate limit, the API returns a 429 Too Many Requests response with details about when you can retry. Use the Retry-After header to know when to try again.
Do failed requests count against my rate limit?
Yes, all requests (successful or not) count against your rate limit. However, rate limits are generous and designed to accommodate normal usage patterns.
Can I have different rate limits for different API keys?
All API keys inherit the rate limit from your account tier. To get higher rate limits, upgrade your account tier.
How do I check my current rate limit usage?
Every API response includes rate limit headers (X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset) so you can track your usage in real-time.
Need Higher Rate Limits?
Contact our sales team to discuss enterprise plans with custom rate limits for your needs.
Contact Sales