How Many Requests Can You Send Before Getting IP Banned? (And How to Fix It)

There's no universal answer to "how many requests before you get blocked" — and any tool or blog post that gives you a specific number without context is oversimplifying a system that's specifically designed to be unpredictable. What there is: a set of signals that every serious anti-bot system evaluates, a range of detection thresholds that vary by site category and protection tier, and a collection of techniques that address each signal at the source.

IP ban prevention is the practice of structuring your automated requests so they don't accumulate the pattern fingerprint that triggers blocking — not by hiding that you're making requests, but by ensuring those requests look, to the detection system evaluating them, like distributed human traffic rather than a single automated agent. This guide covers how IP banning and rate limiting actually work, the request thresholds that typically trigger detection across different site categories, and the practical stack of techniques that keep scraping operations running sustainably.

What Is IP Ban Prevention?

IP ban prevention is the set of techniques used to keep automated requests from being detected and blocked by target websites. An IP ban occurs when a server identifies a specific IP address as a source of suspicious or policy-violating traffic and stops serving responses to it — either temporarily (rate limiting with backoff) or permanently (hard block for that IP).

The distinction between rate limiting and IP banning matters operationally. Rate limiting is a server-side throttle that returns 429 Too Many Requests when you exceed a defined threshold — it's a soft signal that tells you to slow down, often with a Retry-After header specifying when you can resume. IP banning is more severe: the server stops responding to your IP at all, often returning 403 Forbidden, a CAPTCHA challenge, or silently serving empty or misleading responses without explaining why.

Modern anti-bot systems — Cloudflare Bot Management, Akamai Bot Defender, PerimeterX — sit above both: they evaluate dozens of signals simultaneously to assess the probability that a request is automated, and apply a graduated response ranging from CAPTCHA challenge to hard block based on their confidence score. IP address is just one of those signals, which is why IP rotation alone doesn't solve the problem and why comprehensive IP ban prevention requires addressing the full signal stack.

How IP Banning and Rate Limiting Work

To prevent bans effectively, you need to understand what detection systems are actually measuring. The signals fall into three layers, and each layer is evaluated independently — which means you can pass one while failing another.

Layer 1: IP-level signals. The first check most detection systems run is against the request's source IP. Data-center and hosting IP ranges are flagged by default in IP reputation databases. VPN endpoints are similarly catalogued. Request frequency per IP — how many requests per second, per minute, per hour — is tracked and compared against expected human behavior for that site. Sending 500 requests per minute from one IP looks nothing like a human user on any website.

Layer 2: Request pattern signals. Even from a clean residential IP, automated requests often exhibit statistical signatures that differ from human browsing. Perfect regularity of request timing (exactly 2.0 seconds between each request), no variation in request headers across sessions, identical user-agent strings on every request, missing headers that real browsers always include (Accept-Language, Accept-Encoding, referrer chains) — these all contribute to a behavioral fingerprint that detection systems score.

Layer 3: Browser fingerprint signals. For targets that serve JavaScript challenges (the most sophisticated bot-protection systems), fingerprinting goes deeper: TLS client hello characteristics, browser feature detection results, canvas rendering fingerprints, WebGL signatures, and JavaScript execution environment properties. These signals are evaluated by the JavaScript challenge code running in the browser, not by simple HTTP header inspection.

What thresholds actually trigger bans? These are generalizations — every site has its own configuration — but the practical ranges:

Lightly protected informational sites: 100–500 requests per minute from one IP before rate limiting; bans typically only occur at very high sustained volumes
E-commerce and travel sites: 20–50 requests per minute triggers rate limiting; 100+ triggers active blocking; any data-center IP may be blocked immediately
Social media and high-value platforms: 5–10 requests per minute before challenges appear; sustained automation at any rate typically triggers bans; residential IPs and browser fingerprinting required
Financial and government sites: Variable and unpredictable; often block any automated access regardless of rate

The throughput that matters isn't requests per minute from your total operation — it's requests per IP per time window against the specific target, which is why rotation is the foundational mitigation.

Step-by-Step Guide: How to Avoid IP Bans

Step 1: Start Slow and Measure Responses

Never start a new scraping target at your target volume. Begin at 10–20% of your intended rate, monitor the response codes, and look for the first signs of detection: 429 responses, CAPTCHA challenges, empty response bodies, or redirects to access-denied pages. This baseline establishes the practical threshold for that specific target before you've burned through your IP pool.

Log every response status code during this discovery phase. A sudden shift from 200 to 302 to a block page is the target telling you exactly where its limit is.

Step 2: Add Delay Variability — Not Just Delays

Fixed delays are easier for detection systems to identify than variable delays, because human browsing has natural randomness that uniform machine timing lacks. Don't sleep for exactly 2 seconds between requests. Sleep for a random duration drawn from a distribution that approximates human behavior:

import time
import random

def human_delay(base_seconds: float = 2.0, variance: float = 1.5):
    """Sleep for a human-like random duration around a base value."""
    delay = max(0.5, random.gauss(base_seconds, variance))
    time.sleep(delay)

The gauss distribution produces occasional short delays and occasional longer ones, with the mean clustered around your base. This creates a timing distribution that resembles human browsing far more convincingly than uniform delays.

Step 3: Rotate IPs Per Request or Per Session

IP rotation is the primary IP-level mitigation. Each request (or each logical session) should exit through a different IP — specifically a residential IP, not a data-center address that will be blocked regardless of request rate.

def make_request_with_rotation(url: str, proxy_endpoint: str, api_key: str) -> dict:
    """Make a request through a rotating residential proxy endpoint."""
    proxies = {
        "http": f"http://user-{api_key}:@{proxy_endpoint}",
        "https": f"http://user-{api_key}:@{proxy_endpoint}",
    }
    response = requests.get(
        url,
        proxies=proxies,
        timeout=15,
        headers={
            "User-Agent": random.choice(USER_AGENT_POOL),
            "Accept-Language": "en-US,en;q=0.9",
            "Accept-Encoding": "gzip, deflate, br",
        }
    )
    return response

The proxy endpoint format varies by provider — adapt to your provider's connection string specification.

Step 4: Rotate and Randomize Headers

User-agent strings, Accept-Language, Accept headers, and referrer chains should vary across requests. Maintain a pool of realistic user-agent strings representing current browser versions, and select from them randomly per session:

USER_AGENT_POOL = [
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
    "Mozilla/5.0 (X11; Linux x86_64; rv:124.0) Gecko/20100101 Firefox/124.0",
]

Avoid user-agent strings for outdated browser versions — detection systems know which UA strings are commonly spoofed and weight them higher for suspicion.

Step 5: Implement Exponential Backoff on Detection Signals

When a 429 or access-denial response appears, don't retry immediately. Immediate retries signal automation; backing off and increasing the delay before retrying signals tolerance for rate limiting similar to what a throttled human might exhibit:

def request_with_backoff(url: str, max_retries: int = 4) -> requests.Response | None:
    """Retry with exponential backoff on rate limit responses."""
    for attempt in range(max_retries):
        response = make_request_with_rotation(url, PROXY_ENDPOINT, API_KEY)
        if response.status_code == 200:
            return response
        if response.status_code == 429:
            wait = (2 ** attempt) + random.uniform(0, 1)
            print(f"Rate limited. Waiting {wait:.1f}s before retry {attempt + 1}")
            time.sleep(wait)
        else:
            break
    return None

Best Tools for IP Ban Prevention

Residential proxy networks are the foundation. Oxylabs, Bright Data, and Smartproxy all offer rotating residential proxy pools with geo-targeting. The right choice depends on your volume and target sites — see the residential proxy comparison guide for detailed evaluation.

Managed scraping APIs handle the full anti-ban stack as infrastructure. MrScraper's Scraping Browser manages IP rotation, browser fingerprint spoofing, CAPTCHA bypass, and JavaScript rendering together — so each request that reaches a target has already passed the multi-layer detection gauntlet that individual mitigation techniques handle separately. For teams scraping bot-protected targets where assembling and maintaining each mitigation layer independently is a real operational burden, the consolidated approach reduces both integration complexity and failure surface area. More at https://mrscraper.com.

Playwright with stealth plugins for teams building browser-based scrapers: the playwright-stealth library modifies headless browser fingerprints to remove the most common automation detection signatures. Combined with residential proxies, it addresses Layer 2 and Layer 3 detection signals that proxy rotation alone doesn't cover.

Key Features to Look For in an Anti-Ban Scraping Stack

Residential IP pool with automatic rotation: Data-center IPs are pre-blocked on most protected targets — residential rotation is the minimum viable IP strategy.
Request timing controls: Variable delays, not fixed intervals — the ability to configure Gaussian or bounded random delay distributions rather than constant sleep values.
Header randomization: User-agent pool management, realistic browser header profiles, and referrer chain simulation across sessions.
Detection signal handling: Automatic 429 backoff, CAPTCHA detection and routing, and response monitoring that identifies soft-blocks (empty responses, misleading content) as well as hard blocks.
Browser fingerprint management: For targets using JavaScript challenges, headless browser fingerprint spoofing (canvas, WebGL, navigator properties) is required beyond HTTP-layer mitigation.
Session management: Separate cookie jars and browser contexts per logical session to prevent cross-request state from contributing to fingerprint accumulation.

When Should You Prioritize IP Ban Prevention?

Make it a first-class concern when:

Your targets are e-commerce, social, travel, or financial platforms with active bot management
You're running sustained scraping — the same targets repeatedly over days or weeks — rather than one-time extractions
Request volume per target exceeds a few hundred per hour, even with rotation
Any block means data gaps that affect business decisions — competitive pricing intelligence, availability monitoring, time-sensitive research

Lighter mitigation may be sufficient when:

Your targets are informational sites with no anti-bot investment
Request volume is low (tens of pages per hour) and naturally spread across time
Target diversity is high — many different sites rather than repeated deep scraping of one target — which naturally limits per-IP concentration

Common Challenges and Limitations

IP rotation alone doesn't address fingerprinting. Teams that implement residential proxy rotation and then hit fingerprinting-based blocks often conclude their proxies are bad. Usually, the proxies are fine — the headless browser's fingerprint is failing the JavaScript challenge layer that runs after IP reputation passes. Fingerprint spoofing requires browser-level configuration that proxy settings don't affect.

Detection systems adapt. Static mitigation configurations — the same user-agent pool, the same delay distribution — can themselves become fingerprints if used at sufficient volume against the same target over time. Production scraping operations that run at scale need to periodically refresh their header pools, review detection rates, and update mitigation parameters when success rates decline.

Shared proxy pools have shared reputation. If you're using a residential proxy pool shared with other customers, other customers' abuse of the same IPs affects the reputation of IPs you'll use. High-abuse targets (streaming platforms, heavily-guarded e-commerce) often have dynamic blocklists that capture IPs within hours of heavy use by any customer on the pool. Private or dedicated residential IPs are more expensive but carry no shared reputation history.

Soft blocks are harder to detect than hard blocks. A 403 or 429 response is unambiguous. A target that detects you and serves empty product listings, zero-result searches, or misleading prices without any error code is far more dangerous — you collect bad data confidently. Build validation logic that checks response content quality, not just status codes, as part of your detection monitoring.

Conclusion

There is no safe universal request rate — but there is a consistent set of detection layers every target runs, and a proven set of techniques that address each layer. Residential IP rotation handles the IP reputation layer. Variable timing handles the behavioral pattern layer. Browser fingerprint management handles the JavaScript challenge layer. Each layer stacked on the others compounds the protection multiplicatively.

The teams that get reliably blocked are the ones that address one or two layers and assume that's sufficient. The teams that sustain long-running scraping operations have internalized that detection is multi-dimensional, built monitoring that surfaces detection signals early, and iterate when new blocks appear rather than assuming the approach that worked last month still applies today.

What We Learned

"How many requests before a ban" is always site-specific: Lightly protected sites tolerate hundreds per minute from one IP; high-value platforms may block after a handful — the answer depends entirely on the target's anti-bot investment.
Rate limiting and IP banning are different: 429 is a soft throttle that tells you to slow down; a hard IP ban silently or explicitly blocks all further access — recognizing which you're hitting determines the right response.
Detection operates in layers — IP, behavioral pattern, and browser fingerprint: Addressing only one layer while failing another produces the same blocked result as addressing none.
Variable delays beat fixed delays every time: Detection systems recognize uniform machine timing; Gaussian-distributed random delays approximate human behavioral variance at a fraction of the implementation cost.
Residential IPs are the IP-layer minimum for protected targets: Data-center and VPN IP ranges are pre-flagged in bot-detection databases before your request behavior is even evaluated.
Soft blocks are more dangerous than hard blocks: An empty result or misleading data served silently is worse than a 403 — build content validation into your monitoring, not just status code checks.

FAQ

How many requests per second can I send before getting IP banned?

It depends entirely on the target site's anti-bot configuration. Lightly protected informational sites may tolerate 5–10 requests per second from a single IP without triggering rate limiting. E-commerce and travel sites typically limit to 0.5–1 request per second. Social media and financial platforms may challenge or block within seconds of detecting automated patterns regardless of rate. There is no universal safe threshold — test each target conservatively and build detection monitoring before scaling up.
What is the difference between rate limiting and an IP ban?

Rate limiting is a temporary throttle — the server returns 429 Too Many Requests and optionally specifies a Retry-After interval, after which requests are accepted again. An IP ban is more permanent: the server blocks all requests from that IP, typically returning 403, a CAPTCHA, or serving misleading empty responses. Rate limiting is a soft signal to slow down; an IP ban means that IP address is no longer usable for that target without rotation.
Does proxy rotation prevent IP bans?

Proxy rotation prevents IP-level bans by distributing requests across many IPs so no single address accumulates a detectable request volume. However, proxy rotation alone doesn't prevent detection if behavioral pattern signals (uniform timing, identical headers) or browser fingerprinting signals (headless browser characteristics) are detectable. Effective IP ban prevention requires addressing all three detection layers, not just IP rotation.
What is the best way to avoid IP blocking when scraping?

The most effective combination: residential proxy rotation (not data-center IPs), variable request timing using random distributions rather than fixed delays, randomized browser header pools, exponential backoff on 429 responses, and browser fingerprint spoofing for JavaScript-challenge targets. Managed scraping APIs that handle all of these layers as infrastructure remove the integration burden of assembling and maintaining each technique separately.
Can websites detect scraping even with rotating proxies?

Yes. IP rotation addresses the IP-reputation detection layer but not behavioral pattern or browser fingerprint layers. A scraper sending identical headers on every request, making requests with perfectly uniform timing, or running a headless browser with detectable automation signatures will be identified by Layer 2 and Layer 3 detection regardless of how many IPs it rotates through. Comprehensive anti-detection requires addressing all layers.
How do I know if my IP has been banned vs. rate limited?

Rate limiting typically returns HTTP 429 with a Retry-After header. IP banning shows up differently by target: HTTP 403, a redirect to a CAPTCHA or access-denied page, or — most dangerously — a 200 response with empty or misleading content. If your responses suddenly return no data where data should exist, or redirect to a verification page, treat it as a block rather than a rate limit. Building response content validation into your scraper catches soft blocks that status codes alone would miss.