504 Gateway Timeout: What Causes It and How to Fix It
Web ScrapingA 504 Gateway Timeout means an upstream server took too long to respond. Here is what causes it and how to fix it on both sides.
A 504 Gateway Timeout error means a server acting as a gateway or proxy waited for a response from another server upstream and did not get one in time. It is one of the standard HTTP status codes representing inter-server communication delays. It is a server-side infrastructure issue.
Understanding what is 504 gateway timeout helps you resolve issues quickly.
What a 504 Gateway Timeout actually means
Modern web infrastructure routes traffic through proxy chains. When your browser requests a URL, it hits a reverse proxy, load balancer, or CDN (such as Nginx, AWS ALB, or Cloudflare) that forwards the request to an upstream origin server.
A 504 status code occurs when the proxy server's timer expires before the upstream server returns a response. The proxy closes the connection and returns a 504 status code to the client.
The core difference between gateway errors comes down to whether the upstream server responded at all:
502 Bad Gateway error: the upstream server answered, but the answer was invalid or corrupted.
504 Gateway Timeout: the upstream server did not answer at all in time.
Not a 503 Service Unavailable: 503 means server rejection from overload; 504 means execution stalled.
Not a 408 Request Timeout: 408 is client transmission delay; 504 is inter-server delay.
How a 504 error shows up
The generic 504 gateway timeout error displays differently across reverse proxies and cloud platforms:
- 504 Gateway Timeout: Standard response from default load balancers.
- 504 Gateway Time-out: Default Nginx error page formatting.
- HTTP Error 504: Standard browser rendering when no custom HTML is supplied.
- Gateway Timeout Error: Wording generated by IIS, Apache, and custom middleware.
- Error 504: Gateway time-out: Cloudflare page indicating Cloudflare reached the origin IP, but the origin failed to respond within Cloudflare's HTTP timeout window.
- 504 Gateway Timeout on Vercel / AWS / Azure: Error screens triggered when serverless functions or API gateways hit maximum execution limits.
Every variation confirms that an edge proxy gave up waiting for a backend application server to respond.
What causes a 504 Gateway Timeout?
Identifying what causes a 504 error requires checking backend execution times, server load, and proxy configurations.
The upstream server is too slow
Unindexed database queries, slow third-party API calls, or high CPU consumption by background tasks prevent the app from responding before proxy timeout.
How to check: Review APM tools or application logs for slow database queries exceeding 30 seconds.
Proxy timeout settings are too low
The backend application requires 45 seconds to generate a report, but the reverse proxy is configured with a 30-second timeout. The proxy drops the connection while the application is still processing.
In Nginx, default timeout parameters control these limits: proxy_read_timeout, proxy_connect_timeout, proxy_send_timeout, and fastcgi_read_timeout (or max_execution_time in PHP php.ini).
location /api/ {
proxy_pass http://127.0.0.1:8080;
proxy_read_timeout 60s;
}
How to check: Inspect web server configuration timeouts and compare them against backend execution times.
The origin server is overloaded
High concurrent traffic exhausts application workers. When all PHP-FPM processes, Gunicorn workers, or Node threads are occupied, incoming requests queue up until Nginx returns a 504 nginx error response.
How to check: Run top or check server metrics to evaluate CPU usage, RAM saturation, and worker pool counts.
Network problems between proxy and origin
Packet loss, internal routing failures, or misconfigured firewalls between proxy and origin instances cause connection requests to hang until timeout thresholds trigger.
How to check: Test network latency from proxy to origin host using traceroute or curl -v http://origin-internal-ip:port.
Long-running requests that were never meant to be synchronous
Executing heavy background tasks (like PDF generation or bulk imports) inside a standard HTTP request loop blocks worker threads until proxy timeouts trigger.
How to check: Identify endpoints processing heavy jobs synchronously instead of delegating tasks to background queues like Redis Celery or BullMQ.
CDN and Cloudflare-specific limits
Cloudflare enforces a strict 100-second proxy timeout. If an origin server takes longer than 100 seconds to send headers, Cloudflare returns an Error 504.
Cloudflare also returns Error 524 (A Timeout Occurred) when TCP connection succeeds but no response data is received within 100 seconds. Refer to Cloudflare 504 and 524 documentation for details.
How to check: Look for Cloudflare Ray IDs to verify whether the delay occurred during TCP handshake or header transmission.
Serverless and platform limits
Vercel, AWS Lambda, and Azure Functions impose hard execution limits that trigger a 504 gateway time-out error when exceeded.
How to check: Inspect execution logs (AWS CloudWatch or Vercel Function Logs) for timeout termination entries.
Rate limiting and bot protection
WAFs and security tools occasionally drop or delay scraper connections instead of returning HTTP 429 Too Many Requests, triggering a 504 timeout.
How to check: Compare test request response latency using standard browser headers versus automated script runs.
How to fix a 504 Gateway Timeout as a visitor
When encountering a 504 bad gateway error, perform these client-side checks:
- Reload the webpage: Refresh after 30 seconds to clear temporary queue spikes.
- Perform a hard refresh: Use
Ctrl+F5(Windows/Linux) orCmd+Shift+R(macOS) to bypass browser cache. - Clear browser cache: Remove cached session files.
- Flush local DNS cache: Reset network resolution tables using shell commands:
# Windows: ipconfig /flushdns
# macOS: sudo dscacheutil -flushcache; sudo killall -HUP mDNSResponder
# Linux: sudo systemd-resolve --flush-caches
- Disable VPN or proxy services: Test without intermediary VPN nodes experiencing packet drops.
- Check global availability: Use tools like
downforeveryoneorjustme.comto confirm whether the failure affects all users.
If these steps fail, the fault is on the server side.
How to fix a 504 Gateway Timeout as a site owner or developer
Follow this workflow to locate and resolve server-side timeouts:
1. Confirm backend application responsiveness
systemctl status your-app # Verify service status
curl -I http://127.0.0.1:3000 # Test local application port
2. Inspect reverse proxy error logs
tail -f /var/log/nginx/error.log
Example Nginx log entry:
2024/03/15 15:40:12 [error] 4321#0: *8765 upstream timed out (110: Connection timed out)
while reading response header from upstream, client: 192.168.1.50, request: "GET /api/reports HTTP/1.1",
upstream: "http://127.0.0.1:3000/api/reports", host: "example.com"
This confirms Nginx connected to port 3000 on localhost, but the application failed to return headers before Nginx's timer expired.
3. Benchmark backend response times directly
curl -w "Total Time: %{time_total}s\n" -o /dev/null -s http://127.0.0.1:3000/api/reports
If response time exceeds proxy timeout limits, backend performance is the root problem.
4. Identify database and code bottlenecks
Enable slow query logging in MySQL (slow_query_log = 1, long_query_time = 2 in my.cnf) and add missing database indexes.
5. Adjust proxy timeouts and scale workers
Raise proxy timeout limits in Nginx:
location /api/long-job {
proxy_pass http://127.0.0.1:3000;
proxy_read_timeout 300s;
}
Reload Nginx: sudo nginx -t && sudo systemctl reload nginx. Scale worker pools in PHP-FPM or Gunicorn.
6. Move tasks to background queues
Refactor heavy synchronous endpoints to return an HTTP 202 Accepted status with a job ID:
@app.route('/api/export', methods=['POST'])
def handle_export():
job = queue.enqueue(generate_large_report, request.json)
return jsonify({"job_id": job.id, "status": "processing"}), 202
7. Verify CDN configuration
If using Cloudflare, pause Cloudflare in the dashboard to test whether the timeout occurs at the CDN edge or origin host.
How to handle 504 errors when scraping or calling APIs
When calling APIs or web scrapers, 504 errors require retry strategies.
Treat 504 errors as transient and retryable. Implement retries using exponential backoff with randomized jitter:
import requests
import time
import random
def fetch_api_data(url, max_retries=3):
"""Fetch URL with explicit timeouts and exponential backoff retry logic."""
for attempt in range(max_retries):
try:
response = requests.get(url, timeout=(10, 30))
if response.status_code != 504:
return response
except requests.exceptions.Timeout:
pass
sleep_time = (2 ** attempt) + random.uniform(0.5, 1.5)
time.sleep(sleep_time)
return None
result = fetch_api_data("https://example.com/api/data")
Set explicit client timeouts (timeout=(10, 30)) to prevent scripts from hanging indefinitely. Lower concurrent thread counts if timeouts spike under load. If 504 errors cluster on specific IP subnets, rotate residential IP addresses.
For production scraping, MrScraper's Web Unblocker handles IP rotation, proxy retries, and browser fingerprinting automatically.
How to prevent 504 errors
Preventing 504 timeouts requires proactive performance management:
- Monitor response latency metrics: Set alerts on p95 and p99 latency thresholds rather than uptime alone.
- Implement aggressive caching: Use Redis or Memcached to cache database queries and static API payloads.
- Optimize database queries: Index filtered query fields to maintain sub-second response times under load.
- Offload heavy processing to background workers: Use message queues for operations taking longer than 2 seconds.
- Configure load balancer health checks: Ensure load balancers remove unresponsive origin instances automatically.
- Align proxy and application timeouts: Set proxy read timeouts slightly higher than application execution limits.
504 vs 502 vs 503 vs 408 — what is the difference?
| Code | What it means | Whose fault | Typical fix |
|---|---|---|---|
| 502 Bad Gateway error | Upstream server returned invalid or corrupt response | Origin / Proxy | Fix application crashes and proxy routing |
| 503 Service Unavailable | Server rejecting requests due to maintenance or overload | Server Capacity | Scale worker capacity or wait |
| 504 Gateway Timeout | Upstream server failed to respond before proxy timer expired | Origin / Network | Optimize code, index database, raise timeouts |
| 408 Request Timeout | Client browser took too long sending request payload | Client Network | Improve client network connection |
| Cloudflare Error 524 | Cloudflare TCP connected but origin sent no data within 100s | Origin Application | Optimize slow scripts or move job to background |
Frequently asked questions
How do I fix a 504 gateway timeout?
Visitors should reload using Ctrl+F5, clear browser cache, or flush DNS. Developers should inspect web server error logs, optimize slow queries, scale worker pools, or increase proxy read timeouts in Nginx.
Is a 504 error my fault?
No. A 504 error is a server-side issue. It occurs when a proxy fails to receive a timely response from an upstream application server. The issue rests with the host.
How long does a 504 gateway timeout last?
Duration depends on the cause. Transient spikes clear in seconds or minutes. Hard timeouts caused by unindexed queries persist until code or configuration is updated.
What is the difference between 502 and 504?
A 502 Bad Gateway means the upstream server responded with an invalid payload. A 504 Gateway Timeout means the upstream server sent no response before the proxy timer expired.
Can a slow internet connection cause a 504 error?
Rarely. Slow client connections typically trigger HTTP 408 Request Timeout errors. However, packet loss communicating with an edge proxy can surface as a 504 error.
Does a 504 error hurt SEO?
Yes, if the error persists. Googlebot handles brief maintenance timeouts gracefully, but persistent 504 gateway timeouts cause Googlebot to lower crawl frequency and de-index affected URLs.
A 504 Gateway Timeout confirms that an edge proxy gave up waiting for a response from an upstream server. Visitors should refresh and flush DNS, while developers must analyze proxy logs, benchmark backend queries, and offload long jobs to background queues.
Read our guide on the 502 Bad Gateway error or review rate limits in our guide on 429 Too Many Requests. For automated timeout handling, explore MrScraper Documentation.
Summarize this post
Open it in your assistant of choice with the prompt ready to send.
Take a Taste of Easy Scraping!
Find more insights here

What Is a 502 Bad Gateway Error? Causes and How to Fix It
A 502 Bad Gateway means one server got an invalid response from another. Learn the real causes, how…

Data Scraping in Production: Build a Pipeline That Lasts
Extraction is 20% of data scraping. Learn how to build a production data scraping pipeline that hand…

Web Scraping Guide: How to Choose the Right Approach
Compare five web scraping approaches from basic scripts to managed APIs. Learn what breaks at each s…