How to Add Headers with cURL
Web ScrapingcURL (Client URL) is a versatile tool widely used for transferring data to and from servers. One of its powerful features is the ability to customize HTTP requests by adding headers. This article explains how to use cURL to add headers to your HTTP requests, complete with examples and practical applications.
cURL (Client URL) is a versatile tool widely used for transferring data to and from servers. One of its powerful features is the ability to customize HTTP requests by adding headers. This article explains how to use cURL to add headers to your HTTP requests, complete with examples and practical applications.
What Are HTTP Headers?
HTTP headers are key-value pairs that provide additional information with a request or response. They play a crucial role in:
- Authentication: Sending API keys or tokens.
- Content Negotiation: Specifying the content type (e.g., JSON, XML).
- Custom Metadata: Including extra information like correlation IDs for debugging.
Adding Headers with cURL
You can add headers to your cURL requests using the -H or --header option. The syntax is straightforward:
curl -H "Header-Name: Header-Value" <URL>
Examples of Adding Headers
Setting a Content-Type Header
curl -H "Content-Type: application/json" https://api.example.com/data
This tells the server that your request payload is in JSON format.
Adding Multiple Headers
curl -H "Authorization: Bearer YOUR_TOKEN" -H "Accept: application/json" https://api.example.com/secure
Here, we add:
- An Authorization header for authentication.
- An Accept header to specify the desired response format.
Custom Header Example
curl -H "X-Custom-Header: CustomValue" https://api.example.com/custom
This example demonstrates using a custom header for specific use cases.
Sending Headers with POST Data
To include headers along with POST data:
curl -X POST -H "Content-Type: application/json" -d '{"key":"value"}' https://api.example.com/submit
This combines a Content-Type header with a JSON payload.
Practical Use Cases
API Authentication
Most APIs require headers for authentication:
curl -H "Authorization: Bearer YOUR_ACCESS_TOKEN" https://api.example.com/user
Debugging
Adding correlation IDs or custom debugging headers:
curl -H "X-Debug-ID: 12345" https://api.example.com/debug
Web Scraping
Some servers require specific headers to mimic a browser request:
curl -H "User-Agent: Mozilla/5.0" https://example.com
Using cURL in Scripts
You can automate header-based requests in shell scripts:
#!/bin/bash
API_URL="https://api.example.com/data"
TOKEN="YOUR_ACCESS_TOKEN"
curl -H "Authorization: Bearer $TOKEN" -H "Accept: application/json" $API_URL
Advanced Tips
Using Header Files
Store headers in a file and include them in your request:
curl -K headers.txt https://api.example.com/data
Example headers.txt:
arduino
-H "Authorization: Bearer YOUR_ACCESS_TOKEN"
-H "Accept: application/json"
Combining Headers with Proxy
curl -H "Header-Name: Header-Value" -x http://proxy.example.com:8080 https://api.example.com
Conclusion
Adding headers with cURL is essential for crafting precise HTTP requests. Whether you're working with APIs, debugging, or handling custom headers, mastering the -H option is invaluable. Practice these examples to streamline your workflows and improve your cURL proficiency.
Summarize this post
Open it in your assistant of choice with the prompt ready to send.
Take a Taste of Easy Scraping!
Find more insights here

504 Gateway Timeout: What Causes It and How to Fix It
A 504 Gateway Timeout means an upstream server took too long to respond. Here is what causes it and…

What Is a 502 Bad Gateway Error? Causes and How to Fix It
A 502 Bad Gateway means one server got an invalid response from another. Learn the real causes, how…

Data Scraping in Production: Build a Pipeline That Lasts
Extraction is 20% of data scraping. Learn how to build a production data scraping pipeline that hand…