guide

How to Add Headers with cURL

cURL (Client URL) is a versatile tool widely used for transferring data to and from servers. One of its powerful features is the ability to customize HTTP requests by adding headers. This article explains how to use cURL to add headers to your HTTP requests, complete with examples and practical applications.
How to Add Headers with cURL

cURL (Client URL) is a versatile tool widely used for transferring data to and from servers. One of its powerful features is the ability to customize HTTP requests by adding headers. This article explains how to use cURL to add headers to your HTTP requests, complete with examples and practical applications.

What Are HTTP Headers?

HTTP headers are key-value pairs that provide additional information with a request or response. They play a crucial role in:

  • Authentication: Sending API keys or tokens.
  • Content Negotiation: Specifying the content type (e.g., JSON, XML).
  • Custom Metadata: Including extra information like correlation IDs for debugging.

Adding Headers with cURL

You can add headers to your cURL requests using the -H or --header option. The syntax is straightforward:

curl -H "Header-Name: Header-Value" <URL>

Examples of Adding Headers

Setting a Content-Type Header

curl -H "Content-Type: application/json" https://api.example.com/data

This tells the server that your request payload is in JSON format.

Adding Multiple Headers

curl -H "Authorization: Bearer YOUR_TOKEN" -H "Accept: application/json" https://api.example.com/secure

Here, we add:

  • An Authorization header for authentication.
  • An Accept header to specify the desired response format.

Custom Header Example

curl -H "X-Custom-Header: CustomValue" https://api.example.com/custom

This example demonstrates using a custom header for specific use cases.

Sending Headers with POST Data

To include headers along with POST data:

curl -X POST -H "Content-Type: application/json" -d '{"key":"value"}' https://api.example.com/submit

This combines a Content-Type header with a JSON payload.

Practical Use Cases

API Authentication

Most APIs require headers for authentication:

curl -H "Authorization: Bearer YOUR_ACCESS_TOKEN" https://api.example.com/user

Debugging

Adding correlation IDs or custom debugging headers:

curl -H "X-Debug-ID: 12345" https://api.example.com/debug

Web Scraping

Some servers require specific headers to mimic a browser request:

curl -H "User-Agent: Mozilla/5.0" https://example.com

Using cURL in Scripts

You can automate header-based requests in shell scripts:

#!/bin/bash

API_URL="https://api.example.com/data"
TOKEN="YOUR_ACCESS_TOKEN"

curl -H "Authorization: Bearer $TOKEN" -H "Accept: application/json" $API_URL

Advanced Tips

Using Header Files

Store headers in a file and include them in your request:

curl -K headers.txt https://api.example.com/data

Example headers.txt: arduino

-H "Authorization: Bearer YOUR_ACCESS_TOKEN"
-H "Accept: application/json"

Combining Headers with Proxy

curl -H "Header-Name: Header-Value" -x http://proxy.example.com:8080 https://api.example.com

Conclusion

Adding headers with cURL is essential for crafting precise HTTP requests. Whether you're working with APIs, debugging, or handling custom headers, mastering the -H option is invaluable. Practice these examples to streamline your workflows and improve your cURL proficiency.

Get started now!

Step up your web scraping

Try MrScraper Now

Find more insights here

How to Get Real Estate Listings: Scraping San Francisco Zillow

How to Get Real Estate Listings: Scraping San Francisco Zillow

In this guide, we'll walk you through the process of scraping Zillow data for San Francisco using MrScraper, the benefits of doing so, and how to leverage this data for your real estate needs.

How to Get Real Estate Listings: Scraping Zillow Austin

How to Get Real Estate Listings: Scraping Zillow Austin

Discover how to scrape Zillow Austin data effortlessly with tools like MrScraper. Whether you're a real estate investor, agent, or buyer, learn how to analyze property trends, uncover deeper insights, and make smarter decisions in Austin’s booming real estate market.

How to Find Best Paying Remote Jobs Using MrScraper

How to Find Best Paying Remote Jobs Using MrScraper

Learn how to find the best paying remote jobs with MrScraper. This guide shows you how to scrape top job listings from We Work Remotely efficiently and save time.

What people think about scraper icon scraper

Net in hero

The mission to make data accessible to everyone is truly inspiring. With MrScraper, data scraping and automation are now easier than ever, giving users of all skill levels the ability to access valuable data. The AI-powered no-code tool simplifies the process, allowing you to extract data without needing technical skills. Plus, the integration with APIs and Zapier makes automation smooth and efficient, from data extraction to delivery.


I'm excited to see how MrScraper will change data access, making it simpler for businesses, researchers, and developers to unlock the full potential of their data. This tool can transform how we use data, saving time and resources while providing deeper insights.

John

Adnan Sher

Product Hunt user

This tool sounds fantastic! The white glove service being offered to everyone is incredibly generous. It's great to see such customer-focused support.

Ben

Harper Perez

Product Hunt user

MrScraper is a tool that helps you collect information from websites quickly and easily. Instead of fighting annoying captchas, MrScraper does the work for you. It can grab lots of data at once, saving you time and effort.

Ali

Jayesh Gohel

Product Hunt user

Now that I've set up and tested my first scraper, I'm really impressed. It was much easier than expected, and results worked out of the box, even on sites that are tough to scrape!

Kim Moser

Kim Moser

Computer consultant

MrScraper sounds like an incredibly useful tool for anyone looking to gather data at scale without the frustration of captcha blockers. The ability to get and scrape any data you need efficiently and effectively is a game-changer.

John

Nicola Lanzillot

Product Hunt user

Support

Head over to our community where you can engage with us and our community directly.

Questions? Ask our team via live chat 24/5 or just poke us on our official Twitter or our founder. We're always happy to help.