How to Use cURL With REST API

by

cURL is a powerful command-line tool that allows you to make requests to REST APIs directly from your terminal. With cURL, you can test REST APIs, debug issues, and even automate workflows.

In this comprehensive guide, you‘ll learn how to use cURL to interact with REST APIs. We‘ll cover how to send various HTTP requests, add headers and authentication, troubleshoot errors, and more.

An Introduction to cURL

cURL stands for "Client URL". It is a command-line tool that allows you to transfer data to and from a server. cURL supports many protocols including HTTP, HTTPS, FTP, and FTPS. This makes it useful for testing REST APIs which communicate over HTTP/HTTPS.

Here are some key features of cURL:

  • Send GET, POST, PUT, DELETE, HEAD, OPTIONS requests
  • Add custom headers to requests
  • Set authentication details like username and password
  • Follow redirects
  • Submit web forms
  • Specify timeout periods
  • Receive response information like headers, HTTP code, and body
  • Save response contents to a file
  • Continue/resume downloads
  • Use proxies

cURL is installed by default on most Linux/Unix systems. For Windows, you can download binaries from curl.se and add it to your PATH.

To check your cURL version, run:

curl --version

This will display information like:

curl 7.83.1 (Windows) libcurl/7.83.1 WinSSL
Release-Date: 2022-10-05
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz NTLM SPNEGO SSL SSPI TLS-SRP UNIXSockets metalink pkg-config proxy-negotiate

Now let‘s see how to use cURL to test REST APIs.

Sending a GET Request

The GET method is used to retrieve data from a REST API. To send a GET request with cURL, use:

curl https://api.example.com/users

This will send a GET request to the /users endpoint and display the response.

Some common parameters you can add to GET requests:

-i – Include response headers in output

-L – Follow redirects

-u – Add basic authentication in the form username:password

-o – Save response to a file

-v – Verbose output for debugging

For example:

curl -i -L -u myuser:pass123 https://api.example.com/users

This sends a GET request to /users with authentication and includes headers in the output.

Sending a POST Request

To create a new resource in a REST API, you need to send a POST request. The -d parameter allows you to specify the request body:

curl -d ‘{"name":"John", "email":"[email protected]"}‘ -H "Content-Type: application/json" -X POST https://api.example.com/users

This sends a JSON payload with the name and email to create a new user.

We use -H to set the Content-Type header and -X to specify the request method as POST.

Sending a PUT Request

To update an existing resource, you can send a PUT request similar to POST:

curl -d ‘{"name":"John Doe"}‘ -H "Content-Type: application/json" -X PUT https://api.example.com/users/123

This will update the name of the user with ID 123.

The request body contains the updated data and we set the method to PUT using -X.

Sending a DELETE Request

To delete a resource, send a DELETE request:

curl -X DELETE https://api.example.com/users/123

This will delete the user with ID 123.

For DELETE requests, there is usually no request body required.

Specifying Request Headers

REST APIs can accept additional request headers for things like authentication, content-type, etc.

You can add custom headers with the -H flag:

curl -H "Authorization: Bearer token" https://api.example.com/users

This adds an Authorization header with the bearer token to authenticate the request.

Some other common headers include:

-H "Content-Type: application/json"
-H "Accept: application/json"

Handling API Errors

When working with REST APIs, you may encounter client-side and server-side errors.

Client errors like 401 Unauthorized or 404 Not Found indicate the request is invalid. Server errors like 500 Internal Server Error means the API is having issues.

By default, cURL will display the response body even if the status code indicates an error. To fail requests on HTTP errors, use the --fail flag:

curl --fail -i https://api.example.com/invalid-path

Now cURL will return an exit code of 22 if the HTTP status is 4xx or 5xx.

You can also use the --resolve option to map hostnames and bypass certificate issues:

curl --resolve api.example.com:443:127.0.0.1 https://api.example.com

Using REST API Documentation

When working with a new API, consult the documentation for details on the available endpoints, parameters, headers, authentication, etc.

The documentation should provide example requests and responses to help you understand how to use the API.

For instance, the documentation may have examples like:

# GET /users

GET /users

### Response

Status: 200 OK

[
  { "id": 1, "name": "John" },
  { "id": 2, "name": "Jane" }  
]

You can test this out directly in cURL:

curl https://api.example.com/users

Referring to the documentation will help you quickly figure out how to use the API and build requests properly.

Passing URL Parameters

REST APIs often accept additional URL parameters for things like filtering, pagination, etc.

For example, to get users with status "active":

GET /users?status=active

In cURL, append the query string to the URL:

curl https://api.example.com/users?status=active

You can also pass multiple URL parameters:

curl https://api.example.com/users?status=active&limit=50

Handling Pagination

REST APIs commonly use pagination via parameters like page and limit to control the number of items in responses.

For instance:

GET /users?page=2&limit=25

This would give you the second page of 25 users.

To extract all pages, you need to paginate through the responses by incrementing the page and iterating through the loop.

Here is an example script to loop through all pages:

#!/bin/bash

URL="https://api.example.com/users"

PAGE=1

while true; do

  resp=$(curl "$URL?page=$PAGE")

  # extract data from JSON response

  if [ -n "$resp" ]; then  
    ((PAGE++))
  else
    break
  fi

done

This iterates through pages by incrementing $PAGE until no response is returned.

Passing Data in Files

For POST or PUT requests, you can pass the request body data using files instead of directly in the command.

Create a file containing the data to send, e.g:

# user.json

{
  "name": "John",
  "email": "[email protected]"  
}

Then make the request using --data @file:

curl -H "Content-Type: application/json" -X POST -d @user.json https://api.example.com/users

This reads the JSON data from user.json.

For larger bodies of data, passing through a file is easier than an inline string.

Handling Binary Data

REST APIs may accept binary input and output like images, audio, PDFs, etc.

To send binary data in cURL, set the body via -d @filename and add the Content-Type header:

curl -H "Content-Type: application/pdf" -X POST -d @file.pdf https://api.example.com/upload

For binary responses, use -o to save the output to a file:

curl -H "Accept: application/pdf" -o file.pdf https://api.example.com/download

This will save the PDF response directly to file.pdf.

Using cURL with OAuth

OAuth is a common authentication method used by REST APIs to handle authorization flows.

While the full OAuth workflow requires multiple steps, you can use cURL to generate tokens and make authenticated requests.

To fetch an access token from the OAuth server:

curl -d ‘client_id=1234&client_secret=abcd‘ https://accounts.example.com/token

Then use the access token in the Authorization header:

curl -H "Authorization: Bearer ACCESS_TOKEN" https://api.example.com/users

You can also pass the access token as a parameter:

curl https://api.example.com/users?access_token=ACCESS_TOKEN

This allows making authenticated API requests with the proper access token.

Setting a Timeout

By default, cURL requests do not have a timeout. The connection will wait indefinitely for a response.

You can add a timeout period in seconds using the --max-time option:

curl --max-time 10 https://api.example.com/long-request

Now if a response is not received within 10 seconds, the request will fail.

For scripts, you may want to retry timed out requests to handle temporary slowdowns.

Saving Responses to a File

To save API responses directly into files, use the -o flag followed by the filename:

curl -o users.json https://api.example.com/users

This saves the /users response into users.json.

You can combine this with other options:

curl -H "Authorization: Bearer token" -o users.json https://api.example.com/users

This makes an authenticated request and saves the response to a file.

Resuming Downloads

For downloading large files from an API, you can resume interrupted downloads using cURL‘s --continue-at option.

This requires the server to support byte ranges and accept the Range header.

Specify the number of bytes where you want to restart the download:

curl --continue-at 1000 -O https://api.example.com/download

cURL will add the Range header to resume from byte 1000.

Using Environment Variables

To avoid hardcoding values in your cURL requests, you can use environment variables:

API_HOST=api.example.com
API_TOKEN=123

curl -H "Authorization: Bearer $API_TOKEN" https://$API_HOST/users

Here API_HOST and API_TOKEN are environment variables that contain the actual values.

You can also create a .env file to store the variables:

# .env

API_HOST=api.example.com
API_TOKEN=123

And load it before running the request:

# Load .env
source .env

# Make request
curl -H "Authorization: Bearer $API_TOKEN" https://$API_HOST/users

This keeps your credentials separate and avoids hardcoding values.

Using cURL with a Proxy

To send your cURL requests through a proxy server, use the -x option:

curl -x proxy_host:port https://api.example.com

Some proxies may require authentication:

curl -x username:password@proxy_host:port https://api.example.com

Proxies can be useful for:

  • Accessing regional APIs from anywhere
  • Using an intermediary IP address
  • Caching responses
  • Logging and monitoring requests

Rotating between multiple proxy IPs prevents your own IP from getting blocked when making a high volume of requests.

Troubleshooting Issues

Here are some common issues you may encounter and how to debug them:

  • Connection errors – Use -v for verbose output to see where it fails. Check firewall, network, proxy, VPN settings.

  • SSL errors – Add -k to ignore invalid certificates or --resolve to specify certs.

  • HTTP errors – Use --fail to return exit code 22 on any 4xx or 5xx error.

  • Encoding issues – Set Accept and Content-Type headers explicitly based on API requirements.

  • Timeout errors – Use --max-time to increase timeout duration if needed.

  • Proxy errors – Verify proxy URL and credentials are correct. Add -x with the proxy details.

  • Authentication errors – Double check API keys, tokens, username/password. Pass via URL/headers as per docs.

Start with verbose output, check parameters passed, and confirm against API documentation to resolve issues.

Automating with Scripts

While cURL can be used for ad hoc testing, it really shines when automating requests in scripts.

Here is a simple script to get list of users, iterate through them, and make requests for each user:

#!/bin/bash

# Fetch users
users=$(curl https://api.example.com/users)  

# Extract user ids 
ids=$(echo "$users" | jq -r ‘.[].id‘)

# Loop through ids
for id in $ids; do

  # Make request per user
  curl https://api.example.com/users/$id > user_${id}.json

done

This makes an initial request to get users, extracts the ids, and iterates through them to fetch user details.

Scripts allow you to perform all kinds of automation for your API testing and usage.

Community cURL Resources

cURL has been around since the 90s and has an active community. Here are some other useful resources:

  • curl.se – Official site with docs and mailing lists.
  • Everything curl – In-depth tutorials and tips by curl‘s creator.
  • HTTPie – User-friendly cURL replacement.
  • Postman – GUI and scripts for API testing.
  • Insomnia – Open source API client with UI and CLI.
  • cURL Code Examples – Collection of common use cases.

For further help, the #curl channel on Libera IRC has a lively community of users.

The cURL project also welcomes bug reports and contributions on GitHub.

Conclusion

cURL is an indispensable tool for inspecting REST APIs. Its versatility, ubiquity, and simple interface make it perfect for testing.

In this guide, you learned how to:

  • Make various HTTP requests like GET, POST, PUT, DELETE
  • Pass headers, parameters, and data
  • Handle authentication, variables, and files
  • Debug issues and write scripts to automate workflows

These tips will help you get the most out of cURL for your API projects.

cURL takes away the complexity of working with APIs over HTTP. With a few commands, you can quickly test endpoints and integrate REST APIs into your applications.

So next time you need to prototype or troubleshoot an API, reach for cURL to easily send requests right from your terminal.

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

You May Like to Read,