How to use basic authentication with cURL?

Learn how to implement basic authentication using cURL, a powerful command-line tool, to access web resources securely. This guide provides a straightforward approach to enhance your data extraction techniques with authentication credentials.

Best practices

Always use HTTPS instead of HTTP when using basic authentication with cURL to ensure your credentials are encrypted during transmission.
Store your credentials in environment variables instead of hard coding them directly in your scripts to enhance security.
Use base64 encoding for your credentials when constructing the Authorization header manually to comply with the HTTP Basic Authentication standard.
Regularly rotate your passwords and update your cURL commands accordingly to minimize security risks.

# Basic cURL with username and password inline
curl -u "username:password" https://sandbox.oxylabs.io/products

# Using cURL with the -H option for the Authorization header
curl -H "Authorization: Basic $(echo -n 'username:password' | base64)" https://sandbox.oxylabs.io/products

# Storing credentials in a variable and using it in the cURL command
USERPWD="username:password"
curl -u $USERPWD https://sandbox.oxylabs.io/products

# Encode credentials to base64 manually and use in header
ENCODED=$(echo -n 'username:password' | base64)
curl -H "Authorization: Basic $ENCODED" https://sandbox.oxylabs.io/products

# Using cURL with --basic to enforce basic authentication
curl --basic -u "username:password" https://sandbox.oxylabs.io/products

Common issues

Ensure the base64 utility is available on your system to avoid errors when manually encoding credentials for the Authorization header.
Double-check the syntax and spacing in your cURL command, especially around the use of quotation marks and colons, to prevent authentication failures.
Avoid using special characters in usernames or passwords that might not be URL-encoded properly unless they are handled correctly in the script.
Test your cURL commands in a secure environment before deploying them in production to ensure they work as expected without exposing sensitive information.

# Ensure the base64 utility is available
# Good Example:
# Check if base64 is available and then use it
if command -v base64 > /dev/null; then
ENCODED=$(echo -n 'username:password' | base64)
curl -H "Authorization: Basic $ENCODED" https://sandbox.oxylabs.io/products
else
echo "base64 utility not found"
fi

# Bad Example:
# Using base64 without checking if it's available
ENCODED=$(echo -n 'username:password' | base64)
curl -H "Authorization: Basic $ENCODED" https://sandbox.oxylabs.io/products

### Example 2: Correct syntax and spacing in cURL command
# Good Example:
# Properly formatted cURL command
curl -u "username:password" https://sandbox.oxylabs.io/products

# Bad Example:
# Incorrect spacing leading to authentication error
curl -u"username:password" https://sandbox.oxylabs.io/products

### Example 3: Handling special characters in credentials
# Good Example:
# URL-encode special characters in credentials
USERPWD=$(echo -n 'user:name:pass#word' | jq -sRr @uri)
curl -u "$USERPWD" https://sandbox.oxylabs.io/products

# Bad Example:
# Special characters not handled, may cause failure
curl -u "user:name:pass#word" https://sandbox.oxylabs.io/products

### Example 4: Testing cURL commands in a secure environment
# Good Example:
# Use a test environment URL to ensure safety
curl -u "username:password" https://test.sandbox.oxylabs.io/products

# Bad Example:
# Directly using production URL without testing
curl -u "username:password" https://sandbox.oxylabs.io/products