Python Intermediate_005: Working with the Requests Library

---

The requests library in Python is a powerful tool for making HTTP requests, allowing you to interact with APIs, fetch data from websites, and send data to servers. It's simple, user-friendly, and perfect for beginners.

---

Installation

Install the requests library using pip if you don’t have it installed:

pip install requests

Key Features and Examples

Let’s dive into the most useful and practical use cases for the requests library, with clear explanations and code examples.

---

1. Making a Simple GET Request

A GET request is used to fetch data from a server.

Code Example:

import requests

response = requests.get("https://api.github.com")
print("Status Code:", response.status_code)
print("Response Body:", response.text)

Explanation:

1. response.status_code:

• Displays the HTTP status code (e.g., 200 for success, 404 for not found).

2. response.text:

• Prints the response body (data returned by the server) as a string.

Output:

Status Code: 200
Response Body: {
  "current_user_url": "https://api.github.com/user",
  "current_user_authorizations_html_url": "https://github.com/settings/connections/applications{/client_id}",
  ...
}

---

2. Sending URL Parameters in a GET Request

Query parameters are added to a URL to filter or modify the data returned by the server.

Code Example:

import requests

url = "https://api.github.com/search/repositories"
params = {"q": "requests+language:python"}  # Search for Python repositories about "requests"
response = requests.get(url, params=params)

print("Request URL:", response.url)
print("Total Repositories Found:", response.json().get("total_count"))

Explanation:

1. params:

• Used to pass query parameters as a dictionary.
• Converts {"q": "requests+language:python"} to ?q=requests+language:python and appends it to the URL.

2. response.url:

• Outputs the final URL with parameters:

https://api.github.com/search/repositories?q=requests+language:python

3. response.json():

• Parses the JSON response into a Python dictionary.
• total_count gives the total number of repositories matching the query.

Output:

Request URL: https://api.github.com/search/repositories?q=requests+language:python
Total Repositories Found: 230

---

3. Making a POST Request

A POST request is used to send data to a server, such as submitting forms.

Code Example:

import requests

url = "https://httpbin.org/post"
data = {"username": "test_user", "password": "test_pass"}
response = requests.post(url, data=data)

print("Response JSON:", response.json())

Explanation:

1. data:

• Sends form data as a dictionary in the body of the request.

2. response.json():

• Parses the JSON response from the server.
• The server at https://httpbin.org/post echoes back the data you send, 

Output:

Response JSON: {
  "args": {},
  "data": "",
  "files": {},
  "form": {
    "password": "test_pass",
    "username": "test_user"
  },
  'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Content-Length': '37', 'Content-Type': 'application/x-www-form-urlencoded', 'Host': 'httpbin.org', 'User-Agent': 'python-requests/2.32.3', 'X-Amzn-Trace-Id': 'Root=1-6778c520-3c1dde5b75a7d2900d9b00cb'}, 
 'json': None, 
 'origin': 'xxx', 
 'url': 'https://httpbin.org/post'}

}

---

4. Downloading a File

You can use the requests library to download files like images, PDFs, or other content.

Code Example:

import requests

url = "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf"
response = requests.get(url)

with open("dummy.pdf", "wb") as file:
    file.write(response.content)
print("File downloaded successfully!")

Explanation:

1. response.content:

• Returns the binary content of the file (e.g., for PDFs or images).

2. Writing to a file:

• Saves the content to dummy.pdf in the current directory using wb (write binary mode).

Output:

File downloaded successfully!

A file named dummy.pdf will be saved in your current directory.

---

5. Handling Errors and Timeouts

When dealing with external servers, requests might fail. It’s important to handle such errors gracefully.

Code Example:

import requests

url = "https://api.github.com/nonexistent"

try:
    response = requests.get(url, timeout=5)  # Timeout after 5 seconds
    response.raise_for_status()  # Raise an exception for HTTP errors
    print("Response:", response.json())
except requests.exceptions.Timeout:
    print("The request timed out!")
except requests.exceptions.HTTPError as err:
    print("HTTP error occurred:", err)
except requests.exceptions.RequestException as err:
    print("Other error occurred:", err)

Explanation:

1. Timeout: Limits how long the program waits for a response.
2. raise_for_status(): Throws an exception for HTTP errors like 404.
3. Error Handling: Categorizes errors (e.g., timeout, HTTP errors) for better debugging.

Output (Example for a 404 Error):

HTTP error occurred: 404 Client Error: Not Found for url: https://api.github.com/nonexistent

---

6. Working with JSON Data

You can send JSON data in a POST request and receive JSON responses.

Code Example:

import requests

url = "https://httpbin.org/post"
json_data = {"key": "value", "name": "Python Requests"}
response = requests.post(url, json=json_data)

print("Response JSON:", response.json())

Explanation:

1. json: Automatically converts the dictionary to JSON format and sets the Content-Type header to application/json.

2. Response: The server echoes the JSON data you send.

Output:

Response JSON: {
  "args": {},
  "data": '{"key": "value1", "name": "Python Requests1"}'
  "files": {},
  "form": {},
  "headers": {
    ...
  },
  "json": {
    "key": "value",
    "name": "Python Requests"
  },
  "url": "https://httpbin.org/post"
}

---

The requests library is a powerful and easy-to-use tool for making HTTP requests in Python. It covers a wide range of use cases, such as:

1. Fetching data from APIs (GET requests).
2. Sending data to servers (POST requests).
3. Handling errors and timeouts.
4. Downloading files or automating tasks like form submissions.

What Next?

• Explore advanced topics like authentication (BasicAuth, OAuth).
• Use APIs like OpenWeatherMap, Twitter, or GitHub to practice.
• Learn web scraping with requests + BeautifulSoup.