How to send GET requests using requests.get in Python

HTTP is the backbone of the web, a simple protocol that defines how clients and servers communicate. When you open a browser and type a URL, what happens under the hood is an HTTP request is sent from your browser to the server hosting that site. The server then sends back a response—usually some HTML, images, or data.

Among the various types of HTTP requests, GET is the most common and simpler. It’s the method used when you want to retrieve data from a specific resource without causing any side effects. Think of it as asking the server, “Hey, can you please give me this?” without changing anything on the server’s state.

The structure of a GET request is quite simple. It consists of a request line, headers, and sometimes query parameters embedded in the URL. The query parameters come after a question mark and provide additional information about what you are requesting. For example, if you want to search Google for “python requests,” the URL might look like this:

https://www.google.com/search?q=python+requests

Here, q=python+requests is the query parameter telling Google what to search for.

When the server receives your GET request, it processes the URL and any parameters, then responds with the resource if it exists, or an error if it doesn’t. This response contains status codes that tell you what happened: 200 OK means success, 404 Not Found means the resource doesn’t exist, and 500 Internal Server Error means something went wrong on the server side.

Understanding the stateless nature of HTTP is essential. Each GET request is independent; the server doesn’t remember past requests by default. This helps keep the protocol simple but means that if you’re building something more interactive, you need ways to keep track of user sessions or states explicitly, often via cookies or tokens.

Making your first GET request with requests.get

To make your first GET request in Python, you’ll typically use the requests library, which simplifies the process significantly. First, you’ll need to install the library if you haven’t already. You can do this using pip:

pip install requests

Once you have the library installed, making a GET request is as simple as calling the get function. Here’s a basic example of how to retrieve data from a public API:

import requests

response = requests.get('https://api.github.com')
print(response.status_code)  # Should print 200 if the request was successful
print(response.json())  # Print the JSON response

In this example, we’re sending a GET request to GitHub’s API. The response object contains all the information returned by the server. We check the status code to see if the request was successful, and then we can parse the JSON response using the json() method.

Handling errors is also crucial when working with HTTP requests. While a 200 status code indicates success, other codes can provide important information about what went wrong. For instance, if you encounter a 404, it means the resource was not found, and you should handle that appropriately:

if response.status_code == 200:
    print("Success:", response.json())
elif response.status_code == 404:
    print("Error: Resource not found.")
else:
    print("Error:", response.status_code)

This way, your program can respond gracefully to different scenarios, instead of crashing or behaving unpredictably.

Another important aspect is query parameters. Often, you’ll need to pass additional information with your GET request. The requests library makes this easy by so that you can pass a dictionary of parameters to the params argument:

params = {'q': 'python requests', 'sort': 'stars'}
response = requests.get('https://api.github.com/search/repositories', params=params)
print(response.json())

In this case, we’re searching for repositories on GitHub that match a specific query. The parameters are automatically encoded into the URL, so you don’t have to worry about formatting them yourself. This can save you a lot of time and potential errors.

When you work with APIs, you’ll often find that the response data is structured in JSON format. The requests library provides a convenient way to handle this with the json() method, which parses the JSON response into a Python dictionary. This allows you to access individual elements easily:

data = response.json()
for repo in data['items']:
    print(repo['name'], repo['html_url'])

Here, we’re iterating through the list of repositories returned by the GitHub API and printing their names along with their URLs. This kind of data manipulation is common when interacting with web APIs, and mastering it can greatly enhance your ability to build applications that rely on external data sources.

Handling query parameters and response data effectively

Sometimes the response data isn’t JSON or might not be well-formed. To avoid your program crashing when the response can’t be decoded, wrap the call to json() in a try-except block:

try:
    data = response.json()
except ValueError:
    print("Response content is not valid JSON")
    data = None

This simple pattern guards against malformed data, a common issue when dealing with third-party APIs or unstable network conditions.

Another frequent need is to examine headers in the HTTP response. Headers carry metadata about the response like content type, caching policies, or pagination info:

print(response.headers['Content-Type'])
print(response.headers.get('X-RateLimit-Remaining', 'No rate limit info'))

Using the get method to access headers avoids exceptions when the header isn’t present, providing a more robust way to handle optional metadata.

When crafting GET requests with query parameters, there are a few nuances worth noting. Parameters are passed as key-value pairs, but values must be serializable to strings. For example, lists or special objects need formatting or conversion before sending.

params = {
    'tags': ','.join(['python', 'requests']),
    'page': 2,
    'per_page': 15
}
response = requests.get('https://api.example.com/search', params=params)
print(response.url)

Notice how the URL printed shows the fully encoded query string. That’s a great debugging step to ensure your parameters are sent exactly as you expect.

If you need to send complex nested data, GET requests aren’t ideal since the query string has limited expressiveness. In those cases, consider POST requests with JSON payloads. But for simple filtering, sorting, and pagination, query parameters work neatly.

When dealing with large responses, you might want to stream the data instead of loading it all concurrently into memory. The requests library supports streaming responses by setting stream=True:

response = requests.get('https://example.com/largefile', stream=True)
with open('largefile.zip', 'wb') as f:
    for chunk in response.iter_content(chunk_size=8192):
        f.write(chunk)

This pattern reads the response in manageable chunks and is useful when downloading files or handling very large JSON bodies.

Finally, sometimes the server sends compressed data (gzip or deflate) to save bandwidth. Requests transparently decompresses based on the Content-Encoding header, so you rarely have to handle that yourself. But if you want to inspect the raw body:

raw_data = response.content  # bytes, possibly compressed
text_data = response.text    # decoded based on response encoding headers

Use response.text for text data as it respects the charset declared by the server. Use response.content when you need the raw bytes, like for images or other binary files.