How to pass query strings and parameters with Requests in Python

When you’re dealing with HTTP requests, query strings are the secret sauce that lets you pass extra information to the server without modifying the URL path itself. They come after the question mark (?) in a URL and are made up of key-value pairs separated by ampersands (&). For example, in the URL https://example.com/search?q=python&sort=desc, the query string is q=python&sort=desc. This tells the server you want to search for “python” and sort the results in descending order.

Each key-value pair in a query string encodes parameters that the server can interpret to customize the response. That is how web applications filter data, paginate results, or even toggle features without creating a new endpoint for every combination.

But it’s not as simple as just slapping some text after a question mark. Query strings have to be URL-encoded because certain characters (like spaces, ampersands, or equals signs) can break the URL or cause unintended effects. For example, a space gets converted to %20 or a plus sign (+). This ensures that the data reaches the server intact and unambiguous.

Here’s a quick breakdown of what a typical query string looks like:

?key1=value1&key2=value2&key3=value3

If you’re building or consuming APIs, understanding these parameters isn’t optional; it’s fundamental. It’s how you tell the server what you want or how you want it. Sometimes, query parameters are optional – the server can return a default result if none are provided. Other times, they are mandatory, and missing them might result in an error or an incomplete response.

Keep in mind that query strings are part of the URL, so they are visible in browser history, logs, and sometimes in referrer headers. This means sensitive information should never be sent via query parameters, or else you risk exposing it unintentionally.

Also, there’s a length limit to URLs in browsers and servers, typically around 2,000 characters. This puts a practical cap on how much data you can shove into a query string. For larger data payloads, use POST requests with a request body instead.

To summarize the essentials:

• Query strings are appended after ? in a URL.
• Parameters are key-value pairs separated by &.
• Values must be URL-encoded to avoid breaking the URL.
• Visible in URLs, so don’t put secrets here.
• Subject to length limits.

Understanding these basics sets the stage for using tools like Python’s Requests library properly, which handles most of the encoding and formatting for you. That way, you avoid the common mistake of manually concatenating strings and ending up with broken URLs or injection vulnerabilities.

Using the Requests library to pass parameters the right way

Python’s Requests library makes working with query parameters simpler and safe. Instead of manually appending strings to the URL, you pass a dictionary to the params argument in the requests.get() method. Requests takes care of encoding the keys and values, building the query string, and appending it to your URL correctly. This eliminates common bugs like missing ampersands or improper escaping.

Here’s the canonical way to pass parameters:

import requests

url = "https://example.com/search"
params = {
    "q": "python requests",
    "sort": "desc",
    "page": 2
}

response = requests.get(url, params=params)
print(response.url)

The output URL will be:

https://example.com/search?q=python+requests&sort=desc&page=2

Notice how the space in python requests is automatically encoded as a plus sign (+), which is the standard for spaces in query strings. You don’t have to lift a finger for that.

Passing parameters this way also keeps your code clean and readable. Instead of juggling strings, you just work with dictionaries, which are easier to construct dynamically. For example, if you want to add optional filters based on user input, just conditionally add keys to the params dict.

Requests also supports lists as values to represent repeated parameters. If the server expects multiple values for the same key, you can do this:

params = {
    "tag": ["python", "requests", "http"]
}

response = requests.get(url, params=params)
print(response.url)

This generates:

https://example.com/search?tag=python&tag=requests&tag=http

It’s important to remember that params only applies to GET requests (and some others like DELETE). For POST requests, you usually send data in the body, not in the query string. But if you really need query parameters on a POST, you can still use params and Requests will append them to the URL.

One subtle gotcha is when your base URL already includes query parameters. Requests will merge the existing ones with your params dictionary. For example:

url = "https://example.com/search?existing=param"
params = {"q": "python"}

response = requests.get(url, params=params)
print(response.url)

The resulting URL will be:

https://example.com/search?existing=param&q=python

This behavior is usually what you want, but if you need to override existing parameters, you’ll have to parse and manipulate the URL yourself or avoid passing them twice.

For advanced cases, you can use urllib.parse to build or modify query strings before passing them to Requests. But in most situations, letting Requests handle the encoding and formatting is the cleanest and safest approach.

Handling edge cases and common pitfalls when working with query strings

Even with Requests handling most of the heavy lifting, there are still edge cases and pitfalls you need to watch out for when working with query strings.

First, beware of non-string parameter values. If you pass integers, booleans, or other types directly in the params dictionary, Requests will convert them to strings for you. However, complex objects like lists of dictionaries or custom classes won’t serialize properly and will cause errors or unexpected results.

For example, this will work:

params = {
    "page": 3,
    "active": True
}
response = requests.get(url, params=params)
print(response.url)

Output:

https://example.com/search?page=3&active=True

But this will not:

params = {
    "filters": [{"type": "date", "value": "2023-01-01"}]
}
response = requests.get(url, params=params)

This raises a TypeError because Requests doesn’t know how to encode that list of dictionaries into a query string. In cases like this, you’ll need to serialize your data yourself, such as converting it to JSON and then URL-encoding it explicitly:

import json
import requests

filters = [{"type": "date", "value": "2023-01-01"}]
params = {
    "filters": json.dumps(filters)
}

response = requests.get(url, params=params)
print(response.url)

Resulting URL:

https://example.com/search?filters=%5B%7B%22type%22%3A+%22date%22%2C+%22value%22%3A+%222023-01-01%22%7D%5D

Another common headache is handling None values in query parameters. By default, if you include None as a value, Requests converts it to the string None, which is almost never what you want. It’s better to filter out any parameters with None before passing them:

params = {
    "q": "python",
    "page": None,
    "sort": "asc"
}

clean_params = {k: v for k, v in params.items() if v is not None}
response = requests.get(url, params=clean_params)
print(response.url)

This avoids sending page=None in the URL, which could confuse the server or return unexpected results.

Watch out for encoding issues with special characters like ampersands (&), question marks (?), or Unicode characters. Requests handles URL encoding according to RFC 3986, but if you pre-encode parameters manually, you might end up double-encoding, which breaks the query string.

For example, don’t do this:

params = {
    "q": "C++ programming"
}
# Manually encode value (bad idea)
params_encoded = {k: requests.utils.quote(v) for k, v in params.items()}

response = requests.get(url, params=params_encoded)
print(response.url)

The output will have the plus signs and percent signs encoded again, making the URL messy and incorrect:

https://example.com/search?q=C%252B%252B+programming

Instead, just pass the raw strings and let Requests handle the encoding:

params = {
    "q": "C++ programming"
}
response = requests.get(url, params=params)
print(response.url)

Which outputs the correct URL:

https://example.com/search?q=C%2B%2B+programming

Finally, remember that query strings are part of the URL and thus have length limits imposed by browsers and servers. If you need to send large amounts of data, switch to POST requests with a JSON or form-encoded body instead.

To summarize some critical edge cases:

• Don’t pass complex objects directly; serialize them first.
• Filter out None values to avoid sending unwanted strings.
• Avoid manual encoding of parameters to prevent double-encoding.
• Be mindful of URL length limits when passing large data via query strings.