Testing basics

In a first phase of development, testing is often done manually. However, as work progresses and an API grows, it becomes more and more important to assure yourself that making changes in your API does not 'break' other functionality of your API. To avoid this, writing tests and running these every time development has been done is best practice.

Pytest is a Python based testing framework, which is used to write and execute test codes. In the present days of REST services, pytest is mainly used for API testing even though we can use pytest to write simple to complex tests.

There are some key advantages to pytest:

It is available as a free and open source library for Python
Pytest has its own way to detect the test file and test functions automatically
Pytest can run multiple tests in parallel, which reduces the total execution time
Pytest allows us to run a subset of the entire test, giving you the possibility to only run those tests that matter in a specific scenario

Running pytest without mentioning a filename will run all files of format test_*.py or *_test.py in the current directory and subdirectories. Pytest requires the test function names to start with test. Function names which are not of format test* are not considered as test functions by pytest. We cannot explicitly make pytest consider any function not starting with test as a test function.

Tests make use of assert-statements. These are boolean expressions which will yield either True or False. The test code will continue running when the outcome of an assert-statement is True. When it is False however, an AssertionError will be thrown by Python, which will make sure pytest handles the test as a failed test.

We will start with an easy example based on the randomizer.pyopen in new window file we have used a few times before.

Put the randomizer.py file in a new folder and create a new file called test_randomizer.py in the same folder.

Installations

Install pytest using the following command:

pip install pytest

Within the test file, we will call our API as if we didn't know the files were in the same folder. We will use the requests library to perform requests towards our API and read out the result of those requests.

Install the requests library using the following command:

pip install requests

Writing tests

First test

In test_randomizer.py we will create a first test function that will test the GET method for the /percentage endpoint. We will check for two things to start with: whether the status code of the response is 200 ("OK") and whether the percentage we get back from the request is a number between 0 and 100. Besides, we will test that we get a whole number (integer) as value for the key 'percentage'.

import requests
import json

def test_get_random_percentage():
    response = requests.get('http://127.0.0.1:8000/percentage')
    assert response.status_code == 200
    response_dictionary = json.loads(response.text)
    assert type(response_dictionary["percentage"]) == int
    assert 0 <= response_dictionary["percentage"] <= 100

In the code above, a GET request is launched to our API which will be running locally. Immediately after that, an assert-statement will check whether the status code of the request equals 200 ("OK"). If not, the test will fail immediately. In case the first assert-statement succeeds, the response body is read out and converted into a dictionary. A new assert-statement will check whether the value of the "percentage" key is an integer and then yet another assert-statement will check if that value is within an acceptable range.

Make sure your API is running locally using the command uvicorn randomizer:app --reload. Next, open a new terminal in the same folder and execute the following command:

pytest

Running the pytest command will run all tests within all test files that are located in the same folder or subfolder. Beware that functions are only considered as test functions when they start with the word test.

pytest

The output shows that all tests (just one now) passed, which means they succeeded. If you want some more details about the tests that were run, you can run the command in verbose mode:

pytest -v

pytest verbose

Creating multiple test scenarios

Let's now write a few tests for our second GET request which expects two path parameters. It is common practice that multiple test scenarios are written for one endpoint or request. Every test then covers a different scenario to which the endpoint should react properly.

We will first create a test scenario for a 'normal' scenario in which correct values are passed as path parameters. We can 'recycle' our first test to create our second test and make a few adjustments. Note that two path parameters were added in the GET request and that the range for the percentage (last assert-statement) was changed accordingly.

def test_get_random_percentage_lower_and_upper_limit():
    response = requests.get('http://127.0.0.1:8000/percentage/10/50')
    assert response.status_code == 200
    response_dictionary = response.json()
    assert type(response_dictionary["percentage"]) == int
    assert 10 <= response_dictionary["percentage"] <= 50

Info

In the second test, the response body has been read out in a different way compared to the first test. As a matter of fact, the same result is achieved. response.json() will read out the response body of the response and then convert the JSON code into a Python dictionary, similar to json.loads(response.text).

Run the pytest command again to run all tests.

In a third test, we will try to do something we're not meant to do: use an incorrect upper and lower limit as path parameter. Our API should be able to handle these incorrect values and give an error. Therefore we will create an assert-statement that will check if the status code is different from 200.

def test_get_random_percentage_wrong_lower_and_upper_limit():
    response = requests.get('http://127.0.0.1:8000/percentage/50/10')
    assert response.status_code != 200

When we execute the pytest command, we unexpectedly get a failed test now. In the 'FAILURES' section we can see that our last test went wrong: the status code of the request was 200, but we coded that it shouldn't be 200...

pytest failed test

After further examination, we will see that in the randomizer.py file we didn't program a real error, but passed a dictionary containing an error message instead. In a real life situation, this would prompt you to change your API and build in error handling. However, let's not change our API but our last test instead here:

def test_get_random_percentage_wrong_lower_and_upper_limit():
    response = requests.get('http://127.0.0.1:8000/percentage/50/10')
    response_dictionary = response.json()
    assert 'error' in response_dictionary.keys()

Another application might accidentally address our API endpoint with text instead of numbers. Let's write another test to deal with this situation. In the test given below, we check whether the right status code is given back. When a path parameter or a request parameter is present in a request, FastAPI will use the validations you have programmed to check whether all data types are correct. If something would not be right, FastAPI will throw an error and pass status code 422 ("Unprocessable Entity") in the response header. Let's write a test to check that.

def test_get_random_percentage_lower_and_upper_limit_as_string():
    response = requests.get('http://127.0.0.1:8000/percentage/lower/upper')
    assert response.status_code == 422

Besides checking the content of the response body and the status code, it is also possible to test values in the headers of the response.

We know that our API will be sending back a response in JSON. Therefore, the response header will contain a key 'Content-Type' with the value 'application/json'. At least, that will be the case if our API is functioning well. Let's make a test containing an assert-statement that will check on this key-value pair in the headers of the response. At the same time, we will add some more logic to test the last GET request we defined in our API.

def test_get_random_percentage_with_amount():
    response = requests.get('http://127.0.0.1:8000/percentage/10/70/5')
    assert response.status_code == 200
    assert response.headers['Content-Type'] == 'application/json'
    response_dictionary = response.json()
    assert len(response_dictionary["percentages"]) == 5
    for percentage in response_dictionary["percentages"]:
        assert 10 <= percentage <= 70

.headers returns a dictionary-like object, allowing you to access header values by key. There is something special about this dictionary-like headers object, though. The HTTP spec defines headers to be case-insensitive, which means we are able to access these headers without worrying about their capitalization. That means that the line of code highlighted above could also be written as follows:

assert response.headers['content-type'] == 'application/json'

Whether you use the key 'content-type' or 'Content-Type', you’ll get the same value.

Running specific test scenarios

In most cases, we want all of our tests to run to verify whether our API is still working as expected. In some cases however, it is senseful to run specific test scenarios independently from the others, for example to test one specific functionality. We can do so by passing the parameter -k following a string to the pytest command. If we would at the same time want the command to run in verbose mode, we could use the following command to only run the last test we created:

pytest -k amount -v

Pytest will now only run tests containing the string 'amount' in their function name. Other tests will not run and will be indicated as 'deselected' in the output.

pytest deselected tests

Extra: Using `requests` to handle OAuth tokens

One can also use the requests library alongside an API secured with OAuth. Without going further into detail, here is an example of code that:

Uses the /token endpoint with request data containing the username and password
Extracting the token that came with the response and use it in the headers of a secured endpoint

import json
import requests

headers = {
"accept": "application/json",
"Content-Type": "application/x-www-form-urlencoded"
}

# Account 'test@test.be' with password 'test' has already been pre-made, either manually or with another requests.post
request_data = {
    "client_id": "",
    "client_secret": "",
    "scope": "",
    "grant_type": "",
    "refresh_token": "",
    "username": "test@test.be",
    "password": "test"
}

tokenrequest = requests.post("http://localhost:8000/token", data=request_data, headers=headers)

# Printing the information for debugging and illustration purposes
print(tokenrequest.text)
# Extracting the token that came with the response
token = json.loads(tokenrequest.text)['access_token']

# Using the token in the headers of a secured endpoint
headerswithtoken = {
"accept" : "application/json",
"Authorization": f'Bearer {token}'
}

getrequest = requests.get("http://localhost:8000/users/me", headers=headerswithtoken)

# Printing the information for debugging and illustration purposes
print(getrequest .text)

Testing basics

# Installations

# Writing tests

# First test

# Creating multiple test scenarios

# Running specific test scenarios

# Extra: Using requests to handle OAuth tokens

Description