How to create Python unit tests for AWS using the botocore stubber

Although you can create Python unit tests for AWS using the botocore stubber, using it is quite difficult. You have to record the requests and responses, add them to your code, and configure the stub with them. In this blog I introduce you to the botocore-stubber-recorder which makes this pretty easy.

how to record all AWS API calls?

To record all AWS API calls is really simple. Just create a recorder for the session, as shown below:

import boto3
from botocore_stubber_recorder import BotoRecorder

session = boto3.session.Session()
recorder = BotoRecorder(session)
# ... do your thing with the session
for index, call in enumerate(recorder.calls):
   print(f"call {index +1}: {call.service_name}:{call.operation}")

Generate a unit tests for recorded AWS calls

You use the recorded calls to generate a unit test, as follows:

from botocore_stubber_recorder import UnitTestGenerator

generator = UnitTestGenerator(name="my_example",directory="./tests")
generator.generate(recorder, anonimize=True)

This will generate the following file structure:

tests
└── my_example
    ├── __init__.py
    ├── base.py
    ├── call_00001_describe_regions
    │   └── __init__.py
    ├── call_00002_describe_parameters
    │   └── __init__.py
    ├── call_00003_...
    │   └── __init__.py
    └── test_my_example.py

The base.py contains a base unit test class which initializes the stub with all the recorded calls. Note that the base.py and the call directories are overwritten each time you generate. The test_my_example.py contains an example test implementation, which you need to change to an actual test. The generated test just tests that the generated stub.

Run the generated test

You can already run the generated test:

cd tests
python -munittest my_example/test_my_example.py

WARNING:root:TODO: replace MyExampleUnitTest.test_my_example with the actual test. This just tests the generated code.
.
----------------------------------------------------------------------
Ran 1 test in 0.092s

OK

Make sure you change the test in tests/my_example/test_my_example.py to an actual unit test for your code.

all at once

To record and generate the unit test in a single command, use:

import boto3
from botocore_stubber_recorder import BotoRecorderUnitTestGenerator

session = boto3.session.Session()
with BotoRecorderUnitTestGenerator("my_example", session) as generator:
    ## do your thing with the session

install the library

To install the library in your environment, type:

pip install botocore-stubber-recorder

Conclusion

The botocore-stubber-recorder makes it very easy to create unit test for Python code which calls AWS services. Image by jplenio from Pixabay