How to create and deploy an AWS CloudFormation custom provider in less than 5 minutes

In this blog I will show you how to create and deploy an AWS CloudFormation custom provider in less than 5 minutes using a Python copier template.

It is not hard to create a custom resource in CloudFormation, it just requires you to implement three method: create, update and delete in a Lambda. However, the entire setup to create and maintain a custom resource is quite laborious. I used to copy and edit existing setups, but this became to error prone and laborious. I contemplated a cookie-cutter setup for a while, but the fact that you cannot update backed cookies held me back. The copier does allow you to update previously created projects, so I wanted to give it a try. And I must say, I really like it.

The template has the following features: - create the source code for a CloudFormation resource provider quickly - support for semantic versioning using git-release-tag - distribute lambdas to buckets in all AWS regions in the world - re-recordable unit tests using botocore stubber recorder - deployable AWS Codebuild pipeline

getting started!

Let's say you want to create a custom resource for a Custom Domain of an AWS AppRunner service, because it does not yet exist. To get started, type:

pip install copier
copier  https://github.com/binxio/cloudformation-custom-provider-template /tmp/cfn-app-runner-custom-domain-provider

  the name of your custom resource type?
   AppRunnerCustomDomain
  The name of your resource provider project?
   cfn-app-runner-custom-domain-provider
  The name of your Python module?
   cfn_app_runner_custom_domain_provider
  a short description for the custom provider?
   manages app runner custom domains 
  Python version to use
   3.9
  Your full name?
   Mark van Holsteijn
  Your email address?
   [email protected]
  the URL to git source repository?
   https://github.com/binxio/cfn-app-runner-custom-domain-provider
  the AWS profile name
   integration-test
  the AWS region name
   eu-central-1
  prefix for the S3 bucket name to store the lambda zipfiles?
   binxio-public
  Access to lambda zip files?
   public

> Running task 1 of 1: [[ ! -d .git ]] &&  ( git init &&  git add . &&  git commit -m 'initial import'  && git tag 0.0.0) || exit 0
Initialized empty Git repository in /tmp/cfn-app-runner-custom-domain-provider/.git/
[main (root-commit) b2ce863] initial import
 21 files changed, 619 insertions(+)
...
````
This creates a project with a working custom provider for the resource AppRunnerCustomDomain. Change to 
the directory and type make deploy-provider and make demo. Your provider will be up-and-running
in less than 5 minutes!

## what is in the box
When you type make help, you will get a list of all of available actions.

```text
build                -  build the lambda zip file
fmt                  -  formats the source code
test                 -  run python unit tests
test-record          -  run python unit tests, while recording the boto3 calls
test-templates       -  validate CloudFormation templates

deploy               -  AWS lambda zipfile to bucket
deploy-all-regions   -  AWS lambda zipfiles to all regional buckets
undeploy-all-regions -  deletes AWS lambda zipfile of this release from all buckets in all regions

deploy-provider      -  deploys the custom provider
delete-provider      -  deletes the custom provider

deploy-pipeline      -  deploys the CI/CD deployment pipeline
delete-pipeline      -  deletes the CI/CD deployment pipeline

deploy-demo          -  deploys the demo stack
delete-demo          -  deletes the demo stack

tag-patch-release    -  create a tag for a new patch release
tag-minor-release    -  create a tag for a new minor release
tag-major-release    -  create a tag for new major release

run the unit tests

To run the unit tests, type:

make test

The unit test will test the scaffold implementation generated by the botocore stubber recorder. To create unit tests for your resource, edit the source code in ./tests/. To implement your custom resource, edit the source code under ./src/.

re-recordable unit tests

Once you have your custom resource provider, it undoubtedly does some AWS API calls. The botocore stubber recorder will allow you to create unit test by running the test against a real account. The tests will record the actual calls and generate the stubs. To run your unit tests, type:

make test-record

This will run the unit tests and record the AWS calls. To run the unit tests with the newly created stubs, type:

make test

The integration tests are run against the AWS profile and region you specified.

Deploy the zip file to the bucket

To copy the zip file with the source code of the AWS Lambda of the custom resource provider, type:

aws s3 mb s3://<bucket-prefix>-<bucket-region>

As you can see, the zipfile will be copied to a bucket name which consists of the prefix and the region name. This allows the zipfile to be made available for use in all regions.

Deploy the custom resource provider into the account

Now the zip file is available, you deploy the custom resource provider, by typing:

make deploy-provider

This deploys the provider as an AWS Lambda function. To configure the run-time parameters and permissions of the Lambda change the CloudFormation template in the directory ./cloudformation.

Deploy the custom resource demo

To deploy the demo CloudFormation stack using the custom resource provider, type:

make deploy-demo

This deploys an CloudFormation stack with an example custom resource as a CloudFormation stack. the run-time parameters and permissions of the Lambda change the CloudFormation template in the file ./cloudformation/demo.yaml. Change the configuration of the custom resource to match your implementation.

Version your custom resource provider

To version your custom resource provider, you can use the following commands:

make tag-patch-release    -  create a tag for a new patch release
make tag-minor-release    -  create a tag for a new minor release
make tag-major-release    -  create a tag for new major release

This will: - run the pre-tag command in the file ./release - commit all outstanding changes in the workspace - tag the commit with the new version.

To show the current version of the workspace, type:

shell make show-version The utility git-release-tag implements this functionality.

Deploy provider to all regions

To deploy the current version of your provider to all regions, type:

make deploy-all-regions

This assumes you have buckets in all regions with the defined prefix.

Deploy CI/CD pipeline

To deploy the CI/CD pipeline based on AWS Codebuild, make sure that the AWS account can access the source repository. If that is the case, type:

make deploy-pipeline

Now every time you tag a new release, it will automatically be deployed to all regions.

conclusion

This copier template provides everything you need to quickly build, deploy and maintain a new custom AWS CloudFormation Provider! Photo by Saffu on Unsplash