AWS | Cloud
A custom provider for EFS Provisioned Throughput 
Mark van Holsteijn 16 Jul, 2018
Share 
In this blog I will show you how to create and deploy an AWS CloudFormation custom provider
in less than 5 minutes using a Python copier template.
It is not hard to create a custom resource in CloudFormation, it just requires you
to implement three method: create, update and delete in a Lambda. However,
the entire setup to create and maintain a custom resource is quite laborious. I used to copy
and edit existing setups, but this became to error prone and laborious. I contemplated
a cookie-cutter setup for a while, but the fact that you cannot update backed cookies held me back.
The copier does allow you to update previously created projects, so I wanted to give it a try. And
I must say, I really like it.
The template has the following features:
– create the source code for a CloudFormation resource provider quickly
– support for semantic versioning using git-release-tag
– distribute lambdas to buckets in all AWS regions in the world
– re-recordable unit tests using botocore stubber recorder
– deployable AWS Codebuild pipeline
getting started!
Let’s say you want to create a custom resource for a Custom Domain of an AWS AppRunner service,
because it does not yet exist. To get started, type:
pip install copier copier https://github.com/binxio/cloudformation-custom-provider-template /tmp/cfn-app-runner-custom-domain-provider 🎤 the name of your custom resource type? AppRunnerCustomDomain 🎤 The name of your resource provider project? cfn-app-runner-custom-domain-provider 🎤 The name of your Python module? cfn_app_runner_custom_domain_provider 🎤 a short description for the custom provider? manages app runner custom domains 🎤 Python version to use 3.9 🎤 Your full name? Mark van Holsteijn 🎤 Your email address? [email protected] 🎤 the URL to git source repository? https://github.com/binxio/cfn-app-runner-custom-domain-provider 🎤 the AWS profile name integration-test 🎤 the AWS region name eu-central-1 🎤 prefix for the S3 bucket name to store the lambda zipfiles? binxio-public 🎤 Access to lambda zip files? public > Running task 1 of 1: [[ ! -d .git ]] && ( git init && git add . && git commit -m 'initial import' && git tag 0.0.0) || exit 0 Initialized empty Git repository in /tmp/cfn-app-runner-custom-domain-provider/.git/ [main (root-commit) b2ce863] initial import 21 files changed, 619 insertions(+) ... ```` This creates a project with a working custom provider for the resource <code>AppRunnerCustomDomain. Change to the directory and typemake deploy-providerandmake demo. Your provider will be up-and-running in less than 5 minutes! ## what is in the box When you typemake help, you will get a list of all of available actions. ```text build - build the lambda zip file fmt - formats the source code test - run python unit tests test-record - run python unit tests, while recording the boto3 calls test-templates - validate CloudFormation templates deploy - AWS lambda zipfile to bucket deploy-all-regions - AWS lambda zipfiles to all regional buckets undeploy-all-regions - deletes AWS lambda zipfile of this release from all buckets in all regions deploy-provider - deploys the custom provider delete-provider - deletes the custom provider deploy-pipeline - deploys the CI/CD deployment pipeline delete-pipeline - deletes the CI/CD deployment pipeline deploy-demo - deploys the demo stack delete-demo - deletes the demo stack tag-patch-release - create a tag for a new patch release tag-minor-release - create a tag for a new minor release tag-major-release - create a tag for new major release
run the unit tests
To run the unit tests, type:
make test
The unit test will test the scaffold implementation generated by the botocore stubber recorder.
To create unit tests for your resource, edit the source code in ./tests/. To implement your custom
resource, edit the source code under ./src/.
re-recordable unit tests
Once you have your custom resource provider, it undoubtedly does some AWS API calls.
The botocore stubber recorder will
allow you to create unit test by running the test against a real account. The tests
will record the actual calls and generate the stubs. To run your unit tests, type:
make test-record
This will run the unit tests and record the AWS calls. To run the unit tests with the
newly created stubs, type:
make test
The integration tests are run against the AWS profile and region you specified.
Deploy the zip file to the bucket
To copy the zip file with the source code of the AWS Lambda of the custom resource provider, type:
aws s3 mb s3://<bucket-prefix>-<bucket-region>
As you can see, the zipfile will be copied to a bucket name which consists of the prefix
and the region name. This allows the zipfile to be made available for use in
all regions.
Deploy the custom resource provider into the account
Now the zip file is available, you deploy the custom resource provider, by typing:
make deploy-provider
This deploys the provider as an AWS Lambda function. To configure
the run-time parameters and permissions of the Lambda change the CloudFormation
template in the directory ./cloudformation.
Deploy the custom resource demo
To deploy the demo CloudFormation stack using the custom resource provider, type:
make deploy-demo
This deploys an CloudFormation stack with an example custom resource as a CloudFormation stack.
the run-time parameters and permissions of the Lambda change the CloudFormation
template in the file ./cloudformation/demo.yaml. Change the configuration of the custom
resource to match your implementation.
Version your custom resource provider
To version your custom resource provider, you can use the following commands:
make tag-patch-release - create a tag for a new patch release
make tag-minor-release - create a tag for a new minor release
make tag-major-release - create a tag for new major release
This will:
– run the pre-tag command in the file ./release
– commit all outstanding changes in the workspace
– tag the commit with the new version.
To show the current version of the workspace, type:
shell
make show-version
The utility git-release-tag
implements this functionality.
Deploy provider to all regions
To deploy the current version of your provider to all regions, type:
make deploy-all-regions
This assumes you have buckets in all regions with the defined prefix.
Deploy CI/CD pipeline
To deploy the CI/CD pipeline based on AWS Codebuild, make sure that the AWS account can
access the source repository. If that is the case, type:
make deploy-pipeline
Now every time you tag a new release, it will automatically be deployed to all regions.
conclusion
This copier template provides everything you need to quickly
build, deploy and maintain a new custom AWS CloudFormation Provider!
Photo by Saffu on Unsplash
Mark van Holsteijn is a senior software systems architect at Xebia Cloud-native solutions. He is passionate about removing waste in the software delivery process and keeping things clear and simple.
