Running a git binary in AWS Lambda
In some scenarios it can be easier to use a binary in a lambda function. In this blog post I will walk you through how you can run a git binary in an AWS Lambda.
For this blog post I will use the AWS Serverless Application Model(AWS SAM). AWS SAM makes it easy to build, package and deploy lambda functions. The used template can be found on GitHub. AWS SAM will look at the Runtime
of the AWS::Serverless::Function
resource. In this example I used python so it will look at the requirements.txt. It will look at the packages defined in the file. AWS SAM installs them in .aws-sam/build/<ResourceName>
. This folder is compressed and uploaded to Amazon S3 when you package and deploy the template.
You can also invoke the function on your own computer. This makes it easier to develop and troubleshoot. To invoke the function we first need to build it. Afterwards you need to invoke it. I created a Makefile, for more information on this you can read one of my previous blogs.
make invoke
You will see that you will see the following error:
{
"errorMessage": "[Errno 2] No such file or directory: 'git'",
"errorType": "FileNotFoundError"
}
And that makes sense right? We do not have the git client installed. You cannot install it using the requirements.txt
file. This is because it is not in the PyPi package manager.
Install the git client
If you look at the Makefile you can see that it will:
- Build a docker container.
- Install git on the container.
- Move all the dependencies into the
/var/task
structure. - Zip the content of the
/var/task
folder.
- Extract the zip file from the container.
- Unzip the contents into the
git_client
folder. sam build
will copy the content to.aws-sam/build/GitClient
.sam build
installs the dependencies from therequirements.txt
.sam invoke
will use the.aws-sam/build/GitClient
content to invoke the function.
The Git binary is located in /var/task/bin
. Because when we invoke the git command we also supply the /var/task/bin
as PATH
. This allows us to call git
and the container knows it needs to invoke /var/task/bin/git
. The libraries are located in /var/task/lib
. Because we have set the LD_LIBRARY_PATH
to /var/task/lib
the libraries can be found.
I created a Dockerfile that uses public.ecr.aws/lambda/python:3.8
as the base. This is the same runtime of the Lambda function. I prepare 2 folders /var/task/bin
and /var/task/lib
. Then I install the git
and zip
packages within the docker context.
I then copy the /usr/bin/git
and the /usr/libexec/git-core/git-remote-http
executables to the /var/task/bin
folder.
When you try to build the Lambda function and run it it fails on a missing library.
{
"errorMessage": "git: error while loading shared libraries: libpcre2-8.so.0: cannot open shared object file: No such file or directory",
"errorType": "Exception"
}
By using the ldd
command you can find all the shared libraries. We will use this to identify the needed libraries and copy them into the /var/task/lib/
folder.
Once you have all dependencies in place. You can fetch the git --version
running inside of the Lambda function.
Git version: git version 2.32.0
If you combine this with the awscli and git-remote-codecommit packages. You can clone a CodeCommit repository and perform your actions and commit it again.
Conclusion
It is possible to embed binaries in AWS Lambda and use them. It might take you some effort. But it opens new possibilities that you might not considered before.
In my next blog you will read how you can use this to synchronize two CodeCommit repositories.
Photo by Emile Perron on Unsplash