AWS Lambda – Custom Runtime

At re:Invent 2018, AWS announced the Lambda Runtime API and Lambda Layers, two new AWS Lambda features that enable developers to build custom runtimes, and share and manage common code between functions. Lambda Runtime API makes it possible to run any runtime, you can even use Bash as your runtime to power your AWS API Gateway Functions! Lambda Layers are a new type of artifact that can contain arbitrary code and data and are used to provide common dependencies to your lambda functions. Layers can provide bootstrap runtimes and runtime dependencies. In this blog we’ll look at how to use Bash to power an API Gateway Lambda.

New Runtime Setting

The resource type ‘AWS::Lambda::Function’ has a new value for ‘Runtime’, which is ‘provided’. The ‘Handler’ field must point to a filename and a function within the file to be executed.

  CustomLambda:
    Type: AWS::Lambda::Function
    Properties:
      Handler: function.handler
      Runtime: provided
      Role: !GetAtt 'LambdaBasicExecutionRole.Arn'
      MemorySize: 128
      Timeout: 30
      Code:
        S3Bucket: !Ref S3Bucket
        S3Key: !Ref S3Key
        S3ObjectVersion: !Ref S3Version

Bootstrap

When AWS schedules serverless compute, an executable file with the name bootstrap is run. How bootstrap is implemented is arbitrary. It could be a static binary created with Golang or Rust. It can be an executable Bash script. The purpose of bootstrap is to manage a ‘function’ that will be called by the bootstrap. How you wish to call the function is up to you. The function could be part of ‘bootstrap’ ie. you could have a single application created in Go, or you could create a generic handler that calls functions that have been specified in the CloudFormation configuration ie. the ‘Handler’ field.
Bootstrap is responsible for handling the execution of the lambda function. This means it should communicate with the ‘AWS Lambda Runtime API’ to get the event that is ready to be processed by the lambda. Boostrap then invokes the lambda function with the event as an argument. After the function has completed, the response should be returned to the ‘AWS Lambda Runtime API’ in the form of a resposne with a REQUEST_ID. There is more to Lambda custom runtime and for more information you should read the Lambda custom runtime documentation.
In our example, the handler is implemented in Bash:

#!/bin/sh

set -euo pipefail

# Initialization - load function handler
source $LAMBDA_TASK_ROOT/"$(echo $_HANDLER | cut -d. -f1).sh"

# Processing
while true
do
  HEADERS="$(mktemp)"
  # Get an event
  EVENT_DATA=$(curl -sS -LD "$HEADERS" -X GET "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/next")
  REQUEST_ID=$(grep -Fi Lambda-Runtime-Aws-Request-Id "$HEADERS" | tr -d '[:space:]' | cut -d: -f2)

  # Execute the handler function from the script
  RESPONSE=$($(echo "$_HANDLER" | cut -d. -f2) "$EVENT_DATA")

  # Send the response
  curl -X POST "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/$REQUEST_ID/response"  -d "$RESPONSE"
done

Function

The function is a simple Bash script that returns ‘Hello World’ to API Gateway:

function handler () {
  EVENT_DATA=$1
  echo "$EVENT_DATA" 1>&2;
  RESPONSE="{"statusCode": 200, "body": "Hello World"}"
  echo $RESPONSE
}

Example

An example project has been prepared so you can try out working with custom runtimes. To run the example type make dist && make create. To invoke the Bash-based lambda type make invoke. To remove the example type make delete.

Conclusion

AWS Lambda custom runtime makes it possible to define your own bootstrap process. This makes it possible to deploy a highly specialized application when the lambda gets invoked. We have created our own Bash based bootstrap application. We have configured a lambda with CloudFormation that defines a provided runtime and we have created a Bash based handler that processes API Gateway proxy events.
Now, how to put a lambda, inside a Lambda…