S3 Bucket Notifications

17 Nov, 2018
Xebia Background Header Wave

Amazon Simple Storage Storage (S3) is a service that stores objects by means of a key and a value. The data associated with a key can easily be accessed by means of a web service API. S3 is a core service of AWS and is perfectly suited for storing any kind of data. S3 supports a feature called ‘Event Notifications’ that enables you to create event driven systems. When certain events happen in your bucket like eg. creating or deleting an object, an event can be sent to SNS or a Lambda function. In this blog we’ll see how to setup a Lambda as an event handler.

CloudFormation Configuration

The CloudFormation configuation is simple. We need to specify a name for the bucket by means of a custom provider that generates a random string. Next we need to specify a permission so the bucket may invoke the Lambda.

    Type: AWS::S3::Bucket
      BucketName: !Ref '${RandomName}'
        - Function: !GetAtt NotificationLambda.Arn
          Event: s3:ObjectCreated:*

    Type: AWS::Lambda::Permission
      FunctionName: !GetAtt NotificationLambda.Arn
      Action: lambda:InvokeFunction
      SourceAccount: !Ref AWS::AccountId
      SourceArn: !Sub 'arn:aws:s3:::${RandomName}'

Custom Provider

To generate a random name we create a custom provider.

    Type: Custom::RandomNameGenerator
      ServiceToken: !GetAtt 'RandomNameGenerator.Arn'

    Type: AWS::Lambda::Function
      Handler: index.lambda_handler
      Timeout: 30
      Role: !GetAtt 'LambdaBasicExecutionRole.Arn'
      Runtime: python3.6
        ZipFile: |
          import string
          import random
          import cfnresponse
          def lambda_handler(event, context):
              if event['RequestType'] == 'Create':
                  event['PhysicalResourceId'] = ''.join(random.choice(string.ascii_lowercase) for _ in range(16))
              cfnresponse.send(event, context, cfnresponse.SUCCESS, {}, event['PhysicalResourceId'])

The Bucket Handler

The event handler for the bucket is a lambda function that prints the received event to CloudWatch logs.

def handler(event, ctx):

ObjectCreated Event

When we upload a file to the bucket, the following event is received by the lambda. The Records field will contain only a single entry and contains the details for the ObjectCreated event like the bucket and key name and where and when the event occurred.

  "Records": [
      "eventVersion": "2.0",
      "eventSource": "aws:s3",
      "awsRegion": "eu-west-1",
      "eventTime": "2018-11-18T12:42:15.922Z",
      "eventName": "ObjectCreated:Put",
      "userIdentity": {
        "principalId": "AWS:AIDAJSZKO5A3SBJFMYXTE"
      "requestParameters": {
        "sourceIPAddress": ""
      "responseElements": {
        "x-amz-request-id": "B9CC7E95824DAC64",
        "x-amz-id-2": "qufqqLvaJ4w7e9UwHdBply4jPEJhwJXbL5qYr0fbhAtZPn0+lfixoN26JC/80uBGpHPxbvDL9XM="
      "s3": {
        "s3SchemaVersion": "1.0",
        "configurationId": "8450cff2-771d-4a5f-9599-255bc97cbd47",
        "bucket": {
          "name": "qmuixbpinyhhlwpd",
          "ownerIdentity": {
            "principalId": "A97CX0CDO58US"
          "arn": "arn:aws:s3:::qmuixbpinyhhlwpd"
        "object": {
          "key": "",
          "size": 215,
          "eTag": "2e59fd3b13286947aa9d4dcaa03bf568",
          "sequencer": "005BF15E27DB726B35"


It is easy to setup S3 Event Notifications. The s3 bucket must have permission to invoke the lambda. The Lambda can contain arbitrary logic that will be executed when something happens with the bucket. Next time we’ll look at how to send received CloudFront log files from an S3 bucket to AWS Elasticsearch service.


Get in touch with us to learn more about the subject and related solutions

Explore related posts