Jul 3, 2019

How to Handle Email Bounces on AWS SES

To help prevent fraud and abuse, AWS places all new accounts into the Amazon SES sandbox. There is a list of actions that you need to take in order to request moving out of the sandbox. One of the requirements is to handle bounces and complaints. We are going to automate this process by employing SQS, SNS and serverless framework.

Follow AWS qucikstart on serverless homepage to create new service. I will provide link to the source code at the end of this article.

We need to create SQS queue, SNS topic and subscribe SQS to SNS. Instead of doing it manually we will update `serverless.yml` file and let CloudFormation create resources for us. 

resources:
  Resources:

    EmailBouncesQueue:
      Type: "AWS::SQS::Queue"
      Properties:
        QueueName: "EmailBouncesQueue"

    EmailBouncesTopic:
      Type: AWS::SNS::Topic
      Properties:
        DisplayName: "Bounces topic"
        TopicName: "EmailBouncesTopic"

    BouncesQueueSubscription:
      Type: AWS::SNS::Subscription
      Properties:
        TopicArn: 
          Ref: EmailBouncesTopic
        Endpoint: 
          Fn::GetAtt:
            - EmailBouncesQueue
            - Arn
        Protocol: sqs
        RawMessageDelivery: 'true' </pre>

Let's define function with the SQS event trigger. Update `serverless.yml` with the following configuration:

 handle-bounces:
    handler: bounces.handler
    role: HandleBouncesLambdaRole
    events:
      - sqs:
          arn:
            Fn::GetAtt:
              - EmailBouncesQueue
              - Arn

We will define HandleBouncesLambdaRole later.

Go to AWS SES service console and update configuration to connect Bounce Notifications SNS Topic with the newly created topic named `EmailBouncesTopic`. Instructions how to do that can be found here.

Let's create lambda function that is going to handle bounced emails for us. Here is the code: 

const asyncForEach = async (array, callback) => {
  for (let index = 0; index &lt; array.length; index++) {
    await callback(array[index], index, array);
  }
}

const handler = async (event, context) => {
  const { notificationType, bounce }  = JSON.parse(event.Records[0].body);
  if(notificationType == 'Bounce' &amp;&amp; bounce.bounceType == 'Permanent') {
    await asyncForEach(bounce.bouncedRecipients, async ({ emailAddress }) => {
    // here we have bounced emailAddress... do whatever you have to do 
    });
  }
}

module.exports = { handler }

Last step is little bit more complicated if you are not very familiar with CloudFormation and IAM. We have to create policies that will allow our resources to communicate and interact with other resources. Default lambda role that is generated by serverless framework will not be enough for our use case so we will need to create our custom role. What we need is to:

  • allow lambda to access and write CloudWatch logs
  • allow SQS to execute lambda
  • allow SQS to receive message from SNS 
   SnsToSqsSendMessagePolicy:
      Type: AWS::SQS::QueuePolicy
      Properties:
        PolicyDocument:
          Version: "2012-10-17"
          Statement:
            - Sid: "allow-sns-messages"
              Effect: Allow
              Principal: "*"
              Resource: !GetAtt
                - EmailBouncesQueue
                - Arn
              Action: "SQS:SendMessage"
              Condition:
                ArnEquals:
                  "aws:SourceArn": !Ref EmailBouncesTopic
        Queues:
          - Ref: EmailBouncesQueue

    HandleBouncesLambdaRole:
      Type: AWS::IAM::Role
      Properties:
        Path: /bounces/
        RoleName: HandleBouncesLogAccessRole
        AssumeRolePolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Principal:
                Service:
                  - lambda.amazonaws.com
              Action: sts:AssumeRole
        Policies:

          - PolicyName: HandleBouncesLogPolicy
            PolicyDocument:
              Version: '2012-10-17'
              Statement:
                - Effect: Allow
                  Action:
                    - logs:CreateLogGroup
                    - logs:CreateLogStream
                    - logs:PutLogEvents
                  Resource: 
                    - 'Fn::Join':
                      - ':'
                      -
                        - 'arn:aws:logs'
                        - Ref: 'AWS::Region'
                        - Ref: 'AWS::AccountId'
                        - 'log-group:/aws/lambda/*:*:*'

          - PolicyName: HandleBouncesSQSPolicy
            PolicyDocument:
              Version: '2012-10-17'
              Statement:
                - Effect: Allow
                  Action:
                    - sqs:SendMessage
                    - sqs:ReceiveMessage
                    - sqs:DeleteMessage
                    - sqs:GetQueueAttributes
                    - sqs:ChangeMessageVisibility
                  Resource: 
                    - 'Fn::Join':
                      - ':'
                      -
                        - 'arn:aws:sqs'
                        - Ref: 'AWS::Region'
                        - Ref: 'AWS::AccountId'
                        - 'EmailBouncesQueue'       

          - PolicyName: HandleBouncesLambdaPolicy
            PolicyDocument:
              Version: '2012-10-17'
              Statement:
                - Effect: Allow
                  Action:
                    - lambda:CreateEventSourceMapping
                    - lambda:ListEventSourceMappings
                    - lambda:ListFunctions
                  Resource: 
                    Fn::Sub: 'arn:aws:lambda:${{opt:region}}:${AWS::AccountId}:function:handle-email-bounces-aws-ses-${{opt:stage}}-handle-bounces'

Now we are ready to deploy our service. On successful deployment CloudFormation will create Lambda function, SQS queue, SNS Topic, subscribe SQS to SNS topic and IAM role with our policies.

Amazon SES includes a mailbox simulator that you can use to test how your application handles different email sending scenarios. Send test email to `bounce@simulator.amazonses.com` using AWS SES Console in order to test handling bounced email.

BONUS: Serverless framework is using same variable syntax as CloudFormation which creates conflict when using intrinsic functions. Have a look at this issue. You can use serverless-pseudo-parameters plugin or as suggested in the thread above to customize serverless variable format.

 variableSyntax: '\${{([\s\S]+?)}}'

Now we can continue using CloudFormation variables like we used to while referencing serverless variables looks little bit different. 

 region: ${{opt:region}}

Complete source code can be found here:
https://github.com/bind-almir/handle-email-bounces-aws-ses