How to Set Up AWS Lambda with SQS (Tutorial & Examples)

Written by Ran Ribenzaft

July 11, 2018

AWS introduced last year the SQS integration with AWS Lambda. In the following post, I’m going to share a quick-and-simple tutorial on how to get started with message distribution between Lambda functions, using SQS.

In addition, I will also compare between SQS and SNS (i.e. why should we choose one over the other), and present an in-depth performance analysis of using SQS as a message distributor.

Setup: AWS Lambda and SQS

I will use the Serverless Framework to send a message from one Lambda function to another Lambda function via SQS.

Diagram

Prerequisites:

npm installed.
Serverless Framework installed.
Python – Feel free to use your favorite programming language.
AWS account (duh).

Let’s start by creating the serverless project:

serverless create --template aws-python3 --path sqs-lambda-tutorial cd sqs-lambda-tutorial

At this point, we can configure the serverless.yml and handler.py files. The code can be found in the following GitHub repo.

serverless.yml

This yml file configures our functions, permissions and SQS resource.

Serverless.yml

handler.py

This is where our code runs. It contains two functions. Each function represents a different AWS Lambda function. One will be triggered manually, and the other one will be triggered by an SQS message.

We will compare the SQS Lambda and the manual Lambda.

AWS Lambda and an SQS trigger

Deploy!

This is the magic of Lambda. Just run the following command while you are in the same folder as serverless.yml:

sls deploy

The output should look roughly like this:

Serverless Framework deployment

Now we can invoke the “start” function by running the following commands:

sls invoke -f start-lambda

sls logs -f start-lambda

sls logs -f end-lambda

You will be able to see the CloudWatch Logs of the Lambda functions with the SQS message.

We are done. Congrats! ⚡️

Event Structure

If you plan to use SQS in your application, you will need the event structure and data. You can find it in the following Gist.

SQS vs. SNS

When applied to Lambda only, this is a tough call. Up to this moment, only SNS supported triggering Lambda functions. Now both SNS and SQS can trigger. Let’s compare the relevant parameters for Serverless applications:

SQS vs. SNS performance

The main advantage of SQS is batch messaging, which can also lead to cost reduction.

Performance Analysis: Serverless and SQS

Distributing messages between services can be complicated. Understanding the performance impact on our applications is almost impossible. At Epsagon, we love to see the bigger picture for highly distributed applications.

In a previous post, we compared several methods of distributing messages between functions (direct sync invocation, direct async invocation, SNS and Kinesis), and now, using our serverless performance monitoring tool, it’s a great time to introduce the results for SQS:

Lambda SQS Latency

Compared to the results from the previous post – this is much better!

Conclusion

Lambda integrations are great. The more, the better. In this post, we learned how to deploy Lambda functions that distribute messages using SQS. We also saw an updated comparison between SQS and SNS for Serverless applications and understood the performance implications of distributing messages between functions.

Considering features, cost, and performance, SQS is the best choice for the task of message distribution between Lambda functions. For recent trends and examples of AWS Lambda and SQS, subscribe to get notified on all the latest insights.

If you’d like to get a demo of Epsagon, please schedule one here.

Epsagon is now part of Cisco, and offers the most advanced monitoring, now FREE for everyone | Read more.

How to Set Up AWS Lambda with SQS – Everything You Should Know!