Cloud Computing is where the market is migrating. Therefore, we’re here to build an application by leveraging the power of cloud services and see how efficiently we can monitor it.

What Is AWS Lambda?

AWS Lambda is a serverless compute service that triggers the execution of code in response to events. The code that executes on AWS Lambda is called a lambda function and allows the user to focus on code without having to worry about the backend, infrastructure, and scalability. It can also extend the capability of various AWS services, meaning, for example, it can work as an API backend serving incoming requests from API Gateways.

What Is Flask?

Flask is the popular microweb framework based on Werkzeug and Jinja 2, particularly written in Python. It is also known as a lightweight WSGI (Web Server Gateway Interface) web framework and is used to deploy a number of applications ranging from simple to complex. It is relatively simple, easy to use, and well documented. Available under a BSD license, it’s also actively developed and maintained. Flask is aimed at building web applications as well as web APIs and is extensible, hence we can plug in any ORM.

Application Overview

We will build a simple Python Flask application by leveraging the power of AWS cloud services such as Lambda function, Layers, and EC2 instances. The goal of the application is to convert color images (RGB) to grayscale images using a web API and serverless compute. So we will use Flask to create the web API endpoint, and our logic will reside in a lambda function. Finally, we will monitor our API as well as the serverless Lambda function using Epsagon’s insightful dashboard. Below is the architecture diagram of the application that we are going to build with AWS Lambda Flask.

AWS Lambda Flask Architecture

AWS Lambda Flask Architecture

Application Development

Monitor and troubleshoot Lambda and Flask with Epsagon today!

While developing this application, you will interact with AWS services such as S3 bucket and AWS Lambda.

S3 Bucket

Keeping the architecture diagram in mind, create an S3 bucket with two directories: colorImage and grayscaleImage. You can then set a trigger on colorImage, and the output will be stored in grayscaleImage. Let’s name the bucket epsagon-image-process.

Deploying an OpenCV Layer

Your first step will be to create layers since we will require OpenCV as an external package for image processing. So, go ahead and create Lambda layer for OpenCV. You use an EC2 instance to create a package for OpenCV and upload it to S3 bucket. For this, you’ll have to create an IAM role. Navigate to IAM management console, and create a new role for EC2 service with AmazonS3FullAccess permissions. Give an appropriate name (e.g. ec2_s3_access). We will use the name “ec2_s3_access” for the purpose of this article.

To create Lambda Layers, you’ll need a package in a zip file, which you will create in the next step. Before moving on to the next step, you can create the S3 bucket or use an existing bucket (e.g. epsagon-opencv-layer) to store the package.

After creating the S3 bucket, navigate to EC2 Management Console and spin up a t2.micro instancewith Ubuntu Server 18.04 LTS operating system. In step three of creating an instance, make sure to assign the IAM role (ec2_s3_access) that we created in the first step of Deploying OpenCV layer. The rest of the configuration for spinning up an instance remains unchanged.

Deploying an OpenCV Layer

Deploying an OpenCV Layer

After spinning up the instance, SSH to connect to the instance. You can then execute a series of commands to create a package and upload it to the S3 bucket (epsagon-opencv-layer).

sudo apt-get update
python3 -V # to check the version of Python since we are creating the layers for Python 3.6 runtime
sudo apt-get install python3-pip # installing Python package manager
mkdir -p build/python/lib/python3.6/sites-packages # creating the directory structure
pip3 install opencv-python -t build/python/lib/python3.6/sites-packages/ –system # installing OpenCV package/module in the custom directory, i.e. sites-packages
cd build
zip -r package.zip . # creating package in the form of zip file
aws s3 cp package.zip s3://epsagon-opencv-layer # uploading package to S3 bucket

Once the upload is finished, you will navigate to the Lambda Management Console -> Layers -> Create Layers. The configuration looks like this.

Configuring the AWS Lambda Layer

Configuring the AWS Lambda Layer

Click on Create Layer, and now you can terminate the instance.

Setting Up a Lambda Function

A lambda function is where our logic resides. But before creating the lambda function, you have to create an IAM role. So, navigate to the IAM management console, and create a new role for Lambda service with AWSLambdaExecute permissions. Make sure to give an appropriate name (e.g. lambda_role). We will use the name “lambda_role” for the purpose of this article.

To create the function, navigate to Lambda Management console -> Functions -> Create Function. You will select “Author from scratch” and fill in the basic information.

Setting Up an AWS Lambda Function

Setting Up an AWS Lambda Function

Note: Make sure to increase the Memory of your lambda function to 1024 MB and the timeout field to five minutes.

For the next step, you’ll need to bind opencv-layer with our lambda function. From the designer pane, click on Layers -> “Add  layer to function.” Select the layer and its version and click on Add.

Adding the Layer to the Function

Setting Up the Trigger

Now you’re ready to set up a lambda trigger on the colorImage folder in the bucket (epsagon-image-process) that you created in the very first instance of your application building process.

Setting Up AWS Lambda Trigger

Setting Up AWS Lambda Trigger

Image Processing Code

The lambda function is responsible for converting a color image into a grayscale image. Therefore, you will write the image processing code in the lambda function on the console itself. Paste the code below into the lambda_function file.

import json
import boto3
import cv2
import numpy as np

def lambda_handler(event, context):
  s3 = boto3.client("s3")
  if event:
      # extracting event records
      file_obj = event["Records"][0]
      # extracting bucket name
      bucketname = str(file_obj['s3']['bucket']['name'])
      # extracting filename
      filename = str(file_obj['s3']['object']['key'])
      # retrieving file object
      fileObj = s3.get_object(Bucket=bucketname, Key=filename)
      # reading the file object
      file_content = fileObj["Body"].read()
      
      # converting bytes to 1-D array
      np_array = np.fromstring(file_content, np.uint8)
      # reading an image from buffer
      image_np = cv2.imdecode(np_array, cv2.IMREAD_COLOR)
      # converting color  image to grayscale
      gray = cv2.cvtColor(image_np, cv2.COLOR_BGR2GRAY)
      # writing image in tmp directory
      cv2.imwrite("/tmp/gray_obj.jpg", gray)
  filename = filename.split("/")[1]
      # uploading output to s3 bucket
      s3.put_object(Bucket=bucketname, Key="grayscaleImage/{}".format(filename), Body=open("/tmp/gray_obj.jpg", "rb").read())
      
  return {
      'statusCode': 200,
      'body': json.dumps('Image processed successfully!')
  }

The above snippet will process the image and save the output back to the S3 bucket in the grayscaleImage folder.

Now let’s move on to API Development for the § image processing application.

Monitor and troubleshoot Lambda and Flask with Epsagon today

Monitoring With Epsagon

Epsagon enables developers to closely keep track of logs, traces, and invocations via an insightful dashboard. Here is how you can integrate Epsagon with AWS Lambda function in order to easily monitor your lambda function as well as the flask application. Epsagon brings an agentless monitoring tool to the table and also allows you to set up an alert mechanism for code exceptions, function timeouts, out of memory alerts, function errors, etc. Let’s start with AWS Lambda.

Monitoring AWS Lambda

For distributing tracing, you have to instrument your functions under a single application name. Epsagon monitors all the functions without instrumentation but at an individual level. It also supports auto-instrumentation of the functions for Python and Nodejs. Or you can instrument the functions manually using layers. Below is an example of auto-instrumenting the lambda function  with ImgProc as the application name.

Auto-Instrumenting Lambda Functions Using Epsagon

Auto-Instrumenting Lambda Functions Using Epsagon

Monitoring a Flask Application

Instrumenting the flask application works manually. Therefore, you will need to add a few lines in our flask application code. But before doing this, you have to grab the token key from here.

Setting the Epsagon Token

Setting the Epsagon Token

Once you have the token, you will go back to your instance, install Epsagon, and navigate to flaskapp.

pip3 install epsagon

It’s time to update app.py file. The final app.py looks like this.

import os
from flask import Flask, render_template, request
from werkzeug import secure_filename
import boto3
s3 = boto3.client("s3")
bucketname = "epsagon-image-process"
app = Flask(__name__)
UPLOAD_DIR = "images"
@app.route('/upload', methods = ['GET', 'POST'])
def upload_file():
  if request.method == 'POST':
      f = request.files['file']
      filename = str(os.getpid())+"_"+f.filename
      f.save(os.path.join(UPLOAD_DIR, filename))
      file_path = os.path.join(UPLOAD_DIR, filename)
      s3.put_object(Bucket=bucketname, Key="colorImage/{}".format(filename), Body=open(file_path, "rb").read())
      os.remove(file_path)
      return 'file uploaded successfully'

Please make sure to replace your-token-here with the actual token. Now, it’s time to look at some visuals. Let’s start with the architecture that Epsagon has built for AWS Lambda Flask.

Tracing AWS Lambda Flask in Epsagon

Tracing AWS Lambda Flask in Epsagon

You can also look at a few traces and even click on an individual trace to get in-depth information.

Deep-Dive into Traces in Epsagon

Deep-Dive into Traces in Epsagon

Epsagon monitoring is helpful in tracing errors and failures and identifying issues. In this way, it helps you find the root cause of a failure. Without Epsagon, it can be challenging even in the first instance to identify and categorize error-containing logs.

Pros and Cons of AWS Lambda Flask

So, overall what are the advantages and disadvantages of using AWS Lambda Flask.

Pros:

  • Flask is one of the best frameworks for building an API, just like the one we created here with a couple of lines of code.
  • The architecture is decoupled. In our case, every component is independently connected.
  • It is easy to integrate.

Cons:

  • AWS Lambda Flask is not cost-effective since we have to keep our EC2 instance running, which is billed hourly.
  • It is hard to debug the cause of failure without a monitoring tool.

What We Learned

With this exercise, you should have been able to grab a high-level of understanding of AWS Lambda Flask and been able to set up the image processing application in Python. Along the way, you learned how to set up a lambda function, layers, and a server for the flask application and how to deploy a flask endpoint as an API with respect to the architecture diagram. Last but not least, you saw how you can monitor your application with Epsagon’s insightful dashboard in order to stay on top of any errors and issues.

Start your Epsagon Free Trial today