AWS teams help us to continuously improve our experience and capabilities with Lambda functions. This time, AWS announced the AWS Lambda support for EFS, which gives us a great opportunity to handle much larger files than before. According to the AWS Lambda limits, the `/tmp` directory could handle up to 512MB of data, which in many cases is not sufficient. Other alternatives could be storing these large files in S3 and handling them in batches, which introduces complications, and performance impacts.

With the new release today, EFS can be attached to Lambda functions, and it exposes a new directory under `/mnt/<name>`, that can handle any file size. In addition, these file systems can be shared across multiple functions and services. It can become very useful for AI/ML functions that use large data models, or any other processing functions that handle large files.

Setup EFS File System for Lambda

The setup is relatively simple, and consists of creating an EFS file system, adding permissions to your function, and attaching the EFS file system to your Lambda function.

Creating the file system

Head to the EFS console on AWS, click “Create file system”, select a VPC, and the desired AZs:

Creating the EFS file system for Lambda

In the next step, make sure that the General Purpose is selected, and click next:

General Purpose EFS file system

Now create an access point to the file system. You can see an example details to fill in, but make sure that you understand the configuration, and adjust it accordingly:

creating an access point to the EFS file system

Confirm and create the file system:

confirming the creation of the EFS file system

Now let’s edit the IAM role of the desired Lambda function. Head to the permissions tab and click on your role name to open it in the IAM console:

edit the IAM role of Lambda function

In the IAM console, click on Attach Permissions, and add the following permissions:

  • AWSLambdaVPCAccessExecutionRole
  • AmazonElasticFileSystemClientFullAccess

Your role should look like this:

IAM role for EFS file system for Lambda

Head back to the Lambda function, and at the bottom part you’ll be able to find the new File system section and click on the add file system:

adding EFS file system to lambda function

Insert the file system details that you’ve created, and save:

saving the EFS file system for Lambda

Ultimately it should look like this:

Final EFS file system screen

Now you can access files on the mount you’ve created!

Epsagon Integration

As an AWS Lambda service ready partner, Epsagon makes sure the new configuration is visible for you in the same dashboard where you can see your function metrics and invocations. Once configured, you’ll be able to see the attached file system, and get all of the configuration details:

Epsagon EFS dashboard

And get the full configuration when you click on the file system:

Epsagon file system full configuration

Try it yourself here.

Conclusion

Amazon EFS provides shared file storage in the AWS Cloud. With that, you can create a file system, mount the EFS file system from an AWS Lambda function, in a very simple configuration. This allows us to leverage Lambda functions for more scenarios where we need to handle larger files. When using it, verify to you set the right configuration for the file system and benchmark the impact on your current performance.

 

More Guides by Epsagon:

Amazon ECS Quick Start Guide

How to Effectively Monitor AWS Lambda

Troubleshooting AWS Lambda 101

AWS Lambda and Python Flask – Getting Started