Why DevOps Engineers Love AWS Lambda

AWS Lambda provides a wide range of possibilities for automating DevOps tasks and processes. Prior to AWS Lambda, we relied on a server that executed all required tasks and scripts we put on it. With serverless, we can save a lot of effort.

AWS Lambda supports all of the mainstream programming languages, such as Python, Golang, Node.js, and Java. It’s easy to switch a task that was once running on a server to a Lambda function. AWS Lambda Layers enable developers to use other programming languages in AWS Lambda, in addition to the currently supported runtimes. The Epsagon Layer, for example, is used for automated instrumentation and observability.

In this article, we’ll discuss these use cases in more detail. We will explore some examples to make use of AWS Lambda in the real world.

Backup

With AWS Lambda, it’s easy to back up EBS volumes. Let’s say you have an important server that requires an EBS snapshot every day. Using AWS Lambda, you can schedule a shutdown of the server via its cronjob and create an Amazon CloudWatch event which will be triggered when the state of the instance is changed from running to stopped. AWS Lambda will then be triggered and will perform an EBS volume snapshot while the instance is down. Upon successful completion, it will start an EC2 instance.

Generating Reports

You can use AWS Lambda to generate various reports about your infrastructure. For example, you can review all EC2 instances to check for those that don’t contain tags and review all EBS volumes to check for those that aren’t in use. You can also scan all Security Groups and report if a group has a sensitive port open to public networks.

Running AWS Lambda Code When an Object Is Put Into a Specific Bucket

AWS Lambda integrates with Amazon S3. Let’s say you want to create an AWS CodeDeploy deployment whenever an object is put into a bucket under a certain prefix. Based on event data, you can get the full path of the object that is being uploaded, then trigger AWS CodeDeploy deployment, which will deliver that code to your EC2 instances.

Batch Log Processing

You can also use AWS Lambda to transform data gathered from various sources. Amazon Kinesis is a service used for real-time streaming data to Amazon S3, Amazon RedShift, or Elasticsearch. With AWS Lambda, you can normalize data, add metadata, perform ETL functionalities, and combine AWS Lambda with data from another source.

Integrating With Amazon API Gateway to Invoke AWS Lambda

AWS Lambda can be integrated with Amazon API Gateway. You can build an application that will invoke AWS Lambda and get certain information about resources in your AWS infrastructure. For example, based on the instance ID that is passed to API Gateway, you can fetch EC2 instance tags or other information that cannot be obtained from the EC2 link-local address.

Failover Between Two EC2 Instances

Using AWS Lambda, you can set up automatic failover between two EC2 instances. Let’s say you use Elastic IP on one instance, and that instance goes down. In that case, Elastic IP will be detached from the instance, which will cause downtime for your service. Using Amazon CloudWatch and AWS Lambda, you can create an event-based rule. This rule will trigger AWS Lambda when the instance state is changed from running to stopped. AWS Lambda will get the Elastic IP ID, and attach it to the standby instance.

Scheduled Tasks

Cronjobs are also available with AWS Lambda. Using Amazon CloudWatch Events, you can first create an event which will be run periodically. CloudWatch’s scheduled events have a cron-like syntax and can be used to perform scheduled operations. An example of cronjob Lambda usage is running a DynamoDB update using data stored in Amazon S3. In this case, you can create a CloudWatch event that is scheduled to run once per day. Your AWS Lambda can launch an EC2 instance that will fetch data from Amazon S3. Then, it will execute code that will use that data to update databases. Once the job is completed, Amazon EC2 should be terminated upon powering off.

Slack Bots

Slack bots are a great way to automate tasks and increase productivity. They can be used in the DevOps world for reporting about task results, and also actively scheduling them using interactive bots. A good example is the Serverless Slack Bot guide which will get you started with Slack Bots in AWS Lambda.

Conclusion

In this article, we covered some uses for AWS Lambda in the DevOps world. However, not every task is suitable for AWS Lambda, since it has several limitations, such as the maximum memory and running time. For more complex tasks, our guide to AWS Step Functions provides a glimpse of the possibilities in using more advanced serverless services to implement complex state machines. Be sure to check the things to consider when getting started with serverless, and our first steps guide to serverless.

Finally, using open-source tools can greatly help in completing various tasks. In Epsagon, we released various tools that can help:

Be sure to make use of them and the other tools out there! We’ll also publish an article soon about deploying your AWS Lambda for DevOps purposes using command-line tools.

Finally, if you’re worried about permissions and secret management with AWS Lambda, be sure to check this overview.