Software development based on microservices like AWS Lambda results in highly distributed applications and services that are easier to manage. Microservices are smaller apps designed for a specific task or business process. They are abstracted from infrastructure and widely distributed across a production environment.
But…this type of development also requires a modern, complex architecture: a suite of stand-alone components including serverless functions (Function as a Service or FaaS), containers, and managed services. These components each run in their own processes and work together in a well-defined interface across various communication channels, including both synchronous and asynchronous communications.
AWS Lambda – A FaaS Service
AWS Lambda is a FaaS offering from Amazon—basically, a compute service that lets you run code, without provisioning or managing servers, on high-availability compute infrastructure. AWS Lambda performs all of the administration of the compute resources, including server and operating system maintenance, capacity provisioning and automatic scaling, code monitoring, and logging.
Yet, just like on-premises monolithic applications, engineering and DevOps working with microservices such as AWS Lambda or Kubernetes must know whether the serverless or container-based application is working properly, how they can enable greater developer velocity, and what’s currently running in production, so they can more easily and rapidly fix issues and build new cloud services.
Monitoring AWS Lambda and the AWS Console
AWS has done a great job giving you the ability to monitor, debug, and test your Lambda functions through the AWS console. It also gives you several options for configuring how your function will be triggered. The benefit of this is that you can get an instant feedback loop while debugging so as to better understand your functions.
Each function on AWS Lambda gets monitored automatically. By default, you will be able to see the number of invocations, the average duration of the function, and error rates, among other data. Also, AWS Lambda is integrated with AWS CloudWatch, so you can check the logs directly from the same section.
Checking logs with AWS CloudWatch and tracing with AWS X-Ray
AWS Lambda pushes all logs, by default, to AWS CloudWatch. Here, every Lambda function creates something called a Log Group where you can see all the logs, drill down to see all the log streams, and drill further to see log events.
This feedback loop when you’re debugging your Lambda functions helps you to troubleshoot when something breaks, because the logs appear almost in real time. CloudWatch isn’t well suited to finding the root cause of an issue rapidly and increasing MTTR.
AWS X-Ray will visualize the components of your application, identify performance bottlenecks, and troubleshoot requests that resulted in an error. Your Lambda functions send trace data to X-Ray, and X-Ray processes the data to generate a service map and searchable trace summaries. X-Ray can be painlessly integrated with a built-in button, and traces are stored for 30 days by default.
In terms of monitoring and alerting, X-Ray shows only calls, latency, and errors per resource, without the ability to group by any dimension (call type, source element). X-Ray also doesn’t provide any infrastructure information or metrics (CPU, memory, network). It is based only on traces. And lastly, it provides no alerting.
In the area of troubleshooting, X-Ray comes up a little short. X-Ray doesn’t trace across distributed async events (message queues, HTTPs, etc.) or correlate payloads. X-Ray also doesn’t show corresponding logs based on a trace.
Making sense of it all
Microservice-based apps and services require end-to-end-observability based on the three pillars of observability (metrics, logs, and traces), plus payloads. New application performance monitoring and troubleshooting approaches must address a number of key requirements and enable developers with observability in the following ways:
- Develop and fix rapidly. New solutions must keep pace with the need for very rapid, iterative, and automated development, testing, staging, and deployment cycles. Meeting this requirement demands very high levels of monitoring and troubleshooting automation, including the ability to auto-discover the many components that make up the application architecture and the dependencies between them, as well as auto-instrumenting and auto-deploying.
- Automatically monitor asynchronous communications typical of microservices and automatically map and trace these distributed requests in real-time. Unfortunately, it’s hard to anticipate the exact timing and sequencing of these requests as they go across the many components in a transaction. Latency or availability issues anywhere along the request route can significantly impact application performance and the end-user experience. Connecting and automatically correlating metrics, alerts, logs, traces, and payloads in a highly-visual architecture view or service map facilitates rapid issue identification and MTTR.
- Provide visibility with light agents, relying on protocols such as SNMP, WMI, or SSH to relay infrastructure data to the central monitoring system. Modern microservice applications are abstracted from the infrastructure. Thus, developers cannot deploy heavy agents that rely on bytecode instrumentation for application monitoring. New monitoring and troubleshooting approaches must use a light agent or agentless solution for application monitoring to achieve rapid setup and instant tracing.
- Handle growing volumes of diverse data to enable decision-making in real-time. In the modern application, incoming log data from many distributed components can grow exponentially in volume and complexity and overwhelm developers with tedious, manual tasks. Observability solutions must be able to scale to hundreds of trace views and provide payload visibility for operational and business insights, as well as out-of-the-box, developer-oriented metrics representing best practices.
- Easily integrate with a multitude of AWS services, and other management services, programming languages, and API-level integrations so that applications and services can run across any production and any workload: AWS ECS, AWS Lambda, Kubernetes, AWS Fargate, or just a VM. You will want to work in the languages you work in: NodeJS, Python, Go, Java, or .NET and receive alerts where you work: Slack, PagerDuty, Microsoft Teams, and more.
Modern applications built with microservices like AWS Lambda and Cloud managed services require cloud-native monitoring and troubleshooting that is highly automated and provides a holistic, intuitive view of all the “moving parts” as well as the real-time interactions among them, so developers can see and quickly drill down to the issue and fix it in seconds and minutes – not days and weeks.
As a cloud-native platform, Epsagon enables DevOps teams to easily monitor modern applications, troubleshoot less, and fix issues faster by providing automated data correlation, rich payload data, and end-to-end observability within microservice environments. With a lightweight agent SDK, Epsagon provides automated instrumentation and tracing without gaps in coverage, giving you full visibility for containers, VMs, serverless, and more with no training, manual coding, tagging, or maintenance required. Customers report a 95% reduction in troubleshooting time and 75% reduction in errors with Epsagon.
Epsagon’s end-to-end observability of AWS Lambda functions, serverless, and containers can help you de-bug and de-risk your development and production environments, as well as increase developer velocity and speed your time to business value with flawless applications and services.
Epsagon is an AWS Advanced Technology Partner with DevOps, Data and Analytics, and Retail competencies and is sold directly and in the AWS Marketplace as a subscription-based SaaS service. Epsagon is also an AWS Lambda Layers Launch and Public Sector Partner. Epsagon enhances product integration with a multitude of AWS services (Lambda, ECS and Fargate, DynamoDB, API Gateway, ElasticCache, ES, RDS, and more), along with other management services, programming languages, and API-level integrations. Epsagon is the first provider of automated tracing for AWS AppSync–with no code changes required. The solution also automatically connects AWS CloudWatch metrics and AWS metadata with other sources.
Start your Epsagon free trial and see valuable insights immediately.