Companies today are breaking down their big monolithic applications into multiple microservices and one of the challenges of having to deal with multiple microservices is that it’s difficult to isolate issues such as:

  • Error rates
  • High Latencies
  • Error codes across the application

With microservices, there is also no dynamic way to reroute network traffic when failures occur or new containers need to be deployed. One of the workarounds for these problems is to add your code into each microservice or use an alternative open-source tool that can handle communication for each microservice.

Unfortunately, such workarounds present new challenges, as these solutions are difficult to manage, install, and update across different teams. A service mesh, however, can help you solve many of these challenges.

How Does a Service Mesh Work?

A Service Mesh routes a request to different microservices through proxies using a dedicated infrastructure layer built right into the app; this controls how different parts of an application share data with each other. Sometimes we call these proxies sidecars. These sidecars work in conjunction with microservices to route requests to other proxies.

Combining these sidecars proxies with microservices forms a mesh network. But what exactly does a Service Mesh solve?

Without a Service Mesh, you would need to add custom code with logic to manage the communication of each microservice. Debugging failures can also be difficult to diagnose because these microservices communicating with each other are not exposed within each service.

One of the primary reasons why Service Mesh is so popular is the addition of a sidecar proxy, which handles all the network traffic into and out of the microservice. It also provides visibility and traffic control for all of your microservices.

Figure 1: Service Mesh with sidecar proxy.

Open-Source Offerings

There are a myriad of open-source service mesh solutions available in the market. Let’s explore a few of them:

  • Istio: Initially launched by Lyft as a Kubernetes-native solution, Istio later became the service mesh of choice for many big companies. It supports HTTP, HTTP/2, WebSockets, gRPC, and all TCP traffic. Google Cloud provides Istio as a service, pairing it with Envoy as default and using it as a sidecar proxy.
  • Envoy: An open-source proxy written in C++11 with the codebase of only 8MB. It supports level-7 protocols like HTTP, HTTP/2, and gRPC and requires no code change at the application end. AWS AppMesh is based on Envoy.
  • Linkerd: Designed with simplicity in mind rather than flexibility, Linkerd only supports Kubernetes, rather than other open-source solutions (Envoy and Istio) that support Kubernetes and a virtual machine. Linkerd is now a part of the Cloud Native Computing Foundation (CNCF).
  • Consul: Consul is part of various HashiCorp offerings and provides capabilities like service discovery, making it a full-service mesh. It supports Kubernetes and other container management solutions.

AWS App Mesh

AWS App Mesh is a service mesh solution offered by Amazon Web Services that provides functionality to monitor microservice applications on AWS. First launched in March 2019 (beta and product review phase), it is now available and supported for production use.

The Different Components of App Mesh

  • Service Mesh is a namespace that groups the microservices you want to communicate; this serves as a logical boundary for network traffic between the services that reside within it.
  • Virtual Nodes represent services in the mesh and can be an ECS service, Kubernetes deployment, or service running on one or more EC2 instances.
  • Virtual Routers are a logical grouping for all of your routes. Each virtual router listens to HTTP traffic on a specific port, while routes are created to direct traffic appropriately once the virtual router is created.

How Does App Mesh Work?

First, you need to run the Envoy proxy (sidecar) along with each service. App Mesh will take care of configuring the proxy to handle all communications into and out of each container.

Figure 3: After Service Mesh

App Mesh Integration with Various AWS Services

The different AWS Services that support App Mesh are:

Like other AWS Services, you can access App Mesh via command line, AWS Console, and SDK.

App Mesh Integration with CloudWatch and X-Ray

App Mesh is already integrated with various AWS Services and third-party tools that enable you to collect and then export data to the tools of your choice. Using these services (below), you can perform logging, monitoring, and tracing of your microservices.

App Mesh Working Example

In this example, you’ll be setting up the different App Mesh components (Service Mesh, Virtual Nodes, Virtual Routers, Virtual Services, and Routes), deploy a simple microservice to ECS (as shown in Figure 4) using this code repo, and then modifying the App Mesh routes to demonstrate a canary deployment. AWS already automates the entire process using CloudFormation and Shell Script.

Figure 4: App Mesh using Colo App


NOTEAll steps below and their output were executed for this example using Ubuntu 18.04.4

Getting App Mesh Up and Running

Follow the steps below to get App Mesh started, with a microservice deployed.

Step 1: Clone the Repo

git clone
cd aws-app-mesh-examples/

Step 2: Setting up the Environment Variables for Deployment Scripts

export AWS_PROFILE=default
export AWS_ACCOUNT_ID=your-account-ID
export AWS_DEFAULT_REGION=us-west-2
export ENVIRONMENT_NAME=Development
export MESH_NAME=my-first-appmesh
export SERVICES_DOMAIN=colorteller.myapp.local
export KEY_PAIR_NAME=aws-ecs


  • AWS_PROFILE: This sets up your AWS Profile; set to default or named profile.
  • AWS_ACCOUNT_ID: To find out the account ID, run the following command:
aws sts get-caller-identity --query Account
  • AWS_DEFAULT_REGION: This is the region where you want to set up AppMesh (supported App Mesh regions). 
  • ENVIRONMENT_NAME: This serves as a prefix to the name of the deployed CloudFormation stack.
  • MESH_NAME: This refers to your chosen name for the mesh you created. (For this example, we are using my-first-appmesh.)
  • SERVICES_DOMAIN: Here, you have the base name for service discovery. (This example uses myapp.local, so the gateway virtual service will send requests to the colorteller virtual service at colorteller.myapp.local.)
  • KEY_PAIR_NAME: This is the AWS EC2 key pair to log in to the instance.
  • ENVOY_IMAGE: Check this link for the latest envoy images.

Step 3: Infrastructure & Deployment

Execute the steps below to create the infrastructure and deploy your resources.

A: Create the VPC

Create a VPC (think of it as your data center in the cloud that provides isolation from other applications). The script below uses the CloudFormation template examples/infrastructure/vpc.yaml under the hood to create a VPC for your mesh application:


B: Create an App Mesh

To create an app mesh, run the following script:


This will use the CloudFormation template examples/infrastructure/appmesh-mesh.yaml. 

C: Create compute resource

In the next step, you need the infrastructure to run your services. We are going to use the shell script below, which under the hood uses examples/infrastructure/ecs-cluster.yaml CloudFormation script to create the ECS cluster.


D: Configure App Mesh resources

As the infrastructure deployment part is complete, the next step is to configure your App Mesh resources (Virtual Node, Virtual Router, Virtual Service, Route).

For this, you need to use the script below; under the hood, this will utilize the examples/apps/colorapp/servicemesh/appmesh-colorapp.yaml CloudFormation yaml.


E: Build/push images to ECR repository of your account

Here, you will build the image using the source examples/apps/colorteller/src and then push them to your account’s ECR registry using a deploy script.

First, deploy the gateway image by going to the rooof colorapp repo (in this case, /home/ubuntu/aws_app_mesh/aws-app-mesh-examples), and execute the following commands:

cd examples/apps/colorapp/src/gateway
aws ecr create-repository --repository-name=gateway
export COLOR_GATEWAY_IMAGE=$(aws ecr describe-repositories --repository-names=gateway --query 'repositories[0].repositoryUri' --output text)

NOTE: In case you are using awscli v2 and the script fails due to this error:

++ aws ecr get-login --no-include-email
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]

 Execute the below command by replacing XXXX with your account ID:

aws ecr get-login-password | docker login --username AWS --password-stdin

This command retrieves and prints a password that can be used to authenticate any Amazon ECR registry.

Now, you can push the color gateway image again:


The push refers to repository []

d926a2b33a1a: Pushed, where XXXXXXXXX is your AWS account ID.

Time to deploy the colorteller image. Go to the root of colorteller repo (here, it’s /home/ubuntu/aws_app_mesh/aws-app-mesh-examples), and execute the commands below:

cd examples/apps/colorapp/src/colorteller
aws ecr create-repository --repository-name=colorteller
export COLOR_TELLER_IMAGE=$(aws ecr describe-repositories --repository-names=colorteller --query 'repositories[0].repositoryUri' --output text)

F: Deploy gateway and colorteller services

As the final step, you will now deploy your services on ECS using the script examples/apps/colorapp/ecs/ Under the hood, this will use the examples/apps/colorapp/ecs/ecs-colorapp.yaml cloudformation template.

Before executing this script, export the following environment variables:

  • COLOR_GATEWAY_IMAGE : Docker image for ColorApp gateway that you pushed to the ECR repo.
  • COLOR_TELLER_IMAGE: Docker Image for ColorApp colorteller that you pushed to the ECR repo.
export COLOR_GATEWAY_IMAGE=$(aws ecr describe-repositories --repository-names=gateway --query 'repositories[0].repositoryUri' --output text)

Step 4: Test the application

Once you deploy the app, you can curl for the gateway frontend service. To find the endpoint:

colorapp=$(aws cloudformation describe-stacks --stack-name=$ENVIRONMENT_NAME-ecs-colorapp --query="Stacks[0
> ].Outputs[?OutputKey=='ColorAppEndpoint'].OutputValue" --output=text); echo $colorapp

To verify it:

curl -w "\n" $colorapp/color
{"color":"white", "stats": {"blue":0.33,"red":0.33,"white":0.33}}

As you can see, traffic is distributing equally between blue, red, and white colorteller virtual nodes through the default router.

Canary Deployment

To demonstrate a canary deployment, you will need to modify the App Mesh routes. This can be done from the command line or from the AWS Console. Let’s try to perform this activity using AWS Console.

Go to App Mesh and click on the mesh you created (my-first-appmesh):

Figure 5: App Mesh Console

Click on “Virtual routers” and “Router name”:

Figure 6: App Mesh Virtual Router

Select the “Route” and then “Edit”:

Figure 7: App Mesh Route

You will see something like this:

Figure 8: App Mesh Targets

Change the “Weight” of “colorteller-white-vn” to 0 and click on the “Save” button:             

Figure 9: App Mesh Targets

As you can see, the rest of the weights adjusted themselves to 50-50.

Now, if you test your app, you will see that your traffic is only routed between blue and red nodes.

curl -w "\n" $colorapp/color
{"color":"blue", "stats": {"blue":0.5,"red":0.5,"white":0}}

Troubleshooting Issues


Using AWS X-Ray, you can monitor and analyze the distributed application by obtaining:

  • A complete view of how the request is traveling from end to end, which helps you determine the root cause of performance and errors.
  • A visual map to see the distribution of traffic by App Mesh and the ability to use that map to find latencies in your routes.

To access X-Ray go to the AWS Console and search for X-Ray.

The ColorApp that you deployed already has X-Ray support. Now, if you go to the X-Ray console, you will see the segment name “default.”

This is how your request data will be displayed:

Figure 10: AWS X-Ray

Click on my-first-appmesh/color gateway-vn node to display service details; you will see an overview of any latency and that 100% of the requests are OK.

Figure 11: AWS X-Ray Service Details

Click on the “View traces button and then on an ID under the trace list; this provides a detailed view on how traffic flowed for the requests.

Figure 12: AWS X-Ray View traces

App Mesh Use Cases

Using App Mesh for a canary development, you can deploy the new version of the app and then selectively route traffic between the two hosts. Once you are confident that the new version is working as expected, you can gradually increase the traffic on the new version.

For a blue/green deployment, App Mesh Provides the capability through weight, which you can use to switch between the different versions of the app without causing any downtime. As you can see in the canary deployment example, a simple change of weight at the router level can divert the traffic between various hosts.

Wrapping Up

If you are planning to move to microservices from your monolithic application or are already building/using microservices to meet current demand, you’ll need features like the ability to scale rapidly, detect failures quickly, and find error rates within your application fast.

With your microservice architecture’s increasing complexity, your developers are wasting more time coding request logic instead of spending their time writing business logic.

Using AWS App Mesh provides the following benefits:

  • Developers can spend more time writing business logic instead of focusing on how microservices connect with each other.
  • App Mesh can handle failures and automatically reroute requests away from failed nodes.
  • App Mesh integration with X-Ray gives the holistic view of your application, enabling features like debugging and identifying the root cause of performance issues.
  • Integration with CloudWatch and other third-party tools like Epsagon, which enables monitoring of your microservices.