In the modern web, an API is a crucial and inherent part of any application. Each time you open a news website, gaming site, or online banking page, or whenever you use any other online service, you make one or more API calls to a server. And yet, this is something you most likely never think about. 

In this article, we’ll discuss what an API is (or simply remind those who already know) and explain the role of third-party APIs in building your own service. We will then build a small application and get to know how to effectively troubleshoot such a distributed system containing APIs.

Application Program Interface

An Application Program Interface (API) allows a service to respond to particular requests. A real-world example could be a stewardess in an airplane: You press a button (establish a connection), ask the stewardess to bring you a drink (make a request), and then you either get your drink (expected response) or get a refusal, for example, if they don’t have what you requested onboard (kind of error). 

Note that you don’t actually care where your drink may be stored, how to get into the kitchen, who pours it into the glass, etc. You only need to know that you can make such a request, how to do it, and what to expect as a result.

The same principle is applied in web development: Front-end applications loaded in your browser know how to display content and how to request it from a particular API endpoint; in turn, the server knows what to do to accomplish this request and form a proper response. This communication is primarily executed through the HTTP protocol.

HTTP APIs

HTTP is currently the most common way for a client to communicate with a server. 

HTTP requests consist of:

  • Method: Such as GET, PUT, POST, etc.
  • Target: The URL address to the remote server
  • Headers: Responsible for carrying specific data about a particular request, but not the actual request data
  • Body: An optional part used to keep the actual request data to be transmitted to the server if necessary 

HTTP responses components are:

  • Status code: The main indicator of how successful a request was
  • Headers: For info related to the response but not the response data itself
  • Body: The response data, also optional

The status code of the response can be very descriptive—it’s not just a numeric combination. Since codes are globally standardized, it’s easy to determine not only if the request was successful, but also a general reason in the event of failure. Status 200 stands for “OK” and means a totally successful response. However, there are a lot of 4xx status codes, which can be received when there is an error presumably caused by a user, for example:

  • 400 – “Bad Request,” means that inputs in the request are invalid or missing.
  • 403 – “Forbidden” usually indicates that the current user does not have the permissions necessary for performing the request.
  • 404 – “Not Found” means that what was requested could not be found. 

There are also 3xx codes indicating that resources have been moved or that redirects should be applied. Finally, 5xx codes are for various server errors, for example, the 500 “Internal server error” or 503 “Service unavailable.”

Using External APIs While Developing APIs

A very common and useful practice is to use third-party APIs while developing your own APIs. Imagine that you need to build an application where users can register an account, pay for a subscription, receive an email notification, and then be able to use the features provided by your service. Developing and maintaining each step could take ages; so, developers will very often opt to use APIs provided by third parties. For example, you can use Auth0 for user management, Stripe for payments, SendGrid for email subscriptions, etc. 

external APIs provided by third parties

Figure 1: Variety of external services across many industries (Source: Bessemer Venture Partners)

According to a prediction from Bessemer Venture Partners, almost every industry in the future will be reinvented by APIs. And yet, it seems that this future reality is already here, as 90% of features needed by any application can be implemented via external providers.

Of course, the above image does not give a full list of services you may use in your next project; there are countless different APIs, many of which can be used totally or partially free of charge.

Preventing the Risks

Sure, the more third-party APIs you use, the more you depend on their stability, their uptime, how often they implement breaking changes, etc. However, this issue is not very relevant for mammoths like Stripe or Auth0, which boast 99.99% uptime

Still, no matter how reliable your provider is, you should keep in mind alternative services and implement abstraction layers responsible for that particular feature in front of a third-party API. This could be a life-saver in critical situations, helping you to quickly switch to another provider.

Detailed monitoring of your application is also a must to quickly identify an issue, determine which exact API caused the error, and troubleshoot effectively. Fortunately, this can be achieved with Epsagon.

Example “Daily Digest” Application

utilizing third-parties diagram

Figure 2: API utilizing third-parties under the hood

Let’s say you build a backend for an application that gives users a short daily digest of random historical facts about the current date, plus the weather and top news for whatever city was passed as a parameter. Such a service can be implemented using three third-party APIs: OpenWeatherMap for the weather, News API for news, and Numbers API for retrieving historical facts.

A lot of API providers allow you to use their API for free, with some limitations; however, they still require you to obtain a token that needs to be passed with each request. To get this token, you usually just have to complete a simple registration. So, for the purpose of the following tutorial, we’ll assume you already have all the tokens required.

Coding the Application

All the code for this application will be written in Node.js, so in order to proceed, you need to download and install a stable Node.js version. Then, you can create a folder for your project, navigate to this folder via your terminal, and run:

$ npm init


Now, you need to install a few basic modules with another command:

$ npm install express got


Express is one of the most popular frameworks for writing web services with Node.js, and the got npm module is great for performing external HTTP requests.

When everything is installed, go ahead and create an app.js file and input some very basic code that will import all needed modules and declare some constants:

const express = require('express');
const got = require('got');

const TOKEN_WEATHER = 'your-open-weather-api-token-here';
const TOKEN_NEWS = 'your-news-api-token-here';

const WEATHER_API_URL = 'https://api.openweathermap.org/data/2.5/weather';
const NEWS_API_URL = 'https://newsapi.org/v2/top-headlines';
const FACT_API_URL = 'http://numbersapi.com';

Now, right below this, declare the methods responsible for fetching data from the external services:

function getWeather(city = '') {
    const searchParams = { q: city, appid: TOKEN_WEATHER };
    return got(WEATHER_API_URL, { searchParams }).json().catch((err) => null); // json
}

function getNews(city = '') {
    const searchParams = { q: city, apiKey: TOKEN_NEWS };
    return got(NEWS_API_URL, { searchParams }).json().catch((err) => null);; // json
}

function getFactForToday() {
    const d = new Date();
    const month = d.getMonth() + 1;
    const day = d.getDate();
    const URL = `${FACT_API_URL}/${month}/${day}/date`;
    return got(URL).text().catch((err) => null); // string
}

The last part of the code will create an express server, describe its routes, and start it:

const app = express();

app.get('/digest/:city', async (req, res) => {
    const city = req.params.city;
    const [weather, news, fact] = await Promise.all([
        getWeather(city),
        getNews(city),
        getFactForToday()
    ]);
    res.json({ weather, news, fact });
});

app.use('*', (req, res) => {
    res.status(404).send('Not Found');
})

app.listen(3000, () => console.log('App is now online at port 3000'));

In the above code, you set up an API endpoint, expecting “city” as a route parameter. So the final URL for the digest API would be, for example, http://localhost:3000/digest/london.

Testing the App Locally

Open your terminal in the projects folder, and run:

$ node app


Then open your browser and navigate to http://localhost:3000/digest/london; there, you should see the data aggregated from third-party services in one request:

data aggregated from third-party services

Figure 3: Application’s response

Deploying Your Application to AWS

In the above example, you used the Express framework, which is great for standalone monolith apps that can be deployed to EC2 instances, EBS, or any other analog. 

Here, you’ll be deploying your application to AWS Lambda, which is easy to use and very cost-effective, as you never pay for when the service is idle. Fortunately, deploying Express apps to AWS Lambda can be done smoothly with Serverless Framework and a serverless-HTTP plugin. You’ll need an AWS account with AWS credentials keys, as well as an Epsagon account to obtain a token for tracing your application from the settings page.

First, install the Serverless Framework on your machine:

$ npm install serverless -g


Then install all the modules needed for further deployment and Epsagon integration. In the projects folder run:

$ npm install epsagon serverless-plugin-epsagon serverless-http


Now, create an empty serverless.yml file and fill it with the following content:

service: daily-digest
provider:
  name: aws
  runtime: nodejs12.x
  stage: dev
  region: eu-west-1
  memorySize: 1024
functions:
  app:
    handler: ./app.handler
    events:
      - http:
          path: /
          method: ANY
          cors: true
      - http:
          path: /{proxy+}
          method: ANY
          cors: true
plugins:
  - serverless-plugin-epsagon
custom:
  epsagon:
    token: your-epsagon-token-here
    appName: app-daily-digest
    metadataOnly: false



Take a look at the custom field above. It’s important to put your Epsagon token there, as you will not be able to trace your application without it.

Before deploying, you also need to make a few changes in the application’s code. At the top of app.js add these two lines:

const slshttp = require('serverless-http');
const epsagon = require('epsagon');

Also, let’s add some custom tags to your service to make future traces a bit more detailed. In the existing request handler, input:

app.get('/digest/:city', async (req, res) => {
    const city = req.params.city;
    epsagon.label('city', city); // <- this one
    const [weather, news, fact] = await Promise.all([
        getWeather(city),
        getNews(city),
        getFactForToday()
    ]);
    res.json({ weather, news, fact });
});

The final change is to replace the last line with a serverless-http wrapper:

// app.listen(3000, () => console.log('App is now online at port 3000'));
module.exports.handler = slshttp(app);

Now, run the following command, which can take a minute or two to complete:

$ serverless deploy

In the end, you should see the output with URLs to your service endpoint:

application deploy to aws

Figure 4: Application successfully deployed

Congratulations! Your app is now deployed and ready to use.

Use the Service and See the Traces

Just like with local testing, to test your app, insert the endpoint URL into your browser—don’t forget to add “digest/some-city” to the URL. Now you can try it out with different cities. Also, make a few requests with invalid city names to generate traces with errors too.

When you’ve made a few executions of your service, navigate to your Epsagon console and select the “Service Map” tab on the left-side menu:

epsagon service map for API errors

Figure 5: Map of the application

Here, you can see the overall structure of your application. You can also see that the weather API returns an error sometimes; that’s because you probably entered invalid cities.

Now go to the traces panel where you can see a list of all the traces:

epsagon traces for API errors

Figure 6: List of traces captured by Epsagon

You can even select any single trace and view detailed information about the request:

detailed API error request

Figure 7: Information about a particular request

Custom Dashboard

There are some default widgets on the Epsagon dashboard as well, including information about top-used resources, which are useful in determining the stability of external services, the fastest or slowest APIs, and how often an API causes an error.

epsagon dashboard for top used sources and APIs

Figure 8: One of the default widgets on Epsagon’s dashboard

You can even create a custom dashboard with specific visualizations per your given needs. For example, you may want to see which cities are being requested the most and the average time it takes to compile a digest for each city. Open the trace search panel, choose the tags you would like to aggregate, and check the newly created chart.

epsagon custom dashboard

Figure 9: A custom set of parameters to display on the chart

Plus, if you come across a chart that would be useful to see each time you open the Epsagon app, you can easily add it to your dashboard.

You can add multiple various widgets to your custom dashboard, rename them, and shuffle their order. For example, it would make sense to put widgets with important information, such as error count, execution duration, and throughput, on the dashboard to have an overall picture of your entire system.

Figure 10: Custom dashboard in action

What’s Next

The usage of third-party services inside your application is normal practice, allowing developers to focus on the main features and thus save both time and costs. External services can bring some potential risks and create issues from time to time, but you can minimize these losses by detecting problems in time. No matter how distributed your system is, Epsagon can help you find issues quickly and react swiftly and effectively.

When implementing tracing and monitoring, as discussed in this post, it’s worth noting that the principles described can be applied to various applications hosted on different platforms, written in different languages, and implementing different services. 

Remember, monitoring an application has always been a fundamental part of an app’s reliability and stability, so you should definitely consider a monitoring tool in your existing or future projects.

Read More:

How to Effectively Monitor Amazon ECS

How to Effectively Monitor AWS Lambda

How to Scale Prometheus for Kubernetes