NearSt is a cloud-based, retail technology company that links Main Street shops and their available products to customers in real time through the power of Google Search. No more calling to see if your local shop or a local shop within a chain has what you want. With NearSt, you know the product is there waiting for you, and the retailer knows he or she can sell it to you.
From Light Bulb to Online Retail Platform
Most people want vibrant communities and services near where they live. Who wouldn’t prefer doing a quick errand locally or strolling down a street with shops whose inventory has been curated just for you?
Better yet, that inventory is not only curated for you, but, before you leave your house, you already know the product is in stock at that specific location.
Main Street, or the High Street as it is called in the UK, is not dead. It was just waiting for NearSt. And it all began five years ago with a light bulb. In 2015, one of the NearSt founders was having a dinner party and, just before guests arrived, he unexpectedly needed to buy and replace a light bulb. Which of his local shops would be open and have the light bulb after five o’clock?
“Which of his local shops would be open and have the light bulb after five o’clock?”
That need resulted in a solution: a powerful business-to-business platform that seamlessly integrates with all kinds of old and new retail point of sales systems. NearSt can automatically extract real-time, local retail inventory data and feed it to search partners like Google, so people can see the local shop has the product in stock. The result: you can build your local business or a franchise, get more people into your shops, and develop a loyal customer base with real-time local retail services. And the customer has the product he or she wants in real time.
NearSt today has hundreds of UK High Street shops and businesses and is poised to expand globally. Behind the purchase of that light bulb or other products, however, is a platform that must deliver real-time, scalable and reliable apps with a user interface that is easy-to-learn and can change rapidly to accommodate new product SKUs and advanced ad campaigns.
By mid-2018, visibility within the NearSt environment had become a serious issue.
According to Adam Hogge, lead developer at NearSt, the company started out with only containers, which was innovative at the time.
It’s sometimes hard, however, to see inside containers to detect issues, Adam explained. In his environment, there is often a single host running several containers. “Looking at container metrics at the host level, it’s not always obvious what’s consuming all the resources, why timeouts happen, or where memory leaks are, for example.”
“Looking at container metrics at the host level, it’s not always obvious what’s consuming all the resources, why timeouts happen, or where memory leaks are, for example.”
NearSt had added more and more AWS containers, and “system complexity had outstripped the ability to monitor it effectively. The complexity had begun to drag on developer velocity. “
“System complexity had outstripped the ability to monitor it effectively. The complexity had begun to drag on developer velocity. “
The complexity was multifaceted. NearSt had built its software as it built out the company, which Adam noted, did not result in an effective domain model. They were adding more containers and microservices “on the fly” without considering the architecture enough in advance, and that led to unnecessary complexity. “As a result, developers had to iterate rapidly.”
As lead developer, Adam decided he had to improve developer visibility. NearSt was dealing with hundreds of millions of lines of inventory daily and high volumes of data. Today, NearSt is processing billions of lines of inventory per month. There is no option to turn it off for an hour or put a console log in there; any slowdown resulted in backups with other side effects.
Complexity Affects Delivery of New Features and Advertising Services
By 2018, NearSt was not only having trouble monitoring the evolving environment, but also delivering new features, managing online advertising dynamically, and rapidly responding to customer feedback. For example, a chain of 20 stores wanted to dynamically allocate ad spending hour-by-hour over a week.
“By 2018, NearSt was not only having trouble monitoring the evolving environment, but also delivering new features, managing online advertising dynamically, and rapidly responding to customer feedback.
The building blocks for the modern app included creating public-facing login consoles, authentication, authorization, multitenancy, payment processes, credit card management, compliance, integrations with Google (and all of their APIs), as well as the ability to track everything. Customers also required the ability to dynamically adjust ad campaigns in real-time, based on performance and availability.
The challenges then were:
- The real-time, changing nature of the apps
- Time-to-market for new flawless features
- Elasticity for development
- Scalability due to huge waves of data of variable size: often 100,000x fluctuations peak-to-trough
- Cost containment
Receiving retailer requests like this clearly validated Near St’s business model, but also posed a challenge. To build, Adam explained, he really needed to “know what the system was doing end to end. I needed to break it down into domains—shops, products, and inventory and build new features in an elastic, scalable, and reliable way with serverless, which is ideal for this kind of challenge. I also needed to work within my environment and monitor containers as well as serverless. I needed to simplify and contain costs.”
Epsagon Monitoring and Visibility – Containers and Serverless
Today, NearSt uses Epsagon in its current environment with its mix of AWS ECS, Fargate and DynamoDB to allow for modern, decoupled microservices. Epsagon helps with container management and serverless development, enabling new features while helping with cost containment.
With a previous monitoring solution, Adam could not monitor containers and serverless well. Locating CloudWatch logs was impossible, and the third-party solution did not provide a useful dashboard to tell what AWS services were working. Teasing things like network request details out of a failed request were impossible.
NearSt decided to use Epsagon since it provided automated, distributed tracing for both containers and serverless, thus enabling monitoring and troubleshooting with payload visibility. Epsagon is also compatible with CloudWatch.
“NearSt decided to use Epsagon since it provided automated, distributed tracing for both containers and serverless, thus enabling monitoring and troubleshooting with payload visibility.”
Epsagon’s ability to auto-instrument and auto-discover NearSt’s compute stack also helped increase developer velocity. Visualization of the entire environment, seeing everything in production, and automated tracing contributed to rapid troubleshooting and fixing issues fast. With issues fixed rapidly, application reliability increased and end user vendor experience was enhanced. With its troubleshooting and cost-monitoring capabilities, Epsagon helped NearSt save costs. Its AWS service bill today is a quarter of what it was a year ago, Adam said.
“Epsagon helped NearSt save costs. Its AWS service bill today is a quarter of what it was a year ago.”
Using Epsagon, Adam and his team could monitor all 10 ECS clusters, which helped them to be alerted and identify utilization issues quickly. Being able to drill into every service and task, looking at events and logs, and comparing traces, all under the same dashboard helped the team troubleshoot and pinpoint exceptions efficiently, Adam noted.
“Being able to drill into every service and task, looking at events and logs, and comparing traces, all under the same dashboard helped the team troubleshoot and pinpoint exceptions efficiently.”
Every member of the six-person developer team uses Epsagon to work on the code base.
Better container observability and logs in a single view
NearSt developers use Epsagon to track container traffic, where the container is being used and how the container is being used. With containers, NearSt developers were able to get to the logs without digging since Epsagon provides correlation of traces and logs in a single interface.
“NearSt developers were able to get to the logs without digging since Epsagon provides correlation of traces and logs in a single interface.”
Elasticity and observability improved
NearSt developers leveraged the elasticity of serverless development for high volume, “bursty and spiky” apps, Adam said, and Epsagon for managing unpredictable workloads and debugging post production, saving the developers time and increasing developer velocity.
Time saved for engineers and rapid time-to-market for new, flawless features
In serverless, long chains of lambdas can go 30 to 40 events. “Debugging that without Epsagon was painful and took a half day or a day. So, with that many events, Epsagon has saved NearSt a half day of engineering time per engineer per month.”
“Epsagon has saved NearSt a half day of engineering time per engineer per month.”
“Without Epsagon, the engineers were largely working in the dark. With Epsagon, engineers are spending less time digging for information about what might have gone wrong and more time actually fixing the issue and getting features out. ”
Increased efficiencies with cost monitoring
Epsagon’s cost monitoring feature allowed NearSt to very quickly profile the efficiency of different functions. “Functions can be expensive to run if coded wrong, so with Epagon, we can identify expensive or slow functions, see a hit list of the most expensive functions, monitor changes by the last day or last week, and then can predict how much that inefficiency will cost us.”
“With Epsagon, NearSt can predict how much a function inefficiency will cost us.”
Time savings with trace, log and payload correlation
Correlating traces, logs, and payloads happens automatically with Epsagon. NearSt has its own correlation ids for a few services, Adam noted. “Epsagon saved us time in correlating traces, logs and payloads and lets us focus on the product engineering.”
“Epsagon saved us time in correlating traces, logs and payloads and lets us focus on the product engineering.”
Increased third-party compatibility
With Epsagon, NearSt. can filter Cloudwatch log data so that it is easier to find the log, Adam explained. As a result, Adam uses AWS Cloudwatch more.
Increased developer confidence and velocity
Adam advises other businesses and developers to create an “environment where it is safe to experiment, develop, deliver, and learn rapidly. To do that safely, you need to have good monitoring and observability.”
“Create an “environment where it is safe to experiment, develop, deliver, and learn rapidly. To do that safely, you need to have good monitoring and observability.”
Having the right tools attracts also helps attract and retain needed engineering talent. “During an interview, candidates might even ask to see your environment and toolsets. Being able to find engineers to be involved in “the next new thing” is half the equation. The second half is observation: when they see what we have, there is no fear around deployments,” Adam said.
“The value of Epsagon…I can see it developers’ eyes. They are sure they found the problem and fixed it.”
“If you can see everything in your environment, then you are more confident, especially if something is going wrong. The value of Epsagon…I can see it developers’ eyes. They are sure they found the problem and fixed it.”
“Having Epsagon is like having an extra Dev/Ops person watching your platform and telling you how you can improve it, which is amazing.”