AWS realized that most of its customers are using SaaS platforms for various solutions that complement what AWS offers, e.g., Pagerduty for handling infrastructure incidents and Zendesk for customer support. So, it decided to ease the integration of these platforms with AWS services in order to make customers more productive and powerful. This idea became the genesis of Amazon EventBridge in July 2019, which was built using the same event processing model used as the base for CloudWatch Events. Hence, EventBridge inherited the default event bus feature, accepting events from many AWS services and supporting calls for the PutEvents API.
On top of all of this, EventBridge supports custom and SaaS event buses. A SaaS event bus can be connected by a subscribed SaaS partner using an event source. SaaS applications can then be hosted anywhere and can publish events to this event bus, which routes the requests to the selected target based on the rules defined in EventBridge. One event bus can have multiple rules that filter events and route them to the concerned target.
Here is a workflow for an EventBridge service:
There are presently 15 SaaS partners already registered with AWS, 90+ event sources that can be selected as a service provider, and 20 integrated AWS services that can be selected as a target.
Existing Solutions for SaaS Integration with AWS Services
EventBridge has recently been launched, but the use case for integrating SaaS platforms with AWS services has existed for quite a long time. Let’s take a look at what solutions were being used before.
Polling is a pattern in which the receiver explicitly makes a call to the target system and waits for the response. Polling is generally scheduled with a frequency ranging from seconds to days.
In AWS, a Lambda Function behaves as a poller, and CloudWatch scheduler is used to set the frequency of polling. Once the scheduler triggers the Function, it makes a call to the SaaS platform API and retrieves the information required. Then, it calls other AWS services to process the data to complete the flow.
This seems to be a very simple solution, but there are a few bottlenecks with this implementation:
- It does not pull real-time data, thus there will always be an interval between each polling, and not having real-time data might not work for certain use cases.
- If the frequency is increased to reduce the interval of each polling, it may increase the load on the SaaS platform and affect performance; this may also incur additional costs, as the number of API calls will increase and a SaaS platform charges primarily based on the number of hits.
Webhooks is a very common pattern being used widely in software design today. Here, the sender sets up a hook with the receiver application. Then, whenever an event happens on the sender’s side, it triggers a call for the receiver and sends the information.
In this case, an AWS API Gateway behaves as a receiver, and the SaaS platform will have a webhook pointing to one of its endpoints.
This solution also has its pros and cons:
- It ensures that real-time data is being transferred to AWS services from the SaaS platform, but it introduces an API Gateway layer, which needs to be maintained for security (DDoS attacks)
- Maintenance requirements incur added cost due to API call charges
- You need to write the logic in an API Gateway to handle the events and route them to the target based on the business requirement
Amazon EventBridge Common Use Cases
EventBridge can be used in many business scenarios. Let’s discuss some of the most common ones.
Zendesk is a SaaS platform that provides customer support solutions. Any change in the customer support tickets can be sent to EventBridge to trigger a workflow in AWS. Or you can apply ML using the AWS SageMaker service to attach a customer satisfaction score on the support ticket.
EventBridge can be used to implement ETL (Export, Transform, and Load) operations. It can access the SaaS platform data, process it using an AWS service, and send the data from one system to another.
For example, EventBridge can connect to PagerDuty incidents and send them to an Amazon Redshift data warehouse. There, you can analyze the data and generate reports on remediation velocity and average operational load for IT operation teams.
EventBridge can be used for application performance monitoring and prevent downtime by sending an application’s metrics to an analytics engine. It can also raise alerts to systems that are close to timeout or out of memory.
For example, we can create a custom processing pipeline to any error, exception, timeout, or out-of-memory alert that Epsagon detects with custom behavior. This behavior can be deployed on a Lambda function and stream the alert to any destination service, like Lambda or SQS, through EventBridge.
How Amazon EventBridge Compares with Other AWS Services
When AWS launched EventBridge and showcased its features, architects started comparing it with other AWS services and found overlapping features. Here below, we compare EventBridge with a few other similar services.
EventBridge vs. SNS
SNS is similar to EventBridge in terms of having pub-sub capability. It can publish a message and send it to multiple subscribers. SNS can also filter the message per subscription using subscription policy the same way that rules do in EventBridge.
However, SNS can only directly interact with Lambda and SQS downstream services. EventBridge, on the other hand, can natively integrate with 20 target services. For example, if SNS needs to trigger a Step Function, it cannot do it directly. It has to trigger a Lambda Function, which can then trigger a Step Function. But EventBridge can communicate with a Step Function natively without needing a Function.
SNS is very useful when we get thousands or millions of subscriptions per topic and can support millions of TPS through each topic. EventBridge can support only 400 requests per second.
EventBridge vs. Kinesis
Kinesis is a streaming model, meaning that all consumers tagged to the stream will see every message on that stream. There is also a limit to the total number of services that can consume a single stream. Plus, each individual consumer has to filter out any messages that he is not potentially interested in.
Kinesis can buffer messages, whereas EventBridge cannot. EventBridge needs SQS or Kinesis to store the messages if the downstream system cannot scale per the message volume.
EventBridge and Kinesis both retry to deliver events to the configured target service for 24 hours. Beyond that, the transaction will be marked as failed.
EventBridge vs. CloudWatch Events
EventBridge was developed from the same event processing model that was used for CloudWatch Events. So it inherited all the same features.
CloudWatch Events supports a default event bus that can accept events from other AWS services. EventBridge, in addition to the default bus, supports the custom event bus and SaaS event bus. These buses are used to integrate with other services sitting outside AWS.
Pricing of EventBridge
For Amazon EventBridge, we pay for the events published to the event bus. AWS does not charge for the rules and event delivery, and there are no upfront commitments.
- Schema Registry is free.
- Schema Discovery is free for the first 5M events per month; after this threshold, AWS will charge $0.1 per million requests.
- AWS service events are free.
- Custom/SaaS/Cross Account events are charged $1 per million events.
- Size of payload is considered: 64KB chunk = 1 event (e.g., an event with a 256KB-size payload is billed as 4 events).
Build a Use Case with Amazon EventBridge
To integrate SaaS platform services with the AWS service using EventBridge, you must set up three components:
- Partner Event Sources
- Event Buses
We’ll be using the Epsagon SaaS platform for this use case and show the steps required to set it up:
Setting Up Event Source & Event Bus
- First, go to the AWS Console, search for “EventBridge Service,” and launch it.
- On the EventBridge Screen, you will see “Partner Event Sources” as an option in the left-side menu. This will show all the partners that are registered with AWS.
3. Now, click on Epsagon, and read through its details and the use cases it supports.
4. Click on “Set up event source” link. It will show three simple steps to set up an event source.
5. Once the event source is created at the partner website, “Partner event sources” shows that the request for Epsagon is pending.
6. Select the pending event source, and review the details. Once this looks good, you can click the “Associate with event bus” button.
7. The event bus name must match the partner event source. Once the event bus is created and you review all the information, you can click on the “Associate” button.
8. You should now be able to see the new custom bus on the “Event buses” screen.
Rules for Setting Up
- Once you click on the “Rules” link in the left navigation bar and then select the newly created bus, it will show that there are no rules associated with this event bus.
2. You can click on the “Create rule” button to create a rule for the Epsagon events processing. First, you need to give it a meaningful name and a description.
3. Then, you have to define the event-driven pattern and select the service name as “Epsagon.” You can also edit the pattern to change the rule. It can read any content of the event and put rules around it.
4. Now, select the custom event bus that was created for Epsagon events, and then choose the target to where these events need to be passed for further processing. Below, the target is “Lambda function.”
5. Once you create the rule, it will be activated and filter the events being sent from the Epsagon account that was integrated with your AWS account. A few event examples include TicketCreated, CommentCreated, AgentAssignmentChanged, CustomFieldChanged, and StatusChanged.
This completes the Amazon EventBridge event processing workflow for Epsagon’s platform.
AWS has launched Amazon EventBridge to fill in the gaps of integration between SaaS partners and AWS services. This was a much-needed service, given that many small SaaS vendors are coming up with good solutions that complement AWS services.
However, CloudWatch Events has become redundant now that EventBridge has been launched and may be deprecated eventually. AWS has indicated this by allowing access for all CloudWatch Event buses through EventBridge as well. Overall, EventBridge has been a successful launch, making SaaS integration a far smoother process.