Event-Driven Architectures with Amazon EventBridge

As we witness the shift from monolithic architectures towards microservices, maintaining loose coupling within distributed systems still remains as a challenge. As the frontier service takes more and more responsibilities -let’s say as the business evolves and new services emerge- even loose coupled architectures tend to become tightly coupled in the long term.

In a distributed system, the frontier service has to talk with each supporting downstream service. All these interdependent services are related with synchronous APIs. The dependency causes all systems to be blocked by each other, such that no backward incompatible change can be made for the system to work, and one service’s failure might actually mean huge failure for the overall system. While all APIs are developed into complex workflows, it becomes much more difficult for businesses to ensure that each service is healthy.

Amazon EventBridge is a serverless event bus that helps you handle data ingestion and routing of events that are not just custom to your application, but also events from AWS Services and other SaaS providers. The service simplifies and improves the serverless architectures through an event-driven approach and loose-coupling practices. You can publish your custom event into Amazon EventBridge and let your consumers process it. You can route your EC2 Auto Scaling events, CodePipeline events, Chime meeting events and all other AWS services that are mentioned here.

Amazon EventBridge enables you to store your event structure in the schema registry and map these schemas into code, which further allows you to use your events as objects in your code. This way, you don’t have to manage the complexity of interconnected applications and huge amounts of events and write appropriate code and configure APIs for each specific service generating/collecting them.

Benefits of Event-Driven Architectures with Amazon EventBridge

Decoupled Architecture

As discussed above, Amazon EventBridge enables decoupled architecture within distributed systems and services. You no longer need to use APIs with web hook mechanisms which typically lead to monolithic structures even if you use separate services. Within the event-driven architecture, your event targets don’t need to be aware of and directly communicate with the event sources themselves. You just filter and publish the events to Amazon EventBridge.

Simplified Event Routing and Development

It becomes more challenging and difficult for developers to handle errors across multiple workflows. Amazon EventBridge helps you ensure your services, and the event structure remains simple and manageable. Your services listen to events that they care about, as you define in the first place.

Amazon EventBridge allows you to connect your SaaS and business applications together without writing custom codes and bundling SDKs. What is more, you can find common event schemas in Amazon EventBridge, download related code bindings and represent your events as objects in your code.

Improved Application Availability and Resiliency

Synchronous APIs and a monolithic architecture mean that the availability and performance of your entire system actually depends on any individual service. When a single downstream service goes down, your entire workflow becomes impacted. Creating an event-driven architecture helps you improve your ability through asynchronous infrastructure operations.

Fully Managed and Serverless

Amazon EventBridge is a fully managed serverless service, meaning that you don’t have to manage or provision any infrastructure resources and the scaling is automatically handled by AWS. Along with fully managed service offering, Amazon EventBridge provides a pay-as-you-go model that requires no initial or minimum fees.

The service is integrated with currently 30 SaaS providers and various AWS services. What this means is that as your events are collected/routed between integrated SaaS providers and AWS services, your traffic goes through AWS' very own infrastructure rather than public internet, improving security even further. You can check the supported SaaS partner integrations from here:

Amazon EventBridge Examples

AWS Management Console

Amazon EventBridge console lets you create rules and event busses, find an existing schema from registry or create your own schema. Also you can configure partner events, including 30 SaaS integrations that you don’t need to write code to integrate. You can simply subscribe and process those events. Lately, archive and replay options are added. Archive lets you store events for the event bus that you specify for a retention period, or indefinitely whereas replay allows you to replay previous events from the archive.

When you click Create Rule, you can select predefined event schemas that contain all definitions for AWS services under Event Pattern. For services that don’t have an event definition, API Call via CloudTrail option is ready to use. After you select the service, you have a lot of service options available as a target. You can call a Lambda function, you can send a notification via SNS, you can add your event to SQS, or even trigger a CodeBuild project.

If you want to schedule an event, that's also available via Schedule Pattern option. You can enter a schedule or a cron expression for that event to be fired at.

You can also route your failed events to an SQS Dead Letter Queue to process later.

Schema registry lets you discover, create, and manage OpenAPI or JSONSchema Draft4 specification schemas for events on EventBridge. You can generate and download code bindings to help quickly build applications that use those events. Check out AWS documents for more details about creating schemas, code bindings and API Reference.

SAM CLI

Here’s an example event producer and consumer with the route examples.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: EventBridgeTestApp
Globals:
  Function:
    Runtime: nodejs12.x
    Timeout: 30
    MemorySize: 128

Resources:
  EventProducerFunction:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: EventProducerFunction
      CodeUri: producer/
      Handler: handler.producer
      Events:
        ProducerApi:
          Type: Api
          Properties:
            Path: /order
            Method: POST
      Policies:
        - Statement:
          - Effect: Allow
            Resource: '*'
            Action:
              - events:PutEvents      

  ApprovedOrderConsumerFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: consumer/
      FunctionName: ApprovedOrderConsumerFunction
      Handler: handler.approved
      Events:
        Trigger:
          Type: CloudWatchEvent 
          Properties:
            Pattern:
              source:
                - com.sufle.events
              detail-type:
                - order
              detail:
                status: ["approved"]

  RejectedOrderConsumerFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: consumer/
      FunctionName: RejectedOrderConsumerFunction
      Handler: handler.rejected
      Events:
        Trigger:
          Type: CloudWatchEvent 
          Properties:
            Pattern:
              source:
                - com.sufle.events
              detail-type:
                - order
              detail:
                status: ["rejected"]

  SpecificLocationOrderConsumerFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: consumer/
      FunctionName: SpecificLocationOrderConsumerFunction
      Handler: handler.location
      Events:
        Trigger:
          Type: CloudWatchEvent 
          Properties:
            Pattern:
              source:
                - com.sufle.events
              detail-type:
                - order                
              detail:
                cargo:
                  - "prefix": "IST-"
Outputs:
  ApiURL:
    Description: "API endpoint URL for Producer"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"

As you can see, we’ve added three consumer functions. We will pass an array of product information, and route our events according to that information. For our EventBridge consumers, event type is Trigger, type is CloudWatchEvent. source is the namespace of the events that this consumer will be catching, including detail-type, which indicates type of the event.

We will pass data to our producer via an API Gateway. An example request follows:

{
    "products": [{
        "id": 1,
        "amount": 1,
        "name": "MacBook Pro 2020 M1",
        "in-stock": true,
        "cargo": "IST-1234",
        "status": "approved"
    }, {
        "id": 2,
        "amount": 1,
        "name": "MacBook Air 2020 M1",
        "in-stock": false,
        "cargo": "LND-2345",
        "status": "rejected"
    }]
}

We are planning to route three functions with two different methods. 2 of them will catch it by containing an exact property value, third one will catch it by the content of a property.

Let’s prepare our producer:

const AWS = require('aws-sdk');
AWS.config.region = process.env.AWS_REGION || 'eu-west-1';
const eventbridge = new AWS.EventBridge();

exports.producer = async function(event, context) {

    let body = JSON.parse(event.body);
    const orderEvents = body.products.map(product => ({
        Source: 'com.sufle.events',
        EventBusName: 'default',
        DetailType: 'order',
        Time: new Date(),
        // Main event body
        Detail: JSON.stringify(product)
    }))
    const result = await eventbridge.putEvents({
        Entries: orderEvents
    }).promise()
    return {
        statusCode: 200,
        headers: { "content-type": "application/json" },
        body: JSON.stringify({"status": "OK"}),
    };
}

We have prepared events for each product that we are passing to this function. We are adding a wrapper object including Source for event namespace, DetailType for type of events under that and EventBusName for the name of the event bus that we are publishing on.

While our producer is ready, let’s add basic consumers to see what is coming through for events:

exports.approved = function(event, context) {
    console.log("APPROVED ORDER RECEIVED");
    console.log(event);
}

exports.rejected = function(event, context) {
    console.log("REJECTED ORDER RECEIVED");
    console.log(event);
}

exports.location = function(event, context) {
    console.log("ORDER SHIPPED TO ISTANBUL");
    console.log(event);
}

Go ahead and

sam build
sam deploy

Grab your url, and prepare the request, then send it!

After checking if it is successful, let’s check out CloudWatch for each consumer function's log streams.

We’ve routed a few very basic events for different consumers. You can start from here and make use of more routing options through different patterns and create more complex workflows. You can find code for this post in Github repository below: