While building modern scalable applications, there are many scenarios which calls for asynchronous and event-based communication.  Asynchronous messaging enables services to remain loosely coupled by avoiding direct dependencies, which in turn promotes scalability.  Event-driven messaging involves capturing and reacting to various events that occur in a system.  AWS offers various managed services such as Amazon SNS and Amazon EventBridge which can be utilized by architects to design modern scalable applications.

Amazon EventBridge and Simple Notification Service (SNS) work in tandem to enable real-time event-driven communication and instant notifications within the AWS cloud ecosystem.  In this blog let us take the scenario of an application that uses Amazon Translate for doing batch translation. After the basic translation is completed, the application takes the translated document for further processing.  When translating a lot of documents using Amazon Batch Translation, it’s hard to keep track of the status of the individual jobs. How do we enable the application to know when a particular translation job has completed, so that it can trigger the next steps for further processing?  We have a solution – we need to enable asynchronous event-based messaging using Amazon Translate, Amazon EventBridge & Simple Notification Service (SNS).  This means that whenever there’s a change in the translation job status, you get an instant notification based on which you can take further actions.  Let us have a quick look at each of these services, before we delve deeper on how to properly wire them up for our scenario.

What is Amazon Translate?

Amazon Translate is a neural machine translation service that delivers fast, high-quality, and affordable language translation.  The Amazon Translate asynchronous batch processing capability enables organizations to translate a large collection of text or HTML documents.

What is Amazon EventBridge?

Amazon EventBridge is a serverless service that connects different applications and sends real-time data to your intended targets. The event publisher application can be (e.g. legacy) application, SaaS providers or internal AWS services and the target application can be Lambda functions, Amazon EC2 instances, Amazon Kinesis Data Streams, SNS, SQS, Step Functions etc.

Terms associated with EventBridge

    • Events: An event indicates a change in an environment. For e.g. Change of an EC2 instance from pending to running.

    • Rules: Incoming events route to targets only if they match the rule that is specified.

    • Targets: A target can be Lambda functions, Amazon EC2 instances, Amazon Kinesis Data Streams, SNS, SQS, Pipelines in CICD, Step Functions etc that receive events in json format.

    • Event Buses: Event bus receives event. When you create a rule, you associate it with a specific event bus, and the rule is matched only to events received by that event bus.

What is Amazon SNS?

Amazon Simple Notification Service (Amazon SNS) is a managed service that provides message delivery from publishers to subscribers (also known as producers and consumers). Publishers communicate asynchronously with subscribers by sending messages to an SNS topic.  An Amazon SNS topic is a logical access point which acts as a communication channel. A topic lets you group multiple endpoints (such as AWS Lambda, Amazon SQS, HTTP/S, or an email address).  To broadcast the messages of a message-producer system (for example, an e-commerce website) working with multiple other services that require its messages (for example, checkout and fulfilment systems), you can create a topic for your producer system. The first and most common Amazon SNS task is creating a topic.

Each subscriber to the topic will get all the messages (there is a new feature to filter messages). By default SNS offers 10,000,000 subscriptions per topic & 100,000 topics per account.

Solution Details

The solution contains the following steps:

  1. A User starts a batch translation job.
  2. An Amazon EventBridge rule picks up the event and an email notification is sent via an Amazon SNS topic.

To implement the solution,

Step 1: Create an SNS topic and Subscribe to it

  1. In type select “Standard”.
  2. Provide the name for the Topic to be created and click “Create Topic”.

3. Under the created Topic, click “Create subscription”.

4. Select the Protocol to be Email and provide email-id in the Endpoint.

5. After creating the subscription email confirmation is required.

6. Once the email is Confirmed, the topic can send notification to the subscribed email.

Step 2: Create Amazon EventBridge Rule

  1. Navigate to Amazon EventBridge => Rules => Create Rule
  2. Provide the name for the Rule and select Rule type as mentioned above. Click Next

3. Under Creation method select “Custom pattern”.

4. In the Json mention source as aws.translate.

5. In Select target(s) tab, select target as SNS Topic and select the Topic created in Step 1 (BatchTranslationTopic).

6. Click Skip to Review and create.

Step 3: Start an AWS batch translation job

As a final step, create an Amazon Translate batch job and provide the input text Amazon Simple Storage Service (Amazon S3) location, output Amazon S3 location, target language, and the data access service role ARN. For instructions on creating a batch translate job, see Asynchronous Batch Processing or Translating documents with Amazon Translate, AWS Lambda, and the new Batch Translate API.

The Amazon EventBridge rule picks up the StartTextTranslationJob API. When the job is complete, I get an email notification via Amazon SNS.

Conclusion

In this blog, we saw how you can use Amazon EventBridge to design a trigger for an Amazon Translate batch job. For this use case, we configured an email notification to notify when a job status changes (Completed / Failed / Stopped / Completed with Error); however, you can use this framework to trigger other Lambda functions or integrate with Amazon Simple Queue Service (Amazon SQS) for any postprocessing or any other automated steps, enabling you to build an end-to-end automated workflow which is both asynchronous as well as event-based.