Skip to content

Amazon messaging studies

Documentation Updates

Author: Jerome Boyer

Creation date: September 2023 - Updated 1/19/2024

  • 11/23: Add IBM MQ one-way point to point JMS implementation.
  • 12/23: Address an asynchronous implementation for S3 event processing
  • 1/24: Update SQS

This repository includes a set of demonstrations and studies about messaging services in AWS or as Open Source. The services in scope are: Amazon MQ, Amazon SQS, SNS, MSK and Kinesis.

The goal is to group personal notes and knowledge gathering around the different technologies, keep references to valuable sources of information, and do some hands-on work to be able to go deeper on some concepts. This content is as-is, does not represent my employer point of view, and can be shared as an open source. It may be useful for developers or solution architects.

Audience

The main readers of this website are people interested in AWS messaging and open-source messaging systems.

  • Solution architects
  • Developers, new to distributed system solutions, with interest in AWS and messaging systems

This site is a companion of my Event-Driven Architecture book.

The EDA reference architecture, as introduced in the following figure, uses the concept of event backbone, which is a messaging component supporting asynchronous communication between components

When zooming into this component, we can see queueing systems are very important to support event or message driven integration:

This repository tries to go deeper in each of those components.

Amazon MQ

Amazon MQ is a managed message broker for RabbitMQ or ActiveMQ. It runs on EC2 servers, and supports multi-AZs deployment with failover.

The Amazon MQ workshops contain a lot of very good information to get started with Amazon MQ. This repository addresses some of the specific subjects not covered in detail in those workshops: JMS programming model, clear separation between producer and consumer applications, deeper dive to the open source versions, starting developing locally using docker, failover testing... Addressing unit testing, integration test, reactive messaging and reactive programming with queueing systems.

The code will address the standard enterprise integration patterns of one-way point-to-point and request/responce point-to-point. But it may add more in the future.

For the IaC, we will use AWS CDK as much as possible, with different stacks to be able to reuse common infrastructure like VPC, IAM roles, and an optional Cloud9 environment, and finally the broker configuration.

Read more on Active MQ >>>

AWS Samples

AWS Sample git account includes samples for Amazon MQ, which can be used for inspiration:

Rabbit MQ

The other Open Source engine using in Amazon MQ.

To Be done.

How to connect to Rabbit MQ from different vpc or from on-premises?

This Creating static custom domain endpoints with Amazon MQ for RabbitMQ blog presents SSL and DNS resolution to access an NLB to facade brokers. Also the NLB can be used cross VPCs that are peered. Need NLB for broker specific TCP based protocol. Security group in the broker specify inbound traffic from the NLB only. NLB target group uses the broker static VPC endpoint address. NLB can also restrict who can acccess it.

Solutions in this repository

Some concrete code samples are part of this repository to demonstrate some of the important concepts or technology: