DLQ Dashboard using React JS & Mulesoft

This document talks about one of my most passionate projects – the DLQ Dashboard. In short, it's a user-friendly visualization of SQS FIFO Queues, capable of executing basic operations on queue messages. Its frontend is written in React JS, and the backend is coded in Mulesoft. I've polished it a lot and now it's open-source.

Alright, that's enough for the introduction. Without further ado, let us jump right into the demo. I've included English Subtitles too; be sure to enable them if needed.

Demo Video

Now that you've seen the demo, I believe you have a good understanding of what this project is all about. If you want to take a look under the hood and setup these codes in your local environment, I've added repository URLs below.

Repository URLs

• Frontend: React JS Code
• Backend: Mulesoft Code

Don't forget to checkout the README.md files of these projects to learn more.

Useful Insights

AWS IAM Role Setup

I understand that AWS IAM Setup might be tricky for new developers. The Mulesoft code uses SQS Connector, which is configured for Role-based authentication. I've thoroughly discussed configuring Role-based AWS connectivity in Mulesoft in one of my previous blogs. Click this Link to jump to the Local Run Setup section.

Note that the IAM User (Access Key & Secret Key) is only used for running this code in a local environment and is not required (and not recommended) for Cloudhub deployments. Please go through this article - Link in full to learn more.

FIFO Queues Only

This is because of the AWS SQS queue architecture and the workings of AWS SQS REST API's read operation (Under the hood, Mulesoft SQS Connector utilizes AWS SQS REST API). In a Standard SQS queue, the read operation picks messages randomly, and there is no certainty of how many messages will be retrieved (1-2 according to my experience). But in a FIFO queue, the read operation happens on a First-In-First-Out basis, and we receive the exact number of messages as specified (Maximum 10) in the read component. Read more about this on Salesforce article - Link. Also, learn more about the two AWS SQS Queue types here - Link.

Read Operation's Maximum 10 Messages Limit

The read operation in the SQS Connector can fetch a maximum of 10 messages from a SQS Queue. This limit exists on AWS SQS REST API as well as other AWS SDKs. AWS SQS ReceiveMessage documentation - Link. Mulesoft SQS Connector read operation documentation - Link.

SQS Queue Message Visibility Timeout

It is important to understand that when a message is picked or read from a SQS Queue, it goes into the "In-Flight" state for the amount of time set in the Message Visibility Timeout field. This affects all operations that are being performed on the SQS Queue. I recommend that you keep its value somewhere between 3-5 seconds.

E.g., Let's assume you have a SQS Queue that contains 15 messages and its message visibility timeout is set to 3 seconds. Now if you make a read 10 messages call and then within 3 seconds you make a call to check current queue count, you will see that the count will come in at 5 because the first 10 messages are in the "In-Flight" state. If you make another current queue count call after 3 seconds, it will be 15.

This behavior is evident in this DLQ Dashboard as well if consecutive page refreshes are made too rapidly (within 3 seconds).

Multiple Listeners in Mulesoft Code

Ideally, Mulesoft API development starts with a RAML specification, which is scaffolded in the implementation code. Scaffolding generates flows and employs an APIKit Router to handle routing. But this also means that the RAML gets added to the Mulesoft project as a dependency. It cannot be downloaded by anyone other than the Anypoint Platform account holder if the specification is published to the exchange privately.

Hence. in this code, I've neither created a RAML specification nor implemented an APIKit Router. This was done so that anyone can clone, import, and run this code without worrying about private dependencies.

Coding Practices in Frontend

This was my first attempt at hardcore frontend development and React JS. It will get better in the future.