Let’s understand Queue(Q) in the Distributed System

Joke

The queue is Q in Distributed System?……:).

We read a lot of things while designing and working with a distributed system. We know there is queue management, microservices, NoSQL DB, Relational DB, ES, etc….(There is no control on technology .) But do we really know why these components are used? What will be the impact if I don't use any particular component while system designing. Let's find out answers. Let's discuss around Queue System. Why? When?Which ? all questions will be answered. Just keep tuned.

First Focus on WHY

In one word answer, it is used for Asynchronous processing. Wait… The story does not end here. Let's take an example of daily life:-

I have a family function and the guest list is very long. There are 4 counters from where each person can get food. Now the question is how will I manage to serve all guests in minimum time .?

1. I will arrange more counters.
2. I will ask my guests to come in the queue and take the food one by one.

Now, this function in the tech world is a system and guests are like data or request that needs to be served. And yes, counters are my instances of the application. If I will run more instances then there will be more cost, more maintenance, and more headache. So I will choose the second option and use a queue system. I will queue my request, content, data, and will process one by one.

It totally depends on my use case that whether I want to maintain order or not. If my use case is that I want to do logging so order does not matter. But if there is a case of banking transactions then yes, it is extremely important to maintain the order of request.

Let's discuss on When

The answer to when is almost covered in WHY. When we want to do asynchronous processing and do not want to wait for response time. I just want to push in a queue and it will be processed one by one itself.

The second point to use Queue System is when I want to decouple my two components. It can be two microservices , a database, another queue etc.

Commonly used Queue System

Commonly used queue systems are SQS, Kafka, RabbitMQ. We will discuss all the queue systems one by one in my upcoming blogs.

I hope there will be a bit understanding of the queue system after reading this blog. Keep reading. :)