mirror of
https://github.com/ByteByteGoHq/system-design-101.git
synced 2026-04-02 00:57:24 -04:00
ee4b7305a2
This PR adds all the guides from [Visual Guides](https://bytebytego.com/guides/) section on bytebytego to the repository with proper links. - [x] Markdown files for guides and categories are placed inside `data/guides` and `data/categories` - [x] Guide links in readme are auto-generated using `scripts/readme.ts`. Everytime you run the script `npm run update-readme`, it reads the categories and guides from the above mentioned folders, generate production links for guides and categories and populate the table of content in the readme. This ensures that any future guides and categories will automatically get added to the readme. - [x] Sorting inside the readme matches the actual category and guides sorting on production
35 lines
2.2 KiB
Markdown
35 lines
2.2 KiB
Markdown
---
|
|
title: "IBM MQ -> RabbitMQ -> Kafka -> Pulsar: Message Queue Evolution"
|
|
description: "Explore the evolution of message queue architectures: IBM MQ to Pulsar."
|
|
image: "https://assets.bytebytego.com/diagrams/0271-message-queue-evolve.png"
|
|
createdAt: "2024-03-05"
|
|
draft: false
|
|
categories:
|
|
- "database-and-storage"
|
|
tags:
|
|
- "Message Queues"
|
|
- "System Design"
|
|
---
|
|
|
|
|
|
|
|
* IBM MQ
|
|
|
|
IBM MQ was launched in 1993. It was originally called MQSeries and was renamed WebSphere MQ in 2002. It was renamed to IBM MQ in 2014. IBM MQ is a very successful product widely used in the financial sector. Its revenue still reached 1 billion dollars in 2020.
|
|
|
|
* RabbitMQ
|
|
|
|
RabbitMQ architecture differs from IBM MQ and is more similar to Kafka concepts. The producer publishes a message to an exchange with a specified exchange type. It can be direct, topic, or fanout. The exchange then routes the message into the queues based on different message attributes and the exchange type. The consumers pick up the message accordingly.
|
|
|
|
* Kafka
|
|
|
|
In early 2011, LinkedIn open sourced Kafka, which is a distributed event streaming platform. It was named after Franz Kafka. As the name suggested, Kafka is optimized for writing. It offers a high-throughput, low-latency platform for handling real-time data feeds. It provides a unified event log to enable event streaming and is widely used in internet companies.
|
|
|
|
Kafka defines producer, broker, topic, partition, and consumer. Its simplicity and fault tolerance allow it to replace previous products like AMQP-based message queues.
|
|
|
|
* Pulsar
|
|
|
|
Pulsar, developed originally by Yahoo, is an all-in-one messaging and streaming platform. Compared with Kafka, Pulsar incorporates many useful features from other products and supports a wide range of capabilities. Also, Pulsar architecture is more cloud-native, providing better support for cluster scaling and partition migration, etc.
|
|
|
|
There are two layers in Pulsar architecture: the serving layer and the persistent layer. Pulsar natively supports tiered storage, where we can leverage cheaper object storage like AWS S3 to persist messages for a longer term.
|