1 Introduction
RocketMQ is a distributed messaging middleware whose core technology involves several core concepts, including topics, queues, messages, producers, consumers, consumer groups, and subscription relationships. The following is a detailed description of these core technologies:
2 Core technology realization
2.1 Topic
1. Definitions: A topic is a top-level container for message transport and storage in RocketMQ, used to identify messages of the same type of business logic. It is a logical concept, not an actual message container.
2. Role：
- Data classification isolation: RocketMQ recommends splitting data of different business types into different topics for isolated storage and subscription.
- Data Identity and Privilege Management: Messages in RocketMQ are anonymous and unidentified, and messages of the same classification use the same topic for identification and privilege management.
3. Name: The topic name is used to identify the topic and is globally unique within the cluster.
4. Queue lists: As the constituent unit of a topic, it is the actual container in which messages are stored; a topic contains one or more queues within it, and messages are actually stored in each queue of the topic.
4. Types of messages: When creating a topic, you can specify the type of messages to be stored in the topic, including Normal (normal messages), FIFO (sequential messages), Delay (timed/delayed messages), and Transaction (transaction messages).
2.2 MessageQueue
1. Definitions: The queue is the actual container for message transport and storage in RocketMQ, and is the smallest storage unit for RocketMQ messages.
2. Characteristics：
- All topics in RocketMQ are composed of multiple queues as a way to achieve horizontal splitting of the number of queues and streaming storage inside the queues.
- Queues are naturally sequential, i.e., messages are written to storage in the order in which they enter the queue, and there is a natural sequential relationship between messages on the same queue.
- The position of messages in the queue and the order between messages is managed by marking them with a bit (Offset).
2.3 Message
1. Definitions: A message is the smallest data transfer unit in RocketMQ. The producer wraps the data to be sent into a message and sends it to the RocketMQ server.
2. Characteristics：
- Immutability: once a message is created, its content is unchangeable
- Persistence: RocketMQ persists messages by default, saving them to a storage file on the server side to ensure message traceability and recoverability in case of system failure scenarios.
2.4 Producers
1. Definitions: The producer is responsible for sending the message to the specified topic and queue in RocketMQ.
2. Mode of transmission：
- Synchronous sending: there is a return value and you must wait for the message to be sent successfully before it ends.
- Asynchronous Send: Returns a message immediately after sending it, while asynchronously receiving the state of the passed message.
- One-way send: sends a message unilaterally, without waiting for any response.
2.5 Consumer
1. Definitions: Consumers are responsible for subscribing to and consuming messages from RocketMQ on a specified topic.
2. Consumption processes：
- Consumers first pull messages from the Broker to the client and then start a consumer thread to consume those messages.
- Consumers can continuously pull messages from the Broker using the PullMessageService thread and consume them using the ConsumeMessageService thread.
2.6 Consumer Group (Consumer Group)
1. Definitions: A consumer group is a concept used in RocketMQ to manage consumers. Consumers within the same group share messages and consume them.
2. Role：
- Load balancing and fault-tolerant processing of messages can be achieved through consumer grouping.
- RocketMQ requires that the consumption behavior of all consumers under the same consumer grouping be consistent.
2.7 Subscription relationship (Subscription)
1. Definitions: The subscription relationship is the association relationship between a consumer grouping and a topic, as well as the filtering rules when a consumer subscribes to a message.
2. Role：
- With a subscription relationship, a consumer can specify the topics and message types to be consumed.
- RocketMQ implements message filtering and distribution through subscription relationships.
3 Summary
In summary, the core technology of RocketMQ involves several core concepts that collaborate with each other to achieve the high performance, high availability, and high scalability of RocketMQ.