upfront
In today's high concurrency Internet applications, the challenge for every technical team is to ensure that the system can operate reliably even under the impact of huge amounts of traffic. Speaking of which.Message Queuing (MQ) It's the "big man" behind it all.
Whether it's processing requests asynchronously, smoothing out traffic spikes, or keeping system modules independent of each other, MQ is an indispensable helper. So.How does MQ chip away at peaks? Or how does it allow complex systems to be decoupled? Today we will talk about these three core features of MQ, to see how it helps the system to run efficiently and stably.
1. What is MQ (Message Queuing)?
Message Queue (Message Queue, abbreviated as MQ) is actually a "pipe", used to pass messages between different systems or services.
Imagine, it is like a post office, the sender of the letter to the post office, the post office and then in order to deliver the letter to the hands of the recipient, the whole process of everyone doing their own thing, the sender does not have to worry about the recipient has not immediately received the letter, so that everyone's work does not interfere with each other.
In the system, MQ is mainly responsible forMessage Delivery and Asynchronous Processing. It helps in message passing between systems and also enables decoupling of systems and efficient asynchronous processing. Common MQ tools include RabbitMQ, Kafka, ActiveMQ, and so on.
2. Core role of MQ I: Asynchrony
synchronous is one of the most important roles of MQ. Asynchronous means that you don't have to wait for a task to complete before taking the next action. Instead, you leave the task to the MQ and can go on to do something else yourself. It's as if you outsourced a task to a helper (MQ), then continued to work on other tasks, and when the MQ finished the task, you went back to work on the result.
Benefits of using asynchronous MQ:
- Improve system performance: No need to wait for tasks to be completed, able to work on other tasks immediately.
- Better user experience: The system responds quickly to user-initiated requests, while complex operations in the background can be handled slowly.
An example:
In an e-commerce system, after a user places an order, the system needs to send a notification to the warehouse so that they can prepare to ship the order. Without MQ, the system may wait until the warehouse has finished processing the order before telling the user that the order has been successfully placed, so the user has to wait for a long time.
But with MQ, the system can quickly tell the user that "the order has been successful", and the subsequent warehouse processing through MQ asynchronous notification, the user does not have to wait for the end of all the processes in the background.
Recently, I inadvertently obtained a copy of the brush notes written by the big brother of Ali, which immediately opened my two veins, and it was not so difficult to enter the big factory. This is written by the big brother.7,701 pages of brush-up notes written by BAT bigwigs, which made me get a soft offer
Sample code (Spring Boot RabbitMQ):
// autotroph (original producers in a food chain)：Send message to message queue
@Component
public class OrderProducer {
@Autowired
private RabbitTemplate rabbitTemplate;
public void sendOrderMessage(String orderId) {
("orderQueue", orderId); // Send order messages asynchronously
}
}
// consumers：Receive messages from the queue and process them
@Component
@RabbitListener(queues = "orderQueue")
public class OrderConsumer {
@RabbitHandler
public void handleOrderMessage(String orderId) {
// Simulate order processing logic
("Processing order: " + orderId);
}
}
In this example, theOrderProducer will send the order message to theorderQueue Queue.OrderConsumer Asynchronous processing of orders, the user will not feel the complex logic of the backend, and will only receive feedback on the success of the order.
3. Core role of MQ II: peak shaving
peak reduction Another core role of MQ is to "peak" the system so that it doesn't crash in the face of a surge in traffic. Peak shaving means "smoothing out" the sudden influx of highly concurrent requests so that the system doesn't crash in the face of a traffic surge. It's like a "reservoir" that stores the instant influx and then slowly releases it for processing when the traffic returns to normal.
Benefits of using peak-shaving MQ:
- Prevents system overload: In the face of sudden bursts of highly concurrent traffic, the system will not crash as it exceeds its load.
- Smooth handling of flow: Queue the requests through MQ during peak hours and process them gradually after the traffic has stabilized to ensure that the system does not suffer from performance degradation due to short-term traffic surges.
An example:
In a spike event, users initiate a large number of requests at the same time, and if the system processes these requests directly, the server may crash. With MQ, these requests can be queued first, and when the traffic stabilizes, the system will gradually process the requests in the queue. This not only protects the stability of the server, but also allows users to experience the smoothness of the spike service.
Sample code:
// Spike requests are sent to the message queue for peak trimming
@Component
public class SeckillProducer {
@Autowired
private RabbitTemplate rabbitTemplate;
public void sendSeckillMessage(String seckillId) {
("seckillQueue", seckillId); // Seconds request queue
}
}
// Consumers get seconds requests from the queue，sequential treatment
@Component
@RabbitListener(queues = "seckillQueue")
public class SeckillConsumer {
@RabbitHandler
public void handleSeckillMessage(String seckillId) {
// Simulate the logic for processing spike requests
("Processing seckill request: " + seckillId);
}
}
By queuing the spike requests through MQ, you can smoothly handle bursty traffic and prevent the system from crashing in a short period of time due to too much concurrency.
Recently, I inadvertently obtained a copy of the brush notes written by the big brother of Ali, which immediately opened my two veins, and it was not so difficult to enter the big factory. This is written by the big brother.7,701 pages of brush-up notes written by BAT bigwigs, which made me get a soft offer
4. Core role of MQ III: decoupling
decoupled is the third major role of MQ, which is simply that theReduce system dependencies by keeping system modules from interfering with each otherIn the absence of MQ, system A and system B may need to synchronize their communication directly. In the absence of MQ, system A and system B may need to synchronize their communication directly, but this is too coupled, and if one system has a problem, the other system will be affected as well.
With MQ, system A doesn't need to wait for system B to finish processing, it only needs to send the message to MQ, and system B processes the message asynchronously according to its own situation. In this way, system A and system B are decoupled, A does not need to care whether B is busy or not, and B does not need to respond to A's request immediately.
Benefits of using decoupled MQ:
- Reducing dependencies between systems: Each system can handle its own logic independently of each other.
- Increased system flexibility: Systems communicate with each other through MQ. If a system is down, MQ can store the message temporarily and continue to process it when the system recovers.
An example:
In an e-commerce system, the order service and the inventory service need to communicate. Without MQ, the order system must wait for the inventory system to confirm the inventory before continuing the process. However, with MQ, after the order system places the order, it can send the message to MQ, and the inventory system will take its time to process it without affecting the flow of the order service.
Sample code:
// The order system sends messages to the inventory system
@Component
public class InventoryProducer {
@Autowired
private RabbitTemplate rabbitTemplate;
public void sendInventoryMessage(String orderId) {
("inventoryQueue", orderId); // Order messages are sent to the Inventory Service
}
}
// Inventory system asynchronously processes order messages
@Component
@RabbitListener(queues = "inventoryQueue")
public class InventoryConsumer {
@RabbitHandler
public void handleInventoryMessage(String orderId) {
// Simulate inventory deduction logic
("Processing inventory for order: " + orderId);
}
}
After decoupling through MQ, the order service can quickly respond to the user's order operation, while the inventory service asynchronously handles the inventory deduction operation, and the two systems do not interfere with each other, reducing the degree of coupling.
Recently, I inadvertently obtained a copy of the brush notes written by the big brother of Ali, which immediately opened my two veins, and it turned out to be not so difficult to enter the big factory. This is written by the big brother.7,701 pages of brush-up notes written by BAT bigwigs, which made me get a soft offer
5. Scenarios of use in detail
5.1 Asynchronous notifications
In asynchronous notification scenarios, MQ can help the system respond to user requests in a timely manner, while the background slowly processing the subsequent logic. For example:
- Welcome email after user registration: When a user registers successfully, the system sends the email asynchronously via MQ without blocking the user's registration process.
- Coupon sent after order completion: After the user completes the order, the coupon is issued asynchronously via MQ and the order process is not slowed down.
5.2 Peak reduction scenarios
In high concurrency scenarios, MQ can effectively perform peak shaving. For example:
- e-commerce spike campaign: In a spike event, where a large number of users make requests at the same time, MQ smoothes out the traffic by queuing up the requests to avoid server crashes.
- Payment system peaks: When a large number of users initiate payment requests, MQ can help the system to process them sequentially to avoid paralyzing the payment system due to high concurrency.
5.3 System decoupling
MQ is ideal in scenarios where decoupling is required. For example:
- Order and inventory decoupling in e-commerce systems: The order service and inventory service communicate asynchronously via MQ to avoid problems caused by high coupling.
- Decoupling of logging systems from business systems: The logging system can collect log information from each module through MQ, and the business system only needs to send the logs to MQ and does not need to communicate with the logging system directly.
6. Summary
MQ (Message Queue) The central role is mainly reflected in theAsynchronous processing, peak shaving and decoupling。
- With asynchronous processing, the system can improve responsiveness and user experience;
- With peak shaving, the system can operate stably in the face of high concurrent traffic and avoid overloading;
- By decoupling, there are fewer dependencies between systems, increasing flexibility and maintainability.
Whether it is in the order processing of the e-commerce system, the spike scenario, or the decoupling design of the system modules, MQ is a powerful tool. With MQ, the system can better cope with complex business scenarios and high concurrency requirements, and maintain stable and efficient operation.
One final note (please follow, like, don't whore me out)
Recently, I inadvertently obtained a copy of the brush notes written by Ali's big brother, which immediately opened up my two chakras, and it turned out to be not so difficult to enter the big factory.
It was written by the big guy.7,701 pages of brush-up notes written by BAT bigwigs, which made me get a soft offer
This article, which has been featured on, my tech site: Programmer's Programming Resource StationThe company has a complete interview, work technology, architect growth path, and other experience to share.
Ask for a one-click trifecta: Like, Share, Favorite!
Likes are really important to me! Ask for likes online and I'd appreciate a follow!