Location>code7788 >text

An article thoroughly handle Redis and MySQL data synchronization

Popularity：129 ℃/2024-10-23 12:52:09

The Redis and MySQL consistency problem is one of the common challenges in enterprise applications, especially in highly concurrent and highly available scenarios. Since Redis is an in-memory database with very high read and write speeds, and MySQL is a persistent database usually used for reliable data storage, how to ensure the consistency of the two data requires the design and optimization of specific business scenarios.

We will analyze how to ensure data consistency between Redis and MySQL in different scenarios step by step with several typical business scenarios.

1. Cache Update Strategy: Cache Aside Pattern

Scene:

In most business systems, Redis is used as a caching layer to improve the read performance of the system, while MySQL is used as a persistent storage to ensure the reliability of the data. The most common scenario is:

  • The system first queries the Redis cache, and if there is no data in the cache, it then queries from MySQL and writes the data to the Redis cache.
  • When you update the data, you update MySQL and delete the Redis cache, which invalidates the cached data and ensures that you get the latest data the next time you read it.

Typical business scenarios:

  • Product details pageWhen a user requests the details of a product, the Redis cache is first queried, and if it is not in the cache, MySQL is queried and the query results are cached in Redis; if the product information changes, MySQL is updated and the cache in Redis is deleted.

Program Analysis:

  • Read Path: Fetches the cache from Redis and returns the data directly if the cache is hit; if the cache is not hit, it queries MySQL, writes the result to Redis, and returns the data.
  • Write Path: The update operates on MySQL first and then deletes the data from the Redis cache. The next time it is read, the latest data is retrieved from MySQL because of a cache miss.

How to guarantee consistency:

  • Cache Elimination Strategy: Deleting the Redis cache as soon as the MySQL data is updated ensures that you get the latest data when you read it next time. This means that you can avoid dirty data in the cache by "deleting the cache".

  • concurrency issue: When concurrent requests are high, a "cache avalanche" or "cache blow-through" problem may occur. For example, if A updates MySQL data, and B reads the old data at the moment of cache failure and caches it again in Redis, you can solve this problem by using theDouble Deletion Delay Strategy：

    1. Delete the Redis cache.
    2. Updating MySQL.
    3. With an appropriate delay (e.g. 500ms), delete the Redis cache again to ensure that there are no cache inconsistencies in concurrent situations.

  • Business Example:

    // Pseudo-code for updating product details
public void updateProduct(Product product) {
    // 1. Update the database
    updateProductInMySQL(product); // 2.
    // 2. Delete the cache
    updateProductInMySQL(product); // 2. deleteProductCache(()); // 4.

    // 3. Deferred double deletion to resolve inconsistencies under concurrency.
    try {
        (500); // Can be adjusted for real business scenarios.
    } catch (InterruptedException e) {
        // handle exception
    }
    deleteProductCache(()); }
}

2. Update the cache before updating the database

Scene:

In some real-time scenarios, consider updating the Redis cache first and then updating the MySQL database asynchronously.

Typical business scenarios:

  • spike system (computing)For example, when a user purchases an item, the inventory quantity is first updated in Redis to ensure a very low latency real-time experience. The changes are then asynchronously written to MySQL, ensuring consistency in persistent storage.

Program Analysis:

  • Read Path: Reads inventory information from the Redis cache, providing fast read responses.
  • Write Path: After updating inventory quantities in Redis, use a message queue or other asynchronous mechanism to synchronize the updates to MySQL.

How to guarantee consistency:

  • Final data consistency: Redis as a cache for real-time data on the front-end and MySQL as a persistent store for data on the back-end, consistency cannot be guaranteed when using an asynchronous update strategy that isstrong consistency, but can be used by using themessage queueand other means to ensureFinal Consistency. When writing to MySQL asynchronously, if the operation fails, consistency can be restored through a retry mechanism or a compensation mechanism.

  • Business Example:

    // Pseudo-code for reducing stock
public void reduceStock(Long productId, int amount) {
    // 1. Update the stock in Redis first.
    ("stock:" + productId, amount); // 1.

    // 2. Update the stock in MySQL asynchronously via the message queue.
    sendUpdateStockMessage(productId, amount); }
}

// Consuming the message queue to update MySQL
@RabbitListener(queues = "stock_update_queue")
public void updateStockInMySQL(UpdateStockMessage msg) {
    // Deduct stock from MySQL
    ((), ());
}

Consistency Assurance Strategy:

  • (math.) idempotency guarantees: Ensure that the processing of messages is idempotent, i.e., the same message, even if processed multiple times, does not result in duplicate inventory deductions.
  • Message Retry Mechanism: If updating MySQL fails when consuming a message, you can set up a retry mechanism or a message compensation mechanism to ensure final data consistency.

3. Dual-write operations (cache and database updated at the same time)

Scene:

Sometimes the business needs to update both Redis and MySQL data at the same time, such as user balance update, points reward system and other scenarios, Redis and MySQL need to synchronize writes.

Typical business scenarios:

  • point system: Increasing or decreasing points when a user makes a purchase requires updating the points records in both Redis and MySQL.

Program Analysis:

  • synchronous writing: When updating user credits, Redis and MySQL update the data at the same time. Transactional issues must be considered because of the need to ensure synchronization between the two stores.
  • Distributed transactions: If the system architecture is distributed, it may be necessary to use distributed transactions (e.g., the2PC, or more lightweight solutions such asTCC) to ensure consistency.

How to guarantee consistency:

  • Bi-write consistency issues: If you write to both Redis and MySQL, you may face consistency issues. A common solution is to pass theCompensation mechanism for servicesto realize. Specific Steps:

    1. Use database transactions to ensure that MySQL writes are successful.
    2. If a Redis write fails, you can either try to retry, or write the failed data to Redis after the transaction ends through a compensation mechanism.

  • Business Example:

    @Transactional
public void updateUserPoints(Long userId, int points) {
    // 1. update MySQL Integral in
    (userId, points);
    
    // 2. 同步update Redis Integral in
    ().set("user:points:" + userId, points);
}

Transactional safeguards:

  • local affairs: In a monolithic system, consistency can be guaranteed by relying on database transactions and Redis operations. If an operation fails, consistency is restored through a retry mechanism.
  • Distributed transactions: In a microservices architecture, double-write operations involve distributed transactions and may require the use of theTCC(Try, Confirm, Cancel) and other modes, or use message queues for eventual consistency compensation.

4. Write Back Policy

Scene:

Data write-back mode applies when Redis is used as the caching tier and MySQL is used as the persistent storage tier, but instead of synchronously updating MySQL immediately after a data modification in Redis, data write-back is triggered at a specific time.

Typical business scenarios:

  • Advertising Billing System: Ad clicks are saved in Redis to minimize the pressure of frequent database writes, and statistics from Redis are periodically written to MySQL in bulk.

Program Analysis:

  • delayed write-back: You can write data from Redis back to MySQL on a regular basis through timed tasks or triggers, which reduces the strain on MySQL and ensures data consistency.

How to guarantee consistency:

  • Persistence and Batch Synchronization: No data loss in the event of a Redis crash through Redis persistence mechanisms (e.g. RDB, AOF). Batch synchronization of MySQL triggered by timers or event-driven systems.

summarize

Redis and MySQL consistency guarantees need to be weighed against the characteristics of different business scenarios, and the main strategies include:

  1. Cache Aside Pattern: Commonly used in scenarios where there are many reads and few writes, and the cache is deleted during write operations.
  2. Asynchronous update (Write Behind): Update the cache before writing to MySQL asynchronously to ensure eventual consistency.
  3. bi-literalism: Update both Redis and MySQL at the same time, with transactions to ensure consistency.
  4. delayed write-back: Reduce frequent database operations with timed batch writes to MySQL.

Each strategy has a different application scenario, the design needs to consider the balance between consistency, performance and availability. This is considered to be the most complete and detailed, real synchronization program analysis of the whole network, fully integrated with the real business scenarios to consider the design. The so-called gift of roses, hands leave a fragrance, I hope you have a helpful role.

<Previous Next>