An article to completely understand and solve the Redis cache avalanche, cache hit, cache penetration

Cache avalanche, cache hit, and cache penetration are three types of problems commonly encountered when using cache in distributed systems, all of which can have a serious impact on system performance and stability. The following section describes in detail the definitions, causes, hazards, and common solutions for these three.

1. Cache Avalanche

1.1 Definitions

A cache avalanche isAt some point, a large number of caches are invalidated at the same timeThis can lead to a large number of requests hitting the database layer directly, resulting in a sudden increase in database pressure, which may even lead to a database crash and system unavailability.

1.2 Causes

Cache centralized invalidation: Normally, the cache expiration time (TTL) is set, but if a large number of cache keys are set to the same or close expiration point, then when these caches are centrally expired, it will result in a large number of requests not being able to read the data from the cache, and will have to access the database directly.
Cache server down: If the Redis server cluster goes down or fails, all cached data is instantly unavailable, and a large number of requests go directly to the database.

1.3 Hazards

Surge in database pressure: A large number of concurrent requests hitting the database instantly can cause the database to run out of connections, degrade performance, or even go down.
Service unavailable: As the database fails to respond to requests in a timely manner, the overall system response slows down or loses response altogether, resulting in unavailability of services.

1.4 Solutions

Cache expiration time decentralization：
- You can set different expiration times (TTL) for different cache keys, so that the expiration times of the caches are evenly distributed to avoid a large number of caches expiring at the same time. For example, when setting the TTL, add a random value to avoid cache keys expiring at the same time.
```
// When setting up the cache, add a random time to prevent centralized expiration
int randomTTL = ttl + new Random().nextInt(100);
().set(key, value, randomTTL, );
```
Cache Warmup：
- Load hotspot data into the cache in advance before the system goes live to avoid a large number of requests triggering cache misses at the same time.
downgrade strategy：
- In the case of cache avalanche, you can take strategies such as flow limiting and downgrading to slow down the pressure on the database. For example, when the cache is invalidated, directly return the default value or cache expired old data, to avoid the database to process a large number of requests in a short period of time.
Multi-level cache architecture：
- Using a combination of local cache (e.g., Caffeine, Guava, etc.) and distributed cache (e.g., Redis), some of the hot data can be put into the local cache first to reduce the pressure on Redis and the database.
Redis High Availability：
- Deploy a Redis master-slave cluster, using Redis's Sentinel mode or Redis Cluster to achieve high availability and avoid a single point of failure in the cache server.

2. cache hit (computing)

2.1 Definitions

A cache hit is aA hotspot data stored in the cache is invalidated at a certain point in timeThe situation where a large number of concurrent requests go to access this hotspot data at the same time, causing all the requests to hit the database, resulting in a sudden increase in the pressure on the database.

2.2 Causes

Hot Spot Cache Invalidation: When the cache of a hot data expires, a large number of requests pour into the database layer, and at this time the database needs to handle all the requests, resulting in an increase in the instantaneous pressure on the database.

2.3 Hazards

Excessive database pressure: The failure of hotspot data leads to an instantaneous large number of requests hitting the database directly, increasing the pressure on the database, which may trigger problems such as exhaustion of database connections and slower response, and in serious cases, may lead to database downtime.

2.4 Solutions

Hot data never expires：
- For particularly important hotspot data, consider not setting a cache expiration time and letting this data stay in the cache. The data expiration problem can be avoided by manually updating the data in the cache through a timed task.

Mutex mechanism：

In order to solve the problem of a large number of requests accessing the database at the same time at the moment of cache invalidation, a locking mechanism can be used to ensure that only one thread can access the database at the same moment. Other threads need to wait for that thread to write new data to the cache before reading the cache.

String value = ().get(key);
if (value == null) {
    // Get the distributed lock
    if (().setIfAbsent(lockKey, "lock", 10, )) {
        try {
            // Double-check
            value = ().get(key);
            if (value == null) {
                // Query the database
                value = (key); if (value == null) { // query database value = (key); // write result to cache
                // Write the result to the cache
                ().set(key, value, ttl, ); }
            }
        } finally {
            // Release the lock
            (lockKey); }
        }
    } else {
        // Wait for the lock to be released before reading from the cache.
        (100); // Adjust the wait time yourself
        value = ().get(key);
    }
}

Preventive Cache Updates：
- Asynchronously refresh the cache in advance when the hotspot data is about to expire. By detecting the access frequency of the hotspot data, it triggers the automatic update operation when it is about to expire, avoiding the breakdown problem at the moment of expiration.
dual caching mechanism：
- A two-tier caching strategy can be used: a primary caching tier is responsible for caching most of the data, and another secondary caching tier keeps the last cached data. When the primary cache fails, the data can be read directly from the secondary cache layer to avoid hitting the database directly.

3. cache passthrough

3.1 Definitions

Cache penetration is theThe data requested to be queried by the malicious user or program does not exist in either the cache or the database, resulting in each request hitting the database directly, bypassing the cache. Since the cache does not store the result of that request, all such requests bypass the cache and access the database directly, resulting in a huge strain on the database.

3.2 Causes

malicious attack (e.g. on a cell phone): Intentionally construct a large number of requests for non-existent data, such as queries for non-existent user IDs or product IDs, which are not in the cache and are therefore requested directly from the database.
Query for non-existent keys: Some business logics cannot avoid querying data that does not exist, e.g., a user querying for some outdated or incorrect request parameter and there is no corresponding record in the database.

3.3 Hazards

Database performance degradation: Since the queried data is neither in the cache nor in the database, each request hits the database directly, causing increased pressure on the database and even triggering a performance bottleneck.

3.4 Solutions

Cache empty results：

If a key of the query does not exist in the database, the results of the query for that key (e.g., thenull or null) is cached and a short expiration time is set to prevent repeated queries for that key from hitting the database.

// Query the cache
String value = ().get(key);
if (value == null) {
    // Query the database
    value = (key); if (value == null) { // query database
    if (value == null) {
        // Cache empty results to avoid cache-penetration
        ().set(key, "null", 5, ); } else { // Cache empty results to avoid cache-through.
    } else {
        // Write the value from the database to the cache
        ().set(key, value, ttl, ); } else { // Write the value from the database to the cache.
    }
}

Bloom Filter：
- Use Bloom filters to mark all data that may exist, all requests go through the Bloom filter first to check, only the data that the Bloom filter thinks exists will go to query the cache or the database. This effectively blocks out the vast majority of non-existent requests, preventing them from bypassing the cache and hitting the database directly.
```
BloomFilter bloomFilter = ((("UTF-8"))), 100000);

// Add all possible valid keys to the bloom filter
("validKey1");
("validKey2");

// Check the Bloom filter when querying
if (! (key)) {
    return "Invalid Key"; }
}
// Query the cache and database normally
```
parameter calibration：
- Before the query request enters the system, it carries out strict parameter checking and filtering to avoid illegal requests entering the system. For example, whether the user ID or product ID meets the format requirements, to avoid maliciously constructed illegal requests directly to the database.