Location>code7788 >text

Redis expired key deletion and memory elimination strategies [Redis series 4]

Popularity：78 ℃/2025-03-28 19:41:18

0. Preface

Memory resources are very valuable for Redis servers. If some expired keys are not deleted, they will cause waste of resources.

So, this article will briefly introduce two strategies for expiration key deletion and memory elimination based on the information collected by the blogger, for reference only.

Blogger Redis related articles are here:/hnzhengfy/category/

1. Redis expiration time configuration

1.1 Methods to set expiration time

First, specify the expiration time while setting the key-value pair:

# grammar
 SET key value [EX seconds] [PX millionseconds]
 # Example
 SET mykey "hello" EX 60 # Set the value of mykey to "hello" and expires after 60 seconds
 SET mykey "hello" PX 60000 # Set the value of mykey to "hello" and expire after 60000 milliseconds (i.e. 60 seconds)

Then add an expiration time to the existing key, a total of four commands:

# 【expire command】Set the expiration time in [seconds]
 # grammar:
 EXPIRE key seconds
 # Example:
 SET mykey "hello"
 EXPIRE mykey 60 # Set mykey to expire after 60 seconds

 # [pexpire command] Set the expiration time in [milliseconds]
 # grammar:
 PEXPIRE key millionseconds
 # Example:
 SET mykey "hello"
 PEXPIRE mykey 60000 # Set mykey to expire after 60000 milliseconds (i.e. 60 seconds)

 # [expireat command] Set a specific Unix timestamp as expiration time (in [seconds])
 # grammar:
 EXPIREAT key timestamp
 # Example:
 SET mykey "hello"
 EXPIREAT mykey 1743283200 # Set mykey to expire at Unix timestamp 1743283200

 # [pexpireat command] Set a specific Unix timestamp as expiration time (in [milliseconds])
 # grammar:
 PEXPIREAT key timestamp_ms
 # Example:
 SET mykey "hello"
 PEXPIREAT mykey 1743283200000 # Set mykey to expire at Unix timestamp 1743283200000

1.2 How to determine whether a key has expired?

  • First, let’s see how to save the next key with expiration time?

When Redis stores key-value pairs in memory, one is maintained for each keyIndependent data structures to record its meta information (such as expiration time)。

Key-value pairs are stored in a global hash table (dict). Each key corresponds to a redisObject, which contains the value of the key and meta information related to the key (such as type, encoding method, etc.).

Expiration time is stored in a special dictionary (expires dictionary), the key is the normal key name, and the value is the expiration time of the key (stored as a Unix timestamp).

# Example
 SET mykey "hello" EX 60
 # mykey is stored in the main dictionary, pointing to the value "hello".
 # At the same time, mykey is also added to the expires dictionary with a value of current time + a timestamp of 60 seconds.

If a key does not set an expiration time, it will not appear in the expires dictionary.

  • Expiration check will be performed first when reading the key

When accessing a key (such as the GET command), Redis will first check whether the key exists in the expires dictionary. If present, Redis compares the current time to the expiration time stored in the expires dictionary:

If, current time >= Expiration time: This key is considered to have expired, and returns nil.

If, current time < expiration time: This key is considered to be still valid.

2. Expired key deletion strategy

2.1 Lazy Deletion (Passive Deletion)

Lazy removal is part of the Redis core work mechanism, so it is enabled by default and cannot be turned off manually via configuration items.

There are three key points for lazy deletion:

Passive trigger:When the client accesses a key (such as GET, DEL, etc.), Redis will first check whether the key has set an expiration time.

Expiration check:If the key has expired (current time ≥ expiration time), Redis will immediately delete the key and return nil (indicating that the key does not exist).

Not taking the initiative to clean up:The expired key that is not accessed will not be deleted and the check will not be triggered until the next time it is accessed.

  • Advantages

Efficiency: Check for expired only when keys are accessed, avoiding unnecessary CPU resource consumption.

Low latency: There will be no fluctuations in system performance due to the cleaning operation of expired keys.

  • Disadvantages

Memory waste: Expired keys that have not been accessed for a long time will continue to occupy memory, which may lead to memory leakage.

Uncontrollability: Unable to actively clean up expired keys that are not accessed.

2.2 Periodic Deletion (Periodic Deletion)

Regular deletion means scanning the keys in the Redis database at a fixed time interval, and deleting expired keys immediately. You can set the specific frequency through configuration.

Key points:

Active trigger:Redis periodically scans expired keys through a backend task (serverCron) and actively deletes them.

Random sampling:Each time you scan, Redis will randomly select a batch of keys from the expires dictionary of the database (20 by default), check whether it expires, and delete expired keys.

Dynamic adjustment:Control the scanning frequency through the parameter hz (default value 10), that is, perform hz scan tasks per second. Scan time is limited by CPU time (default is no more than 25% of CPU time).

Brief steps:

  1. Traversing all databases: Check each database (DB) in turn.
  2. Random sampling: Randomly select 20 keys from the expires dictionary of each database.
  3. Delete expired keys: Remove expired keys from the main dictionary and expires dictionary.
  4. Repeat Scan: If more than 25% of the sample keys are found to have expired, repeat the scan until the expiration key ratio is below the threshold or reaches the CPU time limit.
  5. Exit condition: Scan stops if the preset CPU time limit is reached (such as 25ms per second).
  • Advantages:

Memory friendly: Actively clean out expired keys to reduce memory waste.

Controllability: Balance performance with memory footprint by adjusting hz and scanning parameters.

  • Disadvantages:

Delay: The expired key may still exist within the scan interval and cannot be deleted immediately.

Resource occupancy: Frequent scans may increase CPU load (but extreme cases are avoided through time limits).

  • How to configure it?

hz: Configure the execution frequency of serverCron (default 10, i.e. 10 times per second).

ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC: This parameter determines that Redis isSingle Expiration Key Cleanup PeriodAmong them, the maximum CPU time can be consumed (default 25%).

# Calculation formula:
 Maximum allowable CPU time (microseconds) = (1000000 * ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC) / ( * 100)
 # For example: when hz=10
 Maximum allowable time = (1000000 * 25) / (10 * 100) = 25,000 microseconds (i.e. 25 milliseconds) # Single cleanup, run only 25 milliseconds
 # For example: when hz=1
 Maximum allowable time = (1000000 * 25) / (1 * 100) = 250,000 microseconds (i.e. 250 milliseconds) # Single cleanup, run only 250 milliseconds

If a cleaning discovers a large number of expired keys (such as 6 out of 20 sample keys), Redis will continue to scan until the expiration key ratio drops or time runs out.

2.3 Comprehensive strategy: lazy deletion + regular deletion

Redis's expired key deletion strategy combines lazy deletion and periodic deletion, which balances performance and memory footprint through the complementarity of the two.

The benefits of combining the two:

Efficiency: Lazy deletion is only triggered when the key is accessed, avoiding unnecessary CPU overhead. Regular deletion balances CPU load and memory cleaning efficiency through random sampling and dynamic adjustment.

Memory friendly: Lazy deletion ensures that the accessed expired keys release memory immediately. Regularly delete and actively clean up expired keys that are not accessed to reduce the risk of memory leakage.

Typical application scenarios:

againstHigh frequency accessIn scenarios, lazy deletion has more advantages, which can quickly clean out expired keys and reduce memory usage. Of course, there are inevitably some keys that are not accessed, and at this time, they need to be deleted regularly to clean up in time.

againstMemory sensitiveIn scenarios, periodic deletion can prevent the expired key from taking up space for a long time, but carefully consider the value of the configuration item to avoid CPU overload. Lazy deletion will clear the invalid key as soon as possible.

againstLow resource environment, the organic cooperation between the two is even more necessary. Lazy deletion reduces CPU overhead, and periodic deletion controls resource consumption through parameters. It may also be necessary to sacrifice some memory utilization in exchange for CPU savings.

3. Memory elimination strategy

3.1 Triggering conditions

Redis's Memory Eviction Policy is a mechanism used to decide how to eliminate (delete) existing data to make room when Redis reaches its maximum memory limit (configured by the maxmemory parameter).

Troubleshooting only if both conditions are met:

  • Maximum memory limit is reached: When the memory used by Redis exceeds the threshold configured by maxmemory.
  • Need to allocate new memory: When trying to write new data or update existing data, if memory is insufficient, triggers an elimination strategy.

3.2 Eight memory elimination strategies

The eight strategies can actually be divided into three categories.

1) No data is eliminated (default policy)

  • maxmemory-policy noeviction

When memory is insufficient, all write operations (such as SET, ADD, etc.) are rejected and an error is returned (OOM command not allowed when used memory > 'maxmemory').

Suitable for scenarios with high requirements for data integrity, avoiding data loss due to insufficient memory.

2) Select Elimination among all keys

This type of policy selects the elimination object from all keys (whether the expiration time is set or not).

  • maxmemory-policy allkeys-lru

Eliminate the least used recently（Least Recently Used）key. Select the longest unused key to eliminate based on the key's last access time (the last read or written time).

Suitable for scenarios where the access mode is not fixed and the need to retain recent active data.

  • maxmemory-policy allkeys-lfu

Eliminate the least used recently（Least Frequently Used）key. According to the access frequency of the key (read and write times), select the key with the lowest frequency to eliminate.

Suitable for scenarios where frequent access to a few hot data (such as popular product cache).

  • maxmemory-policy allkeys-random

Randomly select a key to eliminate.

It is suitable for the fairness of data elimination, and it pursues a simple and fast elimination method.

3) Select Elimination only in the key (volatile) that has the expiration time set

This type of strategy only selects the elimination object from the keys that have the expiration time set.

  • maxmemory-policy volatile-lru

Among the keys that have expired, eliminate the least recently used（Least Recently Used）key.

Suitable for keys that need to be eliminated with high probability of expiration while retaining recent active data.

  • maxmemory-policy volatile-lfu

Among the keys that have expired, eliminate the least recently used（Least Frequently Used）key.

Suitable for accessing low frequency but about to expire.

  • maxmemory-policy volatile-random

Randomly select an Elimination in the keys with the expiration time set.

It is suitable for scenarios where there is little requirement for the fairness of elimination of expired keys.

  • maxmemory-policy volatile-ttl

Prioritize the shortest keys for remaining survival time (TTL: Time To Live). If the TTLs of multiple keys are similar, it may be randomly selected.

Suitable for scenarios where you want to clean up sooner keys and avoid memory waste.

Note: The volatile-ttl strategy may cause certain keys to be phased out early due to time factors, even if they are still frequently accessed.

Business scenarios Recommended strategies
High data integrity requirements Use noeviction to avoid write failures
Access mode is not fixed Use allkeys-lru or volatile-lru
Hot data exists Use allkeys-lfu or volatile-lfu
I hope to give priority to cleaning up expired data Use volatile-ttl
Simple and quick elimination Use allkeys-random or volatile-random

Note: Redis'sLRU and LFU are not implemented accurately, but are approximate calculations through sampling and statistics., to reduce performance overhead. For example, maxmemory-policy allkeys-lru regularly samples partial keys to estimate the least recent usage.

in addition,Redis's memory elimination strategy is independent of the key expiration mechanism (lazy deletion + periodic deletion). Even if the expiration time is set, unreached keys may still be eliminated in advance due to insufficient memory.

3.3 Configuration and viewing

#Configuration format
 maxmemory <bytes> # Set maximum memory limit (such as: maxmemory 1gb)
 maxmemory-policy <policy> # Set an elimination policy (such as: maxmemory-policy allkeys-lru)
 # Dynamic configuration (restart after restart)
 CONFIG SET maxmemory 1gb
 CONFIG SET maxmemory-policy allkeys-lru
# View the current configuration item
 CONFIG GET maxmemory
 CONFIG GET maxmemory-policy

Redis's memory ed out strategy provides flexible options to weigh data retention strategies, performance overhead, and fairness based on business needs. Proper configuration of maxmemory and maxmemory-policy is the key to optimizing Redis performance and reliability.

<Previous Next>