Redis Data Structures and Persistence Mechanisms
Data Structures
string uses SDS Records three fields: len, free, buf[] Characteristics: space pre-allocation, lazy release Advantages: constant time length retrieval, prevents buffer overflow
dict uses hash table Rehash dynamic amortized expansion, maintains two hash tables simultaneously
zset ordered linked list Underlying structure is skip list
ziplist compressed list Is the underlying implementation for list keys and hash keys When there are fewer elements, priority is given to storing in ziplist, converting to standard storage structure when exceeding certain values
Expiration
expire scheduled deletion, deletes expired keys through periodic traversal Scans ten times per second, randomly selects 20 keys from the expired dictionary for deletion. If expired keys account for more than one quarter of all keys, repeat this process
Persistence Mechanisms
RDB Periodic Persistence
Generates a complete snapshot of Redis memory data at regular intervals RDB disadvantages
- Cannot persist at second-level granularity
- Older versions of Redis cannot be compatible with new version RDB RDB advantages
- Compact files, suitable for backup and full replication scenarios. For example, execute bgsave every 6 hours, save to file system, etc.
- Redis loads RDB to restore data much faster than AOF
AOF
Writes each command as a log to the aof file When AOF persistence is enabled, the commonly used disk synchronization strategy is “every second sync” everysec, to balance performance and data security. For this approach, Redis uses another thread to execute fsync to sync to disk every second. When system resources are busy, it will cause the Redis main thread to block
Single Machine Multi-Instance Deployment
Redis single-threaded architecture cannot fully utilize multi-core CPU. The usual practice is to deploy multiple instances on one machine. When multiple instances enable AOF, competition for CPU and IO will occur between them.
How to solve this problem?
Execute all instances’ AOF serially. We write shell scripts based on AOF information from info Persistence, then execute instance AOF persistence serially. By continuously checking AOF status and manually executing AOF rewrite, ensure AOF does not have competition.