Cache Penetration and singleflight

For optimizing high-concurrency request scenarios and alleviating service pressure, singleflight is a valuable tool library. It can resolve high-concurrency request issues with the same parameters, including cache penetration, and is worth promoting in projects.

What is Cache Penetration

Cache penetration refers to the cache being "penetrated," with the actual pressure falling on the database (DB).

When discussing cache penetration, there are actually three related scenarios concerning caches and databases: cache penetration, cache leakage, and cache avalanche.

Cache Penetration

Problem Scenario

In high-concurrency situations, when popular cached data expires, numerous requests fail to find the data in the cache and directly request data from the DB, leading to a sudden surge in DB pressure.

Solution

Cache penetration generally does not lead to DB downtime but can cause periodic pressure on the DB. Since the issue arises from the expiration of hot data, the solution is to prevent it from expiring by setting it to never expire or by implementing a renewal mechanism.

When setting data to never expire, it is important to monitor changes in data access patterns, such as when a "hot key" becomes a "cold key." For example, a configuration that was previously accessed frequently might become less so over time, transitioning from high to low concurrency. In such cases, it is not suitable to keep the data in the cache permanently. Business-driven designs can incorporate manual clearance or introduce Least Recently Used (LRU) algorithms for eviction.

For caches that are meant to be permanently valid, the update mechanism must also be designed. Simply put, updates and reads must be mutually exclusive. In distributed systems, distributed locks are necessary. You can choose between synchronous or asynchronous updates. Synchronous updates use mutual exclusion to block resources during updates, which may delay read operations; asynchronous updates involve periodic cache refreshes, which might result in reading stale data.

Cache Leakage

Problem Scenario

Cache leakage often occurs in attack scenarios. For instance, while normal business IDs are greater than zero, attackers may attempt requests with an ID of -1. When the corresponding data does not exist in either the cache or the DB, these requests can put significant pressure on the DB.

Solution

Cache leakage happens because the DB lacks the requested data. To address this, one solution is to validate requests within the business logic, rejecting those with IDs less than or equal to zero to prevent direct pressure on the DB. Another solution is to create a cache entry for non-existent data and set an expiration time, such as 60 seconds, effectively mitigating the short-term pressure on the DB caused by cache leakage.

Cache Avalanche

Problem Scenario

Cache avalanche occurs when a large amount of cached data expires simultaneously, leading to numerous cache penetrations and a flood of data requests hitting the DB, potentially causing excessive DB pressure or downtime.

Solution

One solution to cache avalanche is to reduce the occurrence of simultaneous expirations by analyzing business scenarios and smoothing the peak expiration times. Another is to distinguish between hot and cold data, similar to handling cache penetration, and applying permanent caching plus updates for hot data.

The cache issues and solutions discussed are all targeted at the cache data layer. For scenarios involving cache penetration, in addition to directly handling cache data, it is also possible to apply caching logic at the business logic layer. This is where the singleflight library, developed based on the Go language, comes into play.

singleflight Implementation Mechanism

The principle of singleflight is quite simple, with its core structure as follows:

// Group represents a class of work and forms a namespace in
// which units of work can be executed with duplicate suppression.
type Group struct {
    mu sync.Mutex       // protects m
    m  map[string]*call // lazily initialized
}

The map[string]*call represents a cache list for requests of the same kind, with the key being the same. When a business request arrives, if there is already a similar request being processed, it will wait for the result instead of executing the business logic directly. singleflight addresses a broader range of issues than just cache penetration, not limited to data access.

Summary

For optimizing high-concurrency request scenarios and alleviating service pressure, singleflight is a valuable tool library. It can resolve high-concurrency request issues with the same parameters, including cache penetration, and is worth promoting in projects.