Location>code7788 >text

Visualization Exploration and Practice of Caffeine, a Caching Framework

Popularity：100 ℃/2024-07-26 14:41:40

Author: Vivo Internet Server Team- Wang Zhi

Caffeine is heavily used as a high-performance caching framework. In this paper, we implement the visualization function based on the existing foundation of Caffeine for custom development.

I. Background

The Caffeine cache is a high-performance, scalable, memory-optimized Java cache library that evolved from Google's Guava Cache and provides near-optimal hit rates.

The Caffeine cache contains the followingspecificities：

  1. Efficient and fast: The Caffeine cache uses optimization techniques such as approximation algorithms and concurrent hash tables to make cache access very fast.

  2. Memory friendly: The Caffeine cache uses a memory optimization strategy that dynamically adjusts the size of the cache as needed to efficiently utilize memory resources.

  3. Multiple caching strategies: Caffeine cache supports multiple caching policies, such as capacity-based, time-based, weight-based, manual removal, timed refresh, etc., and provides rich configuration options that can be adapted to different application scenarios and requirements.

  4. Support for asynchronous loading and refreshing: The Caffeine cache supports asynchronous loading and refreshing of cache items, and integrates seamlessly with frameworks such as Spring.

  5. Clearance strategy: Caffeine uses the Window TinyLFU cleanup strategy, which provides near-optimal hit rates.

  6. Support for auto-loading and auto-expiration: The Caffeine cache can automatically load and expire cache entries based on configuration without manual intervention.

  7. statistical function: The Caffeine cache provides rich statistical functions such as cache hit rate, number of cache items, etc., which facilitates the evaluation of the performance and effectiveness of the cache.

It is because of the above features that Caffeine has, Caffeine as a local cache in the project of choice, more and more projects integrated Caffeine's functions, which in turn derived a series of business perspective requirements.

One of the requirements for daily use wants to be able to evaluate the memory usage of Caffeine instances in real time and provide the ability to dynamically adjust cache parameters, but the existing memory analysis tool MAT needs to analyze the files based on dump can't do it in real time, which is one of the causes of the whole thing.

II. Technical perspective of operations

  • Caffeine's cache instances in the project can achieve near real-time statistics, real-time view of the number of cached instances.

  • Caffeine's cache configuration parameters, memory usage, and cache hit rate can be viewed in real time for each instance, and the cache expiration time and cache entries of a single instance can be dynamically configured and sent.

  • Caffeine's cached data can be viewed in real time for each instance, and can support the immediate invalidation of cached data and other functions.

Based on the above requirement background, combined with the existing functions of caffeine and customized part of the source code development, the whole as a caffeine visualization technology project to promote and land.

III. Visualization capacity

The Caffeine visualization project is currentlySupported FeaturesIncluded:

  • Control of global cache instances in the project dimension.

  • Single cache instance configuration information visualization, memory usage visualization, and hit rate visualization.

  • Functions such as data query, dynamic change of configuration, and invalidation of cached data for a single cache instance.

3.1 Global control of cache instances

图片

Description:

  • Show the cache instance objects contained under this application in application dimension + machine dimension, each instance contains information such as size, expiration policy, expiration time, memory usage, cache hit rate, etc. in cache settings.

  • Memory footprint and cache hit rate for single instance dimension are supported to be displayed as trend graphs.

  • The single instance dimension supports configuration change operations and cache query operations.

3.2 Trends in memory usage

图片

Description:

  • Memory Usage Trend records the trend change of memory usage of this cache instance object in recent period.

  • The time period currently supports displaying data from the last two days.

3.3 Trends in hit rates

图片

Description:

  • Hit Trend records the change in cache hits for this cache instance object over a recent period of time.

  • The time period currently supports displaying data from the last two days.

3.4 Configuration changes

图片

Description:

  • Configuration changes currently support dynamic settings for cache size and expiration time.

  • Currently, single-instance settings are supported for the time being, and the full-volume effect feature will be supported in the future.

3.5 Cache queries

图片

Description:

  • The single-instance dimension supports queries for cached data.

  • Currently supports common cache Key types including String type, Long type, Int type.

IV. Principle realization

4.1 Overall design framework

  • Caffeine Framework Feature Integration

图片

Description:

  • The basic functionality along the lines of Caffeine includes Caffeine's caching functionality and Caffeine's statistical functionality.

  • additionalCaffeine Memory Occupancy Estimation feature, which focuses on estimating the memory occupied by cached instance objects.

  • additionalThe Caffeine instance naming feature, which provides naming capabilities for each instance object, is the basis for global control.

  • additionalCaffeine instance global control function, this function mainly maintains all the cache instances in the project running.

Caffeine Visualization Framework

图片

Description:

  • [Project Engineering Side]: Caffeine's visualization framework is based on the integration of Caffeine framework functionality based on the addition of a communication layer for data data reporting and configuration of the downstream.

  • [Control platform side]: Responsible for receiving and displaying cached data reports and issuing configuration change commands.

  • [The communication layer supports push and pull modes], push mode is mainly used for real-time reporting of statistical data, and pull mode is mainly used for configuration sending and caching data query.

4.2 Source code implementation

Business Layer - Management of Cached Objects

static Cache<String, List<String>> accountWhiteCache = ()
            .expireAfterWrite(("", 10), )
            .recordStats().maximumSize(("", 100)).build();
routineCaffeineHow instances are created
 
 
static Cache<String, List<String>> accountWhiteCache = ().applyName("accountWhiteCache")
            .expireAfterWrite(("", 10), )
            .recordStats().maximumSize(("", 100)).build();
support for instance-namedCaffeineHow instances are created

Description:

  • Added cache instance naming functionality on top of Caffeine instance creation by defining cache instance naming via .applyName("accountWhiteCache").

public final class Caffeine<K, V> {
 
  /**
   * caffeineThe instance name of the
   */
  String instanceName;
 
  /**
   * caffeineinstances maintained by theMaptext
   */
  static Map<String, Cache> cacheInstanceMap = new ConcurrentHashMap<>();
 
  @NonNull
  public <K1 extends K, V1 extends V> Cache<K1, V1> build() {
    requireWeightWithWeigher();
    requireNonLoadingCache();
 
    @SuppressWarnings("unchecked")
    Caffeine<K1, V1> self = (Caffeine<K1, V1>) this;
    Cache localCache = isBounded() ? new <>(self) : new <>(self);
 
    if (null != localCache && (())) {
      ((), localCache);
    }
 
    return localCache;
  }
}

Description:

  • Each Caffeine has an instance name instanceName.

  • The mapping relationship between the name of the Caffeine instance object and the instance is maintained globally via cacheInstanceMap.

  • Maintaining a mapping relationship enables you to query a cached instance object by its name and perform various operations on the cached instance object.

  • The naming functionality of Caffeine instances is the cornerstone for the integration of other features.

Business Layer - Memory Usage Estimation

import ;
 
public abstract class BoundedLocalCache<K, V> extends <K, V>
    implements LocalCache<K, V> {
 
  final ConcurrentHashMap<Object, Node<K, V>> data;
 
  @Override
  public long getMemoryUsed() {
    // Estimated Memory Usage
    return (data);
  }
}

Description:

  • By predicting the cache value of the memory.

  • The data value is the object used by the Caffeine instance to hold the real data.

Business layer -- data reporting mechanism

public static StatsData getCacheStats(String instanceName) {
 
    Cache cache = (instanceName);
 
    CacheStats cacheStats = ();
    StatsData statsData = new StatsData();
 
    (instanceName);
    (()/1000);
    ((()));
    ((()));
    ((()));
    ((()));
    ((()));
    ((()));
    ((()));
    ((()));
    ((()));
    ((()));
    ((()));
 
    Optional<Eviction> optionalEviction = ().eviction();
    (eviction -> ((())));
 
    Optional<Expiration> optionalExpiration = ().expireAfterWrite();
    (expiration -> ((())));
 
    optionalExpiration = ().expireAfterAccess();
    (expiration -> ((())));
 
    optionalExpiration = ().refreshAfterWrite();
    (expiration -> ((())));
 
    return statsData;
}

Description:

  • The relevant values are counted through Caffeine's own statistics interface.

  • Statistical data example dimension for statistics.

public static void sendReportData() {
 
    try {
        if (!("", true)) {
            return;
        }
 
        // 1、Get all thecacheinstance object
        Method listCacheInstanceMethod = HANDLER_MANAGER_CLASS.getMethod("listCacheInstance", null);
        List<String> instanceNames = (List)(null, null);
        if ((instanceNames)) {
            return;
        }
 
        String appName = ("");
        String localIp = getLocalIp();
        String localPort = (());
        ReportData reportData = new ReportData();
        InstanceData instanceData = new InstanceData();
        (appName);
        (localIp);
        (localPort);
 
        // 2、(math.) ergodiccacheinstance object获取缓存监控数据
        Method getCacheStatsMethod = HANDLER_MANAGER_CLASS.getMethod("getCacheStats", );
        Map<String, StatsData> statsDataMap = new HashMap<>();
        ().forEach(instanceName -> {
 
            try {
                StatsData statsData = (StatsData)(null, instanceName);
 
                (instanceName, statsData);
            } catch (Exception e) {
 
            }
        });
 
        // 3、Constructing Reporting Objects
        (instanceData);
        (statsDataMap);
 
        // 4、dispatchHttp(used form a nominal expression)POSTrequesting
        HttpPost httpPost = new HttpPost(getReportDataUrl());
        (requestConfig);
 
        StringEntity stringEntity = new StringEntity((reportData));
        ("application/json");
        (stringEntity);
 
        HttpResponse response = (httpPost);
        String result = ((),"UTF-8");
        (());
 
        ("Caffeine Successful data reporting URL {} parameters {} in the end {}", getReportDataUrl(), (reportData), result);
    } catch (Throwable throwable) {
        ("Caffeine Data reporting failure URL {} ", getReportDataUrl(), throwable);
    }
}

Description:

  • Statistics are collected by getting all the Caffeine instances running in the project and traversing them sequentially.

  • It is responsible for reporting the corresponding statistics via the http protocol, using a fixed-interval period.

Business Layer-Configuration Dynamic Dispatch

public static ExecutionResponse dispose(ExecutionRequest request) {
    ExecutionResponse executionResponse = new ExecutionResponse();
    (CmdTypeEnum.INSTANCE_CONFIGURE.getCmd());
    (());
 
    String instanceName = ();
    Cache cache = (instanceName);
 
    // Setting the maximum number of entries in the cache
    if (null != () && () > 0) {
        Optional<Eviction> optionalEviction = ().eviction();
        (eviction ->(()));
    }
 
    // Setting the expiration time for expire-on-write
    if (null != () && () > 0) {
        Optional<Expiration> optionalExpiration = ().expireAfterWrite();
        (expiration -> ((), ));
    }
 
    // Setting the expiration time for access expiration
    if (null != () && () > 0) {
        Optional<Expiration> optionalExpiration = ().expireAfterAccess();
        (expiration -> ((), ));
    }
 
    // Setting the expiration time for write updates
    if (null != () && () > 0) {
 
        Optional<Expiration> optionalExpiration = ().refreshAfterWrite();
        (expiration -> ((), ));
    }
 
    (0);
    ("success");
 
    return executionResponse;
}

Description:

  • The settings related to cache configuration are performed through Caffeine's own interface.

Business Layer - Cache Data Emptying

/**
     * The value of the invalidation cache
     * @param request
     * @return
     */
    public static ExecutionResponse invalidate(ExecutionRequest request) {
 
        ExecutionResponse executionResponse = new ExecutionResponse();
        (CmdTypeEnum.INSTANCE_INVALIDATE.getCmd());
        (());
 
        try {
            // Find the correspondingcachean actual example
            String instanceName = ();
            Cache cache = (instanceName);
 
            // 处理清空指定an actual example的所有缓存 maybe 指定an actual example的keyCorresponding Cache
            Object cacheKeyObj = ();
 
            // Clear all caches
            if ((cacheKeyObj)) {
                ();
            } else {
                // Clear DesignationkeyCorresponding Cache
                if (((), 2)) {
                    ((().toString()));
                } else if (((), 3)) {
                    ((().toString()));
                } else {
                    (().toString());
                }
            }
 
            (0);
            ("success");
        } catch (Exception e) {
            (-1);
            ("fail");
        }
 
        return executionResponse;
    }
}

Business Layer - Cached Data Queries

public static ExecutionResponse inspect(ExecutionRequest request) {
 
    ExecutionResponse executionResponse = new ExecutionResponse();
    (CmdTypeEnum.INSTANCE_INSPECT.getCmd());
    (());
 
    String instanceName = ();
    Cache cache = (instanceName);
 
    Object cacheValue = (());
    if (((), 2)) {
        cacheValue = ((().toString()));
    } else if (((), 3)) {
        cacheValue = ((().toString()));
    } else {
        cacheValue = (().toString());
    }
 
    if ((cacheValue)) {
        ("");
    } else {
        ((cacheValue));
    }
 
    return executionResponse;
}

Description:

  • Cache information is queried through Caffeine's own interface.

Communication layer -- listening service

public class ServerManager {
 
    private Server jetty;
 
    /**
     * establishjettyboyfriend
     * @throws Exception
     */
    public ServerManager() throws Exception {
 
        int port = ();
 
        jetty = new Server(port);
 
        ServletContextHandler context = new ServletContextHandler(ServletContextHandler.NO_SESSIONS);
        ("/");
        (, "/caffeine");
        (context);
    }
 
    /**
     * activate (a plan)jettyboyfriend
     * @throws Exception
     */
    public void start() throws Exception {
        ();
    }
}
 
 
public class ClientServlet extends HttpServlet {
 
    private static final Logger logger = ();
 
    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        (req, resp);
    }
 
    @Override
    protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
 
        ExecutionResponse executionResponse = null;
        String requestJson = null;
        try {
            // Get the parameters of the request
            String contextPath = ();
            String servletPath = ();
            String requestUri = ();
            requestJson = ((), StandardCharsets.UTF_8);
 
            // Handling of different commands
            ExecutionRequest executionRequest = (requestJson, );
 
            // Dealing with class dependencies through reflection
            executionResponse = (executionRequest);
 
        } catch (Exception e) {
            ("vivo-memory Handling request exceptions {} ", requestJson, e);
        }
 
        if (null == executionResponse) {
            executionResponse = new ExecutionResponse();
            (-1);
            ("Handling of exceptions");
        }
 
        // Assemble the appropriate messages
        ("application/json; charset=utf-8");
        PrintWriter out = ();
        ((executionResponse));
        ();
    }
}

Description:

  • The communication layer starts the http service through jetty to listen, and the port is not open to the public for security reasons.

  • By defining a ClientServlet to handle related requests including functions such as configuration issuance and cache lookup.

Communication Layer - Heartbeat Design

/**
 * Send heartbeat data
 */
public static void sendHeartBeatData() {
 
    try {
 
        if (!("", true)) {
            return;
        }
 
        // 1、Building Heartbeat Data
        String appName = ("");
        String localIp = getLocalIp();
        String localPort = (());
 
        HeartBeatData heartBeatData = new HeartBeatData();
        (appName);
        (localIp);
        (localPort);
        (()/1000);
 
        // 2、dispatchHttp(used form a nominal expression)POSTrequesting
        HttpPost httpPost = new HttpPost(getHeartBeatUrl());
        (requestConfig);
 
        StringEntity stringEntity = new StringEntity((heartBeatData));
        ("application/json");
        (stringEntity);
 
        HttpResponse response = (httpPost);
        String result = ((),"UTF-8");
        (());
 
        ("Caffeine Heartbeat reported successful. URL {} parameters {} in the end {}", getHeartBeatUrl(), (heartBeatData), result);
    } catch (Throwable throwable) {
        ("Caffeine Failure to report heartbeat URL {} ", getHeartBeatUrl(), throwable);
    }
}

Description:

  • The heartbeat feature reports the ip and port of the project instance used for communication and carries a timestamp used to record the reporting timestamp.

  • The actual project because of the machine recycling and other scenarios need to be reported through the timestamp timed to clean up the offline service.

V. Summary

The Vivo technical team has shared their experience in using Caffeine several times, you can refer to the public article "How to use Caffeine Cache as smooth as silk", this article is based on the use of further customization based on the use of pain points.

At present, the project of Caffeine visualization has been landed and played a role in the relevant core business scenarios, and the overall operation is smooth. The more frequently used functions include global control of caffeine instances in the project dimension, memory consumption evaluation and cache hit trend evaluation in the single-instance dimension.

For example, the memory consumption evaluation function of a single instance can reasonably evaluate the relationship between cache entry settings and memory consumption; analyze the overall trend of cache hit rate to evaluate the reasonableness of cache parameter settings.

I look forward to this article bringing some new ideas to the industry on cache usage and monitoring.

<Previous Next>