请注意,本站并不支持低于IE8的浏览器,为了获得最佳效果,请下载最新的浏览器,推荐下载 Chrome浏览器
欢迎光临。交流群:166852192

Using ICacheManager in Orchard with Expensive Factory Code


Using the ICacheManager abstraction to cache frequently used data can significantly improve performance in your Orchard web sites. But when the work required to create that data is resource-intensive and your web site is under heavy load, bad stuff can happen. This post takes a look at how to make your caching code resilient to such circumstances.

The ICacheManager abstraction in Orchard is a very cleverly written piece of code that can be used to cache frequently used data, within as well as across requests. It supports an advanced invalidation mechanism whereby you can associate your cache entry with one or more IVolatileToken instances which can cause the cache entry to expire based on arbitrary events or conditions. There are a few built-in implementations ofIVolatileToken, such as the AbsoluteExpirationToken and FileToken, and you can also implement your own if you want to control expiration/invalidation based on something else.
Because of this nice token-based expiration mechanism, ICacheManager cannot easily be made farm-aware, and as a result, it is typically only used for data that can be considered "node local", i.e. data where neither the generation nor the expiration/invalidation need to be synchronized across farm nodes, but rather can happen on all nodes independently without negative consequences.
The API is pretty straight-forward: you call ICacheManager.Get() and provide it with a cache key and a factory function. If a cached value exists, it is immediately returned to you. If not, your factory function is called to create the data, the result is stored in the cache for subsequent callers, and then returned to you. Optionally, your factory function can also associate the data with one or more IVolatileToken instances if you want the data to expire at some point.
Let's look at a simple example:
ICacheManager _cacheManager; // Injected dependency.IClock _clock;  // Injected dependency.var someCachedValue = _cacheManager.Get("SomeKey", context =>{
    context.Monitor(_clock.When(TimeSpan.FromSeconds(30))); // Expire in 30 seconds.
    string result = null;
    // TODO: Do some expensive work to generate the result value here.
    return result;});
Here, the ICacheManager.Get() function is called with the cache key "SomeKey" and a factory lambda. The factory is provided with a context parameter, and calls context.Monitor() to associate the cache entry with a volatile token. The IClock.When() function is a convenience method that in this case takes a TimeSpan and returns an AbsoluteExpirationToken representing a point in time 30 seconds from now.
The underlying implementation of ICacheManager basically depends on an internal ConcurrentDictionaryfor storage, and does not guarantee thread safety for the factory function. In other words, if multiple threads call ICacheManager.Get() with the same cache key while the value is not in the cache, the factory function might very well be executed multiple times, and only one of the values kept.
For the most part this is fine, but in certain situations it's undesirable. For example, if the factory code is very expensive (e.g. very CPU- or database-intensive) and time-consuming to execute, and your site is under very high user load, this can lead to serious problems. Typically, the unfortunate sequence of events goes like this:
  1. Some expensive value expires from the cache.
  2. The first request after that finds the value missing and the factory function is called to generate the value.
  3. Since the generation is expensive and takes a while, and new requests come in very frequently, in the mean time the next request also finds the value missing, and it also starts generating the value. Now bothrequests are fighting for the same scarce CPU and database resources, and as a result, both will now likely take even longer to complete.
  4. More and more requests come in, and the snowball keeps growing. The more requests start generating the value, the longer they all take to complete because they compete for the same resources, and as a result even more requests start generating the value.
  5. In the best case, at least one of the requests eventually succeed in generating the value, the ones that fail do so in a graceful manner, and your site recovers (at least until the next expiration). In the worst case, your site crashes and burns.
The same problem can occur not only with sudden expiration, but also on the initial construction of the value if you already have significant user load at this point. I have seen this happen frequently when a passive node is reinserted into a running cluster when there is already heavy traffic, and the shit hits the fan running, to use a nice mixed metaphor.
So what can we do to make our code more resilient?
Well, the desired behavior is that only one thread gets to regenerate the value, while other threads block while waiting for that to be finished. Blocked threads are just waiting idle, so they do not consume the critical resources. The first thread can use any and all available resources to generate the value as quickly as possible, and all subsequent threads get it when it's done. In the grand scheme of things, all threads get the result much faster, with much less impact on system resources. Cooperative synchronization at its best - everybody wins and goes home happy.
The easiest way to accomplish this is to cache a Lazy<T> instance rather than the generated value itself, and rely on the thread synchroniation of Lazy<T> for the blocking part. Let's take a look at a revised example:
ICacheManager _cacheManager; // Injected dependency.IClock _clock;  // Injected dependency.var someCachedValue = _cacheManager.Get("SomeKey", context =>{
    context.Monitor(_clock.When(TimeSpan.FromSeconds(30))); // Expire in 30 seconds.
    return new Lazy(() =>
    {
        string result = null;
        // TODO: Do some expensive work to generate the result value here.
        return result;
    });}).Value;
Here, instead of returning the result from the factory, we are returning a Lazy<T> wrapping the factory logic. We use the Lazy<T> constructor that takes neither a LazyThreadSafetyMode mode parameter nor a bool isThreadSafe parameter, which in effect gives us a Lazy<T> instance that is fully thread safe for both execution and publication of the value, and as a result, Lazy<T> guarantees that the factory executes only once.
This simple change dramatically changes how the operation works. Now, if two threads callICacheManager.Get() simultaneously, instead of constructing two result values in parallel, we only construct two Lazy<T> instances. The Lazy<T> instance is extremely fast and cheap to construct; we can safely afford to do this multiple times and throw all but one of the instances.
When we get back the Lazy<T> instance from ICacheManager.Get() we read its Value property. At this point, ICacheManager.Get() (by virtue of its underlying ConcurrentDictionary implementation) will always return the same Lazy<T> instance. Since Lazy<T> doesn’t execute the actual factory until requested, the value is only constructed once.
Another advantage of this code compared to the first example, is that exceptions are cached by the Lazy<T>class, so we are protected against the snowball effect even if the factory code throws for some reason (for example, if it depends on an external resource that is not responding). Depending on your scenario and factory logic, this may or may not be desirable; you can control the exception caching behavior by using one of the other overloads of the Lazy<T> constructor.



作者原创内容不容易,如果觉得内容不错,请点击右侧“打赏”,赏俩给作者花花,也算是对作者付出的肯定,也可以鼓励作者原创更多更好内容。
更多详情欢迎到QQ群 166852192 交流。