Spring Boot Caching

In a microservices environment caching is an important none functional requirement which can help reduce load overhead, latency and improve the user experience. Where ever data is loaded, particularly from an HTTP endpoint on another service, caching should be considered. In Spring Boot it is easy to implement caching on any function and the cache itself can be implemented independently of the service in which it is applied.

When to Cache?

In a microservice environment multiple calls are made to and from different services within the context of a single request to a gateway. In a 'database per service' architecture the same data is retrieved from the same end point multiple times throughout the course of the initial request. Where this is done in the same thread it might be tempting to wrap the data and pass it down the chain. However, this approach breaks the single responsibility rule and restricts the extensibility of our independent services. Caching is a convenient answer but introduces potential problems. 

The amount of data cached, length of time it is cached for and amount of times it is used are all  variables which must be traded off for the price of improved performance. If we cache data for too long or don't refresh it often enough then we run the risk of error or even perceived corruption within the system. Conversely, if we don't cache long enough or refresh too often, we don't get any benefit from the cache.

Let's first consider how to enable caching in the Spring Boot application and introduce a simple cache to improve the performance of the system. In this example an application uses a Feign client to retrieve the same data multiple times from an endpoint on another service via HTTP. These are expensive and caching will clearly help improve performance.

Enable Caching in the Application

To enable caching we need to bring in the relevant dependencies and annotate a class loaded into the Spring container. As usual Spring Boot provides a convenient starter.

Add the enable caching annotation on the initialising class with the main method.

Caching the Data

The @Cacheable annotation tells Spring to cache the data returned by the method against a generated key on the first request and then takes that data from the cache whenever the same parameters are used. In this case a repository method uses a Feign client to call the endpoint. It is this method which we annotate and tell Spring to cache the data it returns against a key generated from the method parameters.

So far we've introduced the concept of caching to the system and told Spring to cache beans returned by the annotated method in the cache called 'mycache'. Now we need to implement and configure that cache. By default Spring will implement a simple in memory Map via the default CacheManager bean. We can override that by defining our own CacheManager implementation in some custom configuration. Spring provides many different manager implementations as hooks for implementing different well known cache providers, such as Ehcache.

Implementing a Cache

In the above example we have told the application to cache data returned from a method the first time it is called with the given parameter and then retrieve that from the cache each time that parameter is used after that. Currently this doesn't solve any of the trade offs mentioned earlier. We only want the same data to be kept in the cache for the duration of time it will be used for. If that data changes after a period of time we want to ensure the call to the method refreshes whats in the cache with data from the actual service that provides it. To solve this we must implement some constraints on the cache to ensure we refresh the data when required and prevent the cache growing too large which could cause resource issues.

Configuration for Ehcache requires us to introduce a config xml file to the classpath. If you're not a fan of this and prefer code configuration, as is the Spring Boot standard then a handy SourceForge library provides some alchemy for wrapping the Ehcache config into a Spring CacheManager.

The max entries, TTL and TTI values are loaded from our applicaiton properties/yml file or from a config server if we're using one. Max entries limits the number of objects in the cache and a 'Least Recently Used' policy manages eviction behind the scenes. Time To Live limits the maximum amount of time objects can reside in the cache. Time to Idle limits the time an object can reside in the cache without being used and should be set to a lower value than TTL. Both these values are critical to the success of the cache. Set them too low and we reduce the effectiveness of caching. Set too high and we run the risk of using data which isn't fresh and causing unintentional error downstream. This trade off can only be optimised through performance testing.

Cache Keys

Along with other configuration we can create a customised KeyGenerator bean. By default Spring uses a SimpleKeyGenerator class which either uses the hashCode of the a single parameter or combines multiple parameters to store the objects against in the cache. If you're using some kind of correlation Id to uniquely identify user request to the gateway then it might be usesful to pass this into the method as a parameter or create a custom KeyGenerator bean to key data so that we ensure different user requests and threads don't use the same cached data. However, it really does depend on your use case.

Testing 

It might be tempting to test that caching is working by using a Mock framework to mock the feign client. I've found this doesn't work because the Mock proxy intercepts the method call before the Cachable interceptor. I used Wiremock to stub the client call instead and verified the number of calls made doesn't increase after the first method request. We can also autowire the CacheManager into the test to access the cache and test its contents.