Idea: Runtime Caching - continued
Chris Frazier writes: “okay, just to play devil’s advocate…>:) I think the asp.net cache framework fits very nicely for its purpose - what you propose seems like another abstraction on top of it, for the benefit of thinking *less* about what items you should cache to *optimize* code execution time? So, just throw everything possible at it. Cache can handle that, right? hehe. nope. That’s why you think about it, apply it where appropriate, and don’t waste cycles calculating if you should even cache or not as part of the caching process of a production site.”
Chris raises a good point - I definitely think it’s important we think about the code we’re writing and to decide whether to cache it. But unless we test heavily, how can we be sure we’re caching the right things? Except in the most obvious cases, the carefully considered assumptions we make while coding end up being nothing more than guesses.
How smart are we?
When it comes to setting indexes in SQL Server databases, the best advice is always to create no indexes at all, and then once you have some sample data (and a lot of it), use the SQL tuning advisor tools to tell you where to put your indexes. For the large majority of us, the SQL tools are going to do a far better job than we could at figuring out where indexes should be placed.
In my Runtime Caching post earlier, the idea I had was to decorate methods with an attribute to indicate that the methods were “deterministic” - that is, if you called them over and over with the same inputs, they’d return the same outputs.
This attribute isn’t intended to force caching, but to tell the runtime that it can treat the method a little differently. I wrote that the runtime might set timers around such method calls, and then by examining things such as garbage collection allocation, the time the method call took, etc., it could figure out whether or not it was worth caching the result. While your application is running, the runtime could figure out whether certain things were worth putting in the cache.
An alternative…
But let’s say - as a few people raised - you don’t want to wear the overhead of measuring the execution times at runtime. In that case, perhaps the runtime could act a little differently?
A different model might be to use some kind of .NET profiler, which hooks into the runtime and looks for the [Deterministic] attribute. When the application is run over test data, the runtime would perform the same calculations, but instead of applying the calculations in real-time, it could generate a list of places where you should put caching code - much like the SQL Server profiling tools. You’ve already told the runtime where possible caching could be done - the tools would just be telling you whether you were right or wrong.
Like functional languages, LINQ, Excel and T-SQL, this is just another example of telling the runtime what to do (or, what can be done), rather than telling it how to do it. Just as in 99.99% of cases the garbage collector is going to know better than you whether it’s time to clean up memory, I bet that the runtime could know better than us whether something is worth caching. Whether it can be implemented or not remains to be seen 
Technorati tags: chris frazier, runtime caching, deterministic
Filed under: C#
As mentioned in the earlier comments, this could be done using AoP. In fact, it looks like the Policy Injection Application Block comes with some of the bits you need to do it [1].
The extra bits that would be required would probably be a version of the caching handler that included the profiling bits, and policies that use the caching handler when the right attributes are applied (and the profiling output says so).
[1] http://blogs.msdn.com/tomholl/archive/2007/02/23/announcing-the-policy-injection-application-block.aspx
hehe good sport that Paul Stovell XD
[…] Link to Article c# Runtime Caching - continued » Posted at PaulStovell.NET on Wednesday, July […]
Don’t forget another option, which is to push the caching work and knowledge into an appliance. That’s what a company called StrangeLoop is doing. They’ll have their first shipping boxes in about a month, and one of the initial features will intelligently detect if certain requests *always* get the same response, and applying caching to such requests (before the request even reaches the web server). Some slick stuff with ViewState elimination, too, that saves a lot on bandwidth overhead. Check it out: http://www.strangeloopnetworks.com/ (no i don’t work for them)
while kind of cute, i think this feature would lead to more harm then good.
for any non-trivial operations you’re going to need to be looking hard to see if your function will return the same thing to the same inputs.
i.e. if you call another method, (like .ToString()) say, is _that_ Deterministic? will that always return the same value for the same input? (does it have that flag?) no. should you only be able to call methods that have that same flag from a flagged method? maybe. but that’ll limit you to a seriously small set of operations.
but also, this flag should force the method to be final; if it could be overridden, you’re in a serious amount of trouble.
anyway, isn’t this the sort of stuff the hotspot (in java; i can’t remember the equiv .net name) compiler should be doing anyway?
I just noticed this in the spring docs:
http://www.springframework.net/doc/reference/html/aop-quickstart.html#aop-quickstart-cookbook-caching