The New .NET 9 HybridCache That You Must Upgrade To!
HTML-код
- Опубликовано: 26 сен 2024
- Use code GRAPHQL20 at checkout to get 20% off our new Getting Started with GraphQL in .NET course: bit.ly/3KaOI8z
Become a Patreon and get special perks: / nickchapsas
Hello, everybody. I'm Nick, and in this video, I will show you the brand new type of cache that was added to .NET 9 called HybridCache. This cache is supposed to replace both the in-memory cache and the distributed cache and fix all the problems with each one of those approaches.
Workshops: bit.ly/nickwor...
Don't forget to comment, like and subscribe :)
Social Media:
Follow me on GitHub: github.com/Elf...
Follow me on Twitter: / nickchapsas
Connect on LinkedIn: / nick-chapsas
Keep coding merch: keepcoding.shop
#csharp #dotnet
I like it when they add things to .Net that ARE actually useful instead of just pretty.
we've spent quite some time to kind of build those features on top of IDistributedCache so I am glad it is now a built in feature. no idea why it is a class tough...
Looks great, but I _really_ dislike the magic registration, when HybridCache detects automatically what type of distributed cache is registered. If you know, sure, easy, however if you find yourself in a project without that knowledge, there is "no breadcrumb" to lead you on to what is really happening. Feels like unnecessary bar rising for less experienced developers or less lucky to be around when this feature is being advertised...
Still, looking forward to take it for a spin :)
This is the best ever thing Microsoft has released in the past 2 years. Only if this was released like 6 months ago when I needed something similar. Was a pain in the ... to build this kind functionality
This is pretty cool to have out the box, but also pretty simple to implement yourself if you are not ready to dive into .NET9 just yet.
The real challenge is this. In the scenario where you want to retrieve something that has expired, and the API you are calling also fails for some reason, or if the API you are calling is slow for some reason, now your user suffers as a result. The only way I have resolved this, is to have a background worker that updates the cache on a regular basis.
It's cloudy, what a surprise
That's actually quite nice. I think that's going to be a very useful tool in the future to consider.
FINALLY THEY ADDED A TAG TO CACHE ITEMS, GOD BLESS YOU MICROSOFT
caveat: tag-based eviction didn't make it into preview 4 (or 5); we have the design etc, and are working on it
Great video Nick!
With Hybrid Cache you also use your own custom serializers which means that one does not have to rely only on standard system text json or newtonsoft json providers. 🚀
They should just bring over fusion cache. Tagging is a nice feature.
As I understand it doesn't lock across multiple instances. In case of multiple instances making the same call all of the instances will make the same call to the weather API. So if you have 10 instances of the same service and a very loaded environment there still might be several simultaneous calls.
Was going to ask the same but I understand why, we did implement something with distributed locking also and it was quite complex and requires a real distributed locking solution, which redis does not really have out of the box and then you have to make a lot of more decisions on exactly where your prepared to take a penalty, redundant external calls or slower updates due to failed calls and retries.
And in most cases you only have a few instances and 3-4 extra calls once every 5 minutes or so is not going to be a big problem.
If it is, then you have to bite the bullet and roll your own.
And if you have som many instances that that becomes a problem, then you most likely already have some distributed locking :)
Ok, HybridCache seems like a good addition to the collection of cache implementations.
But one thing that I don't like about it is that it don't have a factory solution where I can specify a cache create / refresh method centrally. Much like the IConfiguration provider solution.
The problem arises when I want to reuse the same cached value in multiple places. I then have to specify the factory in multiple places...
Awesome feature and video, thank you! The only downside that it's in .NET 9 and not earlier :)
IMHO: HybridCache looks like the way moving forward, but please also mark IMemoryCache and IDistributedCache [Obsolete] and whatever jargon made into BCL with .NET 9 Release and completely remove both of these APIs in .NET 10. The BCL Team may consider using the [Obsolete] attribute more aggressively between alternating .NET Releases - keep the BCL clean and tidy. PLEASE!!!
Well, is it actually obsolete? Perhaps the HybridCache uses both those interfaces behind the scenes.
@@joost00719 It literally does; IDistributedCache is the out-of-process L2 backend for HybridCache, and IMemoryCache is the in-process L1 endpoint; source: me
@@joost00719 It is actually using them
making it thread safe is really nice, one thing I'm concerned about here is bugs introduced by the duality usage option. for example if somehow you didn't register the redis cache you won't know about it until the memory leak start to happen or cache miss in case of multiple instances. it would be nice if there is a way to make it a requirement for redis cache to be registered in some cases. I know you can do this manually but it would be nice if it can be implemented in the options setup in AddHybridCache directly.
Implicitly autowiring objects like these is the reason I dislike working with SpringBoot.
Although it's worse there because everyone fully leans into it and you have something 20 layers of nested beans all automatically checking if others have been registered and changing their behavior based on that. It's a nightmare to know exactly what's going on in your application.
Also god forbid you wanna change something, because you'll be overriding 5 different abstract classes.
Does the HybridCache support just returning the last cached value while the cache is being updated? That behavior is really handy for performance critical services like whisperers where you don't want to unnecessarily block multiple clients when the cache is stale.
That behaviour is called "stale-while-revalidate". I wish it were the default (or at least a built-in option) for more caching frameworks.
Hi; this is on the roadmap, but is unlikely to ship in time for .NET 9; since HybridCache ships OOB, we have some options to expand this between releases
source: me (I'm the lead for Hybrid Cache)
@@MarcGravell let's get a move on it, chop chop... Jp
I think Microsoft has a long way to go with Hybrid caches. But yet, its a very nice start. Basically, what hybrid cache is, it will leverage both in-memory and distributed cache at the same time but in a way that the request does not have to travel to the distributed cache in order to reduce latency.
The idea is, at the time of storing a record, it will store the data into the in-memory cache and send a message asynchronously to any kind of bus (i.e. redis bus) indicating that a WeatherCached event has occurred. And on the other side, the same instance will subscribe to that event and store that data into the distributed cache. if other instances of the same app/different microservice/a different module that is consuming that same event from the same channel will store it in their own in-memory cache. this way Hybrid Caching helps us leveraging both in-memory cache and distributed cache at the same time and the incoming requests to web/api server do not need to travel over the network to get the data because its already available in-memory.
Nice feature! I am currently using the NuGet package LazyCache instead of the plain memory cache, primarily for the Stampede-protection but also for a feature that immediatly evicts items from the cache on expiration. Is that something HybridCache supports?
This is amazing, just what I needed!
Can you clarify? If a different request for weather comes in... does it still block? Or is the blocking purely based upon the cache key?
One more solution with implicit dependency.
There is no declaration that says - HybridCache functionality depends on Redis. MS need to change this as it done in many cases already.
I agree. Avoiding lots of setup code is nice, but it should not be abstracted to the point where things become magic, and you just have to "know" about it. Would have preferred if there was an explicit statement "use Redis" while setting up HybridCache.
because HybridCache *does not* depend on Redis - it optionally uses IDistributedCache as an L2 backend; HybridCache will work with any implementation - Redis, SQL Server, NCache, CosmosDB, etc
source: me
I was dealing with this problem last week. Excited for .net 10 when I can use it lol
FYI, the library LazyCache does the same as HybridCache. Better to use HybridCache going forward but wanted to set the record straight that this functionality has been available for years.
Cache stampede protection? Yes.
Multi level? Nope.
1) Synchronization of requests for any memory cache can be handled easily by inserting a Lazy.
The Lazy minimizes (or even eliminates) the chance of opportunistic concurrency, and the Task is the actual request that gets made and shared.
Sure, you have to do some smart eviction if either of those fail, but IMO, I think this kind of operation should have be built in to (or provided as an extension) any memory cache.
2) I would always have a short term expiry in memory cache for anything that is cache-able. Ideally tuned with a short term sliding expiry that still honors an absolute. Then of course, the distributed cache API would be the next layer that the in memory cache calls to.
3) Either way, each cache layer should have an option to use pessimistic concurrency.
AKA "stampede protection" should be an option for both, and IMO, its should be the default.
1) Sure, could work, but only in a GetOrAdd scenario, where immediate invocation of the Lazy is guaranteed.
2) Yes, we have that, with an option to have in memory cache invalidation using Redis PubSub.
Since IMemoryCache came out I've been wishing they just had an ICache type with MemoryCache and DistributedCache implementations (and now HybridCache). Would have been a lot cleaner and simpler that way.
Did they just embrace, extend and extinguish FusionCache? 😂
The maintainer of FusionCache is actively participating in the design discussions regarding HybridCache.
@@HHJ79 That's great to know - This hasn't always happened with previous things embedded in the Framework. But in that case I'm very happy to see it embraced.
FusionCache creator here: eh, I hear you, you’re not the first one to tell me that 😅
I’m sharing my experiences and design ideas with the team, and they are listening! I even suggested the name HybridCache, the previous name was not that clear imho (naming is hard).
I hope the two will be able to work together in the end, with HybridCache as a potential shared abstraction.
@@jodydonetti Awesome to hear you are working with MS on this. The work you've done on FusionCache is incredible!
@@Sam-ey1nnto be fair I’m not actively “working” on it, the team is doing the work and I’m having some conversations, throw ideas, playing with the first bits, giving design suggestions and so on.
And they are listening, so kudos to them!
My issue here is that we are cache:ing null for N minutes if we get a bad-request... but assuming you handle that, I think this new cache could be very helpful - especially with db-lookups or external API-calls
Will it use in-memory if the distributed cache fails?
I think that's an awesome one! Looking into the docs/options, it's not clear to me whether it behaves in a multi-layer manner, or whether it is an option that you can switch on. I.e., if I want the cache to first try to grab a value from the local (IMemoryCache) cache, and then if not found - go to the shared (IDistributed) one, and once found or refreshed - update the local cache entry back. I used to build something like this to reduce some redis calls. Of course this comes with a cost of extra delay for the updated value to get it to the consumers, however, if configured the TTL properly for both shared and local caches, might be worth it for some scenarios. So, is this now achievable out-of-the-box with the new HybridCache thing?
thanks
Which IDE and CLI do you use please ?
Btw, could you make a video on Garnet?
Never heard of. But looks great. Thank you
It's coming
@@nickchapsas fun observations: 1) preview 4 *also* includes the code-changes to make IDistributedCache work with Garnet, and 2) I have an experimental hack which implements IDistributedCache on the Garnet storage core *without* actually going through any of the redis layer, intended for in-process disk-based local node caching (for cold-start or larger-than-RAM scenarios)
Why did they choose to make it an abstract class instead of interface?
Is almost the same, the benefitial is that in an abstract class you can opt to override implementations instead of extending an "option or builder" class that you then have to override implementation
To ease evolvability in the future: with interfaces they can’t add new stuff without breaking existing implementations, whereas with an abstract class they can provide a default implementation.
@@jodydonetti You can also have default implementation in the interfaces though
@@iamprovidence-xj5wf yes, kinda, but less powerful and more fragile. They touched on this in the gh issue, maybe there are more details there about the full rationale, I don’t remember right now. To me personally, as an implementer (the only one for now?) of the abstract class it does not make much difference honestly.
@@jodydonetti I mean, it is Microsoft afterall. they have discussed and justified the design with all the experts they have😒. It's just that making this shift from interfaces to abstract classes is what mentally difficult for me😅
Very nice!
Can you do a video about how to debug applications that run on docker/kubernetis?
I don't know who has built this framework, congratulations and thanks 👍
RemoveByTag distributed is killer, finally.
Yay I can get ride of all the semaphoreslims
Why isn't there a Get method. What should I do to just check is something in cache?
That has been left out from preview 4, will probably be in preview 5.
Very interesting
Don't like the abstract approach. The benefit of an interface is simply that you choose how to implement all methods; in an abstract class there might be additional methods you can't overwrite or there are constructors expecting something. So there is clear drawbacks but I can't imagine any real benefits (I think even the calling speed of interface methods and v-calls is the same).
What I really never like about IMemoryCache is that there's basically no namespace. Tags are a nice workaround for cleanup but it doesn't solve the $"{namespace}:{id}" issue.
Interfaces cannot be evolved without a breaking change, classes can, that’s the reason they picked a class.
love it!
That's weird, I don't see any course abut Azure stuff like cosmos db or azure function or blob storage or azure DevOps in dometrain , I will be happy if you add them , thanks
This is interesting. But can you make sure that all the code is shown on the screen. Some of it was cut off.
How is this multi-call protection works? So if I call for weather of London twice, the second call is waiting. But if I call with othe city, do both calls go through to the openweather server?
Does in have "automatic" population after some time like old .NET frameworks MemoryCache had with its callbacks?
12:00 Distributed cache*
I wonder if they made the HybridCache an abstract class for the same reasons they're recommending you use collection classes for performance. You see that more and more. If that is the case I would agree with it being an abstract class. Too bad indeed they made another package. I hope they remove IMemoryCache to prevent any confusion. Seems like a simple upgrade tool could fix this easily when going from .NET 8 to 9. They should promote the tool for that more.
Nope, they are doing it because the idea is that it can become a shared abstraction that other libs can implement (like FusionCache), like a lingua franca: because of that, evolving the design with an interface would lead to breaking changes, whereas with an abstract class you don’t have this problem.
Tags don't work yet.
It is London, its might 😁😁
So this in NET9 is replacing the "Microsoft.Extensions.Caching.Memory" ?
Syntactic sugar for ConcurrentDictionary. Nice!
its a christmas miracle!
Have redis changed their license and no longer open source?
If so what are some good alternatives?
Confused .NET 9 preview 4 is available at the moment, how can you be using .NET 9 preview 5?
Where i can get this source code
This is why you shouldn't directly code to the M$ interfaces. You should always wrap them within your own interface and call the MS implementation from your own implementation so you can swap them out at will without changing your client code.
Can we get this somehow in .net 8 too?
yes, in fact it should work today with the binaries from .NET 9 Preview 4; Hybrid Cache includes down-level support for older TFMs
What happen if, during the GetCreate cache, the API request fails? Will it caches the bad response or will it ignores it?
I think this is quite common case, where normally you would end up with all incoming requests receive cached bad response
at the moment: all in-flight shared calls (from the stampede protection) will get the failure; new calls will retry; if you want a cached failure, my suggestion is for the thing you cache to be a "result or failure" object, i.e. you have a "catch" in your fetch code that returns a similar API signifying failure
A lack of common interface seemed weird to me, having a fake distributed cache for testing was odd.
Where can I find the source code for this video DEMO?
Thank you.
Is this code avaiable on github?
I don't get how this is new tech, thats nothing new. I'm using 2 tiered cache including proper event driven cache invalidation for 10+ years in .NET.
It is great they finally include something like this into their caching abstraction, but new tech? Bro, na, not even close :p
Also, you suggest that everyone should replace the IDistributedCache interface with this feature? That's risky. Unless you know exactly what you are doing and why you need it, don't. Caching this way is more complex and has other challenges, so if you have to rely on cache validity and never expect stale data, don't use this ever.
If you do not know what I'm talking about, don't use it either.
Does this work if you hit multiple instances of the application? or would each of them run the request to the weather api?
If you add the distributed cache it will use that for every app
@@nickchapsas that I understand, but my question was more about if the flood protection would synchronize between multiple instances of the application or if there is a small window where multiple requests could go off (one per instance)
@@Petoj87 at the moment the stampede protection is per-node; centralized stamped (distributed locking) is a future possibility, but is unlikely to make it into .NET 9 (because: timing)
@@MarcGravell thanks!
I like this video because it's flaming London
Ever since I started working with colleagues from UK the weather in my town is all the time cloudy and slightly raining throughout the year. I think I should quit the job to save others from this misery. 😂
I'm the one who always looks for js framework alternatives, basically i don't like blazor because it has over engineering and it's not compatible for MCV, MVC which is real flex of web development, really looking forward hydro
🎯 Key Takeaways for quick navigation:
00:00 *🌐 Introduction to Caching in .NET 9*
- Explanation of the need to replace the ID distributed cache interface
- Introduction to the new caching mechanism in .NET 9
- Comparison between the in-memory cache and the new caching mechanism
05:15 *🔍 Issues with Existing Caching Methods*
- Challenges with stampedes in caching operations
- Limitations of the ID distributed cache interface
- Lack of features like tag-based eviction and instrumentation in current caching methods
08:27 *💡 Introduction to Hybrid Cache in .NET 9*
- Overview of the new Hybrid Cache introduced in .NET 9
- Benefits of using the Hybrid Cache over the ID distributed cache and in-memory cache
- Simplification of caching operations with the Hybrid Cache solution
Made with HARPA AI
I want to use Ai chatbots and see which ones are best for .Net? like problem solving, code analysis ect
Cloudy in London... Who would have thought?
London weather can be cached for 20 hours without a loss of accuracy.
Microsoft named a built-in cache extension after a specific company, StackExchange? Very unusual.
firstly: this is technically OOB, not built-in; but that is moot; the Microsoft.Extensions.Caching.StackExchangeRedis package is named for the library that it uses: "StackExchange.Redis"; could it / should it have been named M.E.C.Redis instead? maybe, but that ship has sailed
ironically, StackExchange.Redis is now maintained not by Stack Exchange, but by 2 ex-Stack Exchange employees (me and Nick), and by someone from Azure Redis (Philo); so that's 3 Microsoft employees ;p But it is basically impossible to rename the library, because: breakage
They named it after an implementation of a redis client, the best one in the .NET world.
@@chris-pee That makes sense. Thank you.
Oh caching...
MS SHOULD SLOW DOWN WITH NEW .NET VERSIONS!! EVERY 2 WEEKS NEW VERSIO
well then, how I do I know MS didn't steal my code from Github?
Anyone knows what is the Nick's preferred Redis interface over IDistributedCache?
IConnectionMultiplexer
I manually implemented something similar to HybridCache, but it was impossible to do proper Cache Invalidation. You can clear the distributed cache, you can clear the current instance's local cache, but you can't clear a different instance's local cache. This meant we had to use a plain distributed cache if the use case required cache invalidation. Fortunately most use cases don't need it.
But this doesn't cache locally (if distributed is used) from what I understood. The name comes from the fact that uses local cache or distributed cache with the same interface. I think it just uses distributed locks for a specific resource (key) and interacts with the distributed cache back and forth multiple times for acquiring the lock for the specific resource and then changing the resource and then releasing the lock. The lock itself is an entry in Redis with the resource id as key and the unique instance id as value.
you can implement distributed invalidation using pub/sub or similar. I like the idea of tiered caching as long as there are sane defaults and plenty of options to customize when needed.
Take a look at FusionCache, it does that already (shameless plug)
The roadmap for HybridCache includes active cache invalidation, for example use redis notifications; it didn't make it into preview 4, but I hope to ship it in time for .NET 9; with that, you get indirect cache invalidation of other nodes via the shared L2
source: me
I use IOutputCache with an extensions to support redis, so in this scenario between hybrid and ioutputcache you can go either way, however i heavly tested the ioutputcatch and seems to be pretty fast too!
something that occurred to me (I'm the lead for Hybrid Cache and did the redis-based IOutputCache implementation): once tag-based eviction and active invalidation is complete in Hybrid Cache, it would actually be possible to implement IOutputCache's backend *on top of* Hybrid Cache, giving output cache support for all IDistrubutedCache implementations