Not yet! Let's draw a call graph for this function:
decimalGetCartTotal(string cartId)
{
var cartTotal = 0M;
var cart = Carts.Get(string userId); // Carts is ICartServicevar specialOffers = SpecialOffers.GetActive(); // etc.foreach (var item in cart.Items) {
cartTotal += item.Price * item.Quantity;
cartTotal -= specialOffers.Max(
o => o.GetDiscount(item.Product.Id, item.Price, item.Quantity))
}
return cartTotal;
}
THEY CONVERGED!
A new Incremental Builder build you must!
Recompute everything → saturate CPU May the Cache be with you!
Chatty client-server RPC → saturate NIC And the Client-Side Cache too!
Add caching behavior using a Higher Order Function
Add caching behavior using a Higher Order Function
Add caching behavior using a Higher Order Function
Func<TIn, TOut> ToCaching<TIn, TOut>(Func<TIn, TOut> fn)
=> input => {
var key = Cache.CreateKey(fn, input);
if (Cache.TryGet(key, outvar output)) return output;
lock (Cache.Lock(key)) { // Double-check lockingif (Cache.TryGet(key, out output)) return output;
output = fn(input);
Cache[key] = output;
return output;
}
}
var cachingGetCartTotal = ToCaching(Carts.GetCartTotal);
var cartTotal = cachingGetCartTotal.Invoke("cart1");
A tiny issue
To make it work, fn must be a pure function.
⇒ You saw a Vaporware Version™ of ToCaching.
Solutions*
Plan 😈: Purify every function!
Plan 🦄: Add dependency tracking + cascading invalidation
Plan : Add Dependency Tracking + Cascading Invalidation
Func<TIn, TOut> ToAwesome<TIn, TOut>(Func<TIn, TOut> fn)
=> input => {
var key = Cache.CreateKey(fn, input);
if (Cache.TryGet(key, outvar computed)) return computed.Use();
lock (Cache.Lock(key)) { // Double-check lockingif (Cache.TryGet(key, out computed)) return computed.Use();
computed = new Computed(fn, input, key);
using (Computed.ChangeCurrent(computed))
computed.Value = fn(input);
Cache[key] = computed;
return computed.Use();
}
}
Dependency capture
public TOut Use()
{
if (Computed.IsInvalidating) { // Will explain this later
Invalidate();
returndefault;
}
// Use = register as a dependency + "unwrap" the Value
Computed.Current.AddDependency(this);
return Value;
}
Everything is cached
and (re)computed incrementally
Dependencies are captured automatically
So we invalidate just what's produced externally!
It's a transparent abstraction that doesn't change functions' signatures, code, and even their output!*
Do we really need delegates?
We don't. Making DI container to provide a proxy implementing such decorators is a
So it can be absolutely transparent!
Can I use this now?
Not quite:
No async/await, thread-safety
We need GC-friendly cache and UsedBySet
No actual impl. of Computed
Etc.
Boring technical problems!
– Elon Musk*
Let me show 100+ more slides first!
What about eventual consistency?
What about React and Blazor?
We need to go deeper!
What is worse than
eventual consistency?
Permanent inconsistency.
Two eventually consistent systems were left at your doorstep.
Which one you should marry?
#1
#2
How this is relevant to real-time, again?
Real-time updates require you to...
Know when a result of a function changes Invalidate all the things!
Recompute new results quickly Incrementally build all the things!
Send them over the network Blazorise and AspNetCorise all the things?
Ideally, as a compact diff to the prev. state Diff can be computed in O(diffSize) for immutable types (details).
"There are only two hard things in Computer Science: cache invalidation and naming things."
– Phil Karlton
See, we've made a meaningful progress with an easy one!
WHAT ABOUT...
Blazor is:
.NET running in your browser on top of WASM!
100% compatible with .NET 5:
Expression.Compile(...), Reflection, Task<T>, etc. – it just works!
Nearly all of your managed code will run on Blazor too.
(Blazor Components, React Components) ≍
(,) – same, but better! Oh, this is so Microsoftey!
Blazor – cons:
1 for now – but JS developers live with this for 25 years, SO WE CAN!
No JIT / AOT yet, MSIL is interpreted. .NET 6, don't disappoint us!
10 x 20x AOT ≃ 200x
Even a small project downloads 2…4 MB of .NET .dlls (gzipped!) - and that's after linking with tree shaking. Cmon, it's 21 century – size doesn't matter.
At least online.
React and Blazor are "make"-like incremental builders for your UI
Just specialized ones – designed to incrementally update DOM
(actually, any UI control tree) after any component's render() that
actually just defines the new desirable UI state.
Remember Caching Decorator with Dependency Tracking?
Func<TIn, TOut> ToAwesome<TIn, TOut>(Func<TIn, TOut> fn)
=> input => {
var key = Cache.CreateKey(fn, input);
if (Cache.TryGet(key, outvar computed)) return computed.Use();
lock (Cache.Lock(key)) { // Double-check lockingif (Cache.TryGet(key, out computed)) return computed.Use();
computed = new Computed(fn, input, key);
using (Computed.ChangeCurrent(computed))
computed.Value = fn(input);
Cache[key] = computed;
return computed.Value;
}
}
interfaceIComputed<T> {
ConsistencyState ConsistencyState { get; }
T Value { get; }
Exception Error { get; }
event Action Invalidated; // Event, triggered just once on invalidationTask WhenInvalidated(); // Alternative way to await for invalidationvoidInvalidate();
Task<IComputed<T>> Update(); // Notice it returns a new instance!
}
TodoApp: ITodoService API
Note:CancellationToken argument is removed here & further to keep things simple.
It's not the actual type you consume - the actual runtime-generated replica service (AKA Fusion client) implements the same ITodoService. ITodoClient type just maps its endpoints to the web API relying on RestEase under the hood.
publicvirtualasync Task<User?> TryGet(long userId)
{
awaitusingvar dbContext = DbContextFactory.CreateDbContext();
var user = await dbContext.Users.FindAsync(new[] {(object) userId});
return user;
}
// Many readers, 1 (similar) mutatorasync Task<long> Reader(string name, int iterationCount)
{
var rnd = new Random();
var count = 0L;
for (; iterationCount > 0; iterationCount--) {
var userId = (long) rnd.Next(UserCount);
var user = await users.TryGet(userId);
if (user!.Id == userId)
count++;
extraAction.Invoke(user!); // Optionally serializes the user
}
return count;
}
Caching performance
Sqlite EF provider: 16,070x
With Stl.Fusion:
Standard test:
Speed: 35708.280 K Ops/sec
Standard test + serialization:
Speed: 12481.940 K Ops/sec
Without Stl.Fusion:
Standard test:
Speed: 2.222 K Ops/sec
Standard test + serialization:
Speed: 2.179 K Ops/sec
In-memory EF provider: 1,140x
With Stl.Fusion:
Standard test:
Speed: 30338.256 K Ops/sec
Standard test + serialization:
Speed: 11789.282 K Ops/sec
Without Stl.Fusion:
Standard test:
Speed: 26.553 K Ops/sec
Standard test + serialization:
Speed: 26.143 K Ops/sec
And that's just plain caching, i.e. no benefits from "incrementally-build-everything"!
Caching Sample & more data points on caching
The same service converted to Replica Service:
20,000 → 130,000 RPS = 6.5x throughput
With server-side changes only, i.e. regular Web API client.
20,000 → 20,000,000 RPS = 1000x throughput!
If you switch to Fusion client (so-called "Replica Service")
RestEase Client -> ASP.NET Core -> EF Core Service:
Reads: 20.46K operations/s
RestEase Client -> ASP.NET Core -> Fusion Proxy -> EF Core Service:
Reads: 127.96K operations/s
Fusion's Replica Client:
Reads: 20.29M operations/s
How 10x speed boost looks like?
Limits are meant
to be broken.
Fusion vs Redis, memcached, ...
Almost always consistent
Local = 1000x faster:
No network calls
No serialization/deserialization
Reuse vs deep copy on use
Incrementally-Build-Everything™
Supports "swapping" to ext. caches.
Fusion vs SignalR
Automatic & transparent pub/sub
"X is invalidated" vs all SignalR events
Guaranteed eventual consistency
And:
λ Substance (build) vs form (update)
(think React vs jQuery)
Web API-first design Sub-1ms responses, CQRS, multi-host,
custom update delays → high-load ready Single abstraction on the client, server, ...
Learning curve: relatively shallow in the beginning, but getting steeper once you start to dig deeper. ~ Like for TPL with its ExecutionContext, ValueTask<T>, etc.
Other risks: "We wants it, we needs it. Must have the precious!"
If you need a real-time UI or a robust caching, Fusion is probably the lesser of many evils you'll have to fight otherwise.*