-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
JIT: De-abstraction in .NET 10 #108913
Comments
Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch |
I worked up a suite of related benchmarks: dotnet/performance#4522. Ideally all these (save for the With recent main we see:
Note there is a penalty for enumerating an array inside another loop. That's something else we should try and fix.
|
De-Abstraction
In .NET 10 we hope to further enhance the JIT's ability to remove abstraction overhead from code.
Stack Allocation Improvements
See #104936
During .NET 10 we would like to implement 2-3 enhancements to stack allocation of ref class instances. Priority may be given to issues that enable array de-abstraction (see below).
Delegate GDV Improvements
We currently only GDV instance delegates. We'd like to extend support to static delegates.
PGO Improvements
We currently lose a bit of performance in R2R compiled methods with Tiered PGO, because the instrumented version of the method doesn't collect profile data for inlinees.
#44372
Inlining Improvements
We'd like to enable inlining of methods with EH.
#108900
Array Enumeration De-Abstraction
#108153
The goal of this work is to (to the best of our ability) eliminate the abstraction penalty for cases where an
IEnumerable<T>
is iterated viaforeach
, and the underlying collection is an array (or is very likely to be an array).In previous releases we've built a number of optimizations that can reduce abstraction overhead. But there is still a lot of room for improvement, especially in cases like the above, where the abstraction pattern involves several abstract objects acting in concert.
What is the Abstraction Penalty?
Consider the following pair of benchmark methods that both sum up an integer array:
These two methods do the exact same computation, yet benchmarking shows the second method takes 4.5x as long as the first (with 512 element arrays, using very early .NET 10 bits incorporating #108604 and #108153):
This sort of overhead from an abstract presentation of computation is commonly known as the abstraction penalty.
Note things used to be far worse; .NET 6's ratio here is 12.6.
Why is there an abstraction penalty?
The IL generated for the
foreach_static_readonly_array_via_interface
is expressed in the shape of the abstract enumeration pattern: firste.GetEnumerator()
is called on the abstract collection to produce an abstract enumerator, and then loop iterates viaMoveNext()
andget_Current()
interface calls on this enumerator, all wrapped in a try finally to properly dispose the enumerator should an exception arise.Seeing through all this to the actual simple computation going on in the loop requires a surprising amount of optimization machinery. In past releases we've built many of the necessary pieces, and now it's time to get them all working together to remove the remaining overhead.
In particular we need to leverage:
More generally the JIT will need to rely on PGO to determine the (likely) underlying type for the collection.
Why focus on Arrays?
Arrays are the most common and also the simplest collection type. Assuming all goes well we may try and stretch the optimization to cover Lists.
What needs to be done?
When the collection is an array, the enumerator is an instance of the ref class
SZGenericArrayEnumerator<T>
. Thanks to #108153 we can devirtualize (under guard) and inline the enumerator constructor, and devirtualize and inline calls on the enumerator. And in some cases we can even stack allocate the enumerator (note in the table above, .NET 10 no longer has allocations for the benchmarks).Current inner loop codegen:
However, we cannot yet fully optimize the enumeration loop:
SZGenericArrayEnumerator
constructor has an optimization for empty arrays, where instead of constructing a new enumerator instance, it returns a static instance. So at the enumerator use sites there is some ambiguity about which object is enumerating. For cases where the array length is known this ambiguity gets resolved, but too late in the phase order.SZGenericArrayEnumerator
, so all three definitions can reach through the enumerator GDV tests (see JIT: Support for devirtualizing array interface methods #108153 (comment) for a picture and more notes). And we may get confused by the try/finally or try/fault which will also contain a reference to the enumerator (for GDV).While these may seem like small problems, the solutions are not obvious. Either we need to disentangle the code paths for each possibility early (basically do an early round of cloning, not just of the enumeration loop but of all the code from the enumerator creation sites to the last use of the enumerator and possibly the EH regions) or we need to think about making our escape and address propagation logic be flow-sensitive and contextual and introduce runtime disambiguation for the reaching values (that is, at each enumerator use site, we have to test if the enumerator is the just-allocated instance from the
SZGenericArrayEnumerator
that we hope to stack allocate and promote).The cloning route seems more viable, but it is costly to duplicate all that code and we'll have to do it well in advance of knowing whether the rest of the optimizations pay off. So we might need to be able to undo it if there's no real benefit.
The text was updated successfully, but these errors were encountered: