RuntimeContexts 2.0

[Washi1337]

Summary

This is a PR that improves RuntimeContext to make it more similar to an AssemblyLoadContext (as seen in the BCL, but for static binaries), and will further pave the way towards a full robust implementation of AsmResolver.Workspaces (#298). It aims to give RuntimeContext a lot more responsibility, including assembly and metadata resolution. This is to make things more predictable, robust, and more diagnoseable when loading and constructing new metadata references across multiple assemblies.

This is a breaking but important change: Ideally we want to get this in before we fully release 6.0.

Overview Changes

Revamp of RuntimeContext:

• IAssemblyResolver is drastically simplified and does not do caching anymore.
• IMetadataResolver and friends are removed.
• RuntimeContext construction is drastically simplified.
• RuntimeContext is now responsible for resolving and maintaining assembly and metadata caches.
• Addtion of RuntimeContext::LoadAssembly methods to load assemblies into the context (similar to AssemblyDefinition::FromX, but using the context for reader parameters, cache and automatic wiring of dependencies).

Revamp of Metadata Resolution System:

• All parameterless Resolve methods are removed.
• All T? Resolve(ModuleDefinition) methods are replaced with a throwing T Resolve(RuntimeContext), a non-throwing bool TryResolve(RuntimeContext, out T?) and a more general ResolutionStatus Resolve(RuntimeContext, out T?).
• All methods that require some form of metadata resolution now require a RuntimeContext parameter as well. This was done to avoid implicit (failing) resolutions.
• Removal of bool ITypeDescriptor::IsValueType property in favor of a method bool GetIsValueType(RuntimeContext) and bool? TryGetIsValueType(RuntimeContext). Note that TypeSignature and TypeDefinition still define the property, as these are defined by metadata itself.

Other changes:

• Add TargetRuntimeProber to detect the target runtime of a PE image without loading it as a full ModuleDefinition.
• Add MetadataDirectory::GetImpliedStreamSelection()

Examples

Creating new contexts

var context = new RuntimeContext(DotNetRuntimeInfo.NetCoreApp(10, 0));

var context = new RuntimeContext(DotNetRuntimeInfo.NetFramework(4, 0), is32Bit: true, searchDirectories: [@"C:\Path\To\Directory"]);

PEImage image = ...;
var context = new RuntimeContext(image); // Also auto-detects runtimeconfig.json settings.

RuntimeConfiguration config = ...;
var context = new RuntimeContext(config, sourceDirectory: @"C:\Path\To\Directory\");

IAssemblyResolver resolver = ...;
var context = new RuntimeContext(DotNetRuntimeInfo.NetCoreApp(10, 0), resolver);

BundleManifest manifest = ...;
var context = new RuntimeContext(manifest);

Loading assemblies into RuntimeContexts

var assembly = context.LoadAssembly(@"C:\Path\To\File.exe");

var assembly1 = context.LoadAssembly(@"C:\Path\To\File.exe");
var assembly2 = context.LoadAssembly(@"C:\Path\To\File.exe"); // Same file results in same instance.

Resolving metadata

ITypeDescriptor descriptor = ...;
AssemblyDefinition definition = descriptor.Resolve(context); // Throws on failure.

ITypeDescriptor descriptor = ...;
if (descriptor.TryResolve(context, out var definition))
{
   // ...
}

ITypeDescriptor descriptor = ...;
switch (descriptor.Resolve(context, out var definition))
{
    case ResolutionStatus.Success:
        Console.WriteLine($"Resolved type: {definition}");
        break;
    case ResolutionStatus.AssemblyNotFound:
        Console.WriteLine("Declaring assembly not found.");
        break;
    case ResolutionStatus.TypeNotFound:
        Console.WriteLine("Type was not found in resolved assembly.");
        break;
    ...
}

IsValueType, GetIsValueType and TryGetIsValueType

For references or generic descriptors, use GetIsValueType or TryGetIsValueType:

ITypeDescriptor type = ...;
bool isValueType = type.GetIsValueType(context); // Throws on resolution failure.

ITypeDescriptor type = ...;
bool? isValueType = type.TryGetIsValueType(context); // null on resolution failure.

For definitions and signatures, IsValueType property remains:

TypeDefinition type = ...;
bool isValueType = type.IsValueType;

TypeSignature type = ...;
bool isValueType = type.IsValueType;

Rationale

IMemberDefinition::Resolve()

The main reason for this big change is that many users have experienced seemingly random output corruptions or type resolution failures when constructing new binaries, in particular when adding metadata to an AssemblyDefinition using the various helper methods (e.g., CreateTypeReference, ToTypeSignature, MakeGenericInstanceTypeSignature). Currently, these helper methods often rely on the value of ITypeDescriptor::IsValueType to encode blob signatures properly. This property may implicitly trigger a resolution if the type needs to be resolved to a TypeDefinition to check its base type.

Experience by users has shown that the method TypeDeifnition? ITypeDescriptor::Resolve(), while convenient for many use-cases, has a pretty brittle design:

1. It returns null on failure, and leaves out the reason for why it failed. This makes it hard to diagnose where in the resolution process it failed (e.g., wrong type name, could not find containing assembly file...)
2. It relies on ContextModule's metadata resolver. This is inaccurate when new type references are created on-the-fly. In such a case, a TypeReference doesn't always have a context module set yet, causing its resolution to fail despite it being a valid reference. It is also inaccurate when TypeReference instances are reused in other assemblies, which happens a lot after auto-import was introduced (Automatically import Type/Field/MethodDefintions when building #689). Some low-hanging fruits were fixed (e.g., in Address Auto Import Regressions #697), but the core problem remains.
3. Finally, since Resolve() returns null but methods like ToTypeSignature() still need a proper value for TypeDefinition::IsValueType, usually a default value is set (i.e., false). While this is in most cases correct, it is not sound for all types.

This PR aims to make things more predictable, robust, and more diagnoseable when things go wrong. All methods that may require type resolutions now are explicitly marked as such.

Processing multiple assemblies at once

Additionally, users have reported that it is sometimes difficult to process a collection of assembly definitions, especially when a binary has many dependencies which in turn have dependencies etc. Currently, assemblies kind of live in their own bubble, with their own metadata and asembly resolvers attached. This works if assemblies are processed in isolation, but falls apart when processing assemblies with complex dependency graphs. The isolated resolvers would maintain own cachecs of resolved assemblies and types, resulting in multiple instances of the same assembly or type being loaded.

The first version of RuntimeContext aimed to fix this by providing a shared IAssemblyResolvers to ModuleDefinitions. However, this had bugs (#580) and the manual wiring of assembly objects in user code is very unintuitive.

This PR extracts most of the resolution and caching mechanism out of ModuleDefinition, IMetadataResolver and IAssemblyResolver, and moves it into the shared RuntimeContext. All resolutions now always go through the runtime context, ensuring consistency and more efficient lookups.

Related Issues

Closes #580, #703

[github-actions]

Test Results

	Total	Skipped	Passed	Failed
Unique	2132	5 💤	2132 ✅	0 ❌
Total	4264	36 💤	4228 ✅	0 ❌

Failing runs