RFC: Exhaustive traits. Traits that enable cross trait casting between trait objects.

[izagawd]

This RFC proposes #[exhaustive] traits to enable sound cross-trait casting for trait objects.

For any concrete type T, the set of #[exhaustive] traits it implements is finite and deterministic, allowing runtime checks like “if this dyn A also implements dyn B, cast and use it.”

The design adds a per-type exhaustive trait→vtable map and enforces four rules (type-crate ownership of implementation, trait arguments determined by Self, object safe, and 'static only) to keep the mapping coherent under separate compilation.

Use cases include capability-based game entities (e.g., Damageable, Walkable traits) and GUI widgets (e.g., Clickable, Scrollable),
avoiding manual registry/macro approaches such as bevy_reflect.

This enables patterns such as: "if dyn Character is dyn Flyable, then character.fly()"

Rendered

[burdges]

We could avoid Rule 1 by building the vtable lookup table externally in ./target, no?

As &'static [(TypeId, TraitVTable)] is a nasty lookup, https://internals.rust-lang.org/t/casting-families-for-fast-dynamic-trait-object-casts-and-multi-trait-objects/12195 proposed defining fresh indices for each "casting family", so that lookup takes place in &'static [TraitVTable]. In that, the families kept the vtable lookup tables small, but this could be avoided perhaps. And it could all be done via procmacros, without changing rustc.

[izagawd]

I made another RFC: #3888.

If that lands, rather than having every trait object being forced to store metadata for casting, there could be a

trait Castable: 'static {
    const self EXHAUSTABLE_IMPLEMENTATIONS: &'static [(TypeId, TraitVTable)];
}

which could have a blanket implementation of:

impl<T: 'static> Castable for T {
   const self EXHAUSTABLE_IMPLEMENTATIONS: &'static [(TypeId, TraitVTable)] = std::intrinsics::exhausive_implementations::<T>();
}

Which would make the metadata for casting opt in

Though this will require a good chunk of changes to the exhaustive_traits RFC

It would be ideal to make this Castable trait use final for the EXHAUSTIBLE_IMPLEMENTATIONS const self field. final coming from this RFC: #3678, as making the EXHAUSTIBLE_IMPLEMENTATIONS final would be better than having a blanket implementation. The exhaustible_trait RFC doesnt allow exhaustible traits to have blanket implementations after all, so the final keyword would do wonders. It would be weird that the Castable trait itself was not considered as exhaustible.

Of course we can do compiler magic, and make the Castable trait the only exhaustible_trait that is blanket implemented, but that does not seem ideal to me if we can just do final const self EXHAUSTABLE_IMPLEMENTATIONS: &'static [(TypeId, TraitVTable)] = std::intrinsics::exhausive_implementations::<Self>();

[tmccombs]

To me "exhaustive" suggests that the trait can only be implemented within the defining crate (like sealed). I'm not really sure what a better name is though.

[izagawd]

@tmccombs I would use a better name than exhaustive if I could, but I can't think of a better name either ¯_(ツ)_/¯

[burdges]

Appears intertrait already provides a much cleaner solution, by using the linkme.

At present linkme needs linker support, but one could bypass the linker, using only #[no_mangle] and procmacros. Also, linkme cannot control ordering, which demands a hashmap like in this RFCs, so maybe best if all the procmacros compute trait indices too, thereby avoiding the hashmap entirely.

[izagawd]

@burdges i tried intertrait and it doesnt... work, not even the example code in their crates.io page

I also tried out trait-cast. It does not support generic parameters at all, and requires that i must have the ptr_metadata feature gate available. And again, I have to do the extra step of registering the relationship between a type and interface, which i think should be implicit rather than explicit

cast_trait_object seems to support generic parameters, but it involves some boilerplate setup that registers relationships, which again, I feel should not be explicit.

[burdges]

Interesting thanks. Appears 4 years since their last version, and the owning blockchain has no updates since May 2023, so bitrot. It nevertheless shows that rule 1 maybe excessively restrictive.

Anyways I definitely agree that generic statics or similar ala #3888 sound useful.

[izagawd]

It nevertheless shows that rule 1 maybe excessively restrictive.

I agree, but I cannot find a way around this due to how separate compilation works. I would gladly remove that rule if it were soundly possible

[Scripter17]

Is it possible for sidecasting from trait A to trait B to make a subtrait of both, downcast to that subtrait, then upcast to B?

I don't think it's that farfetched but I only have vague surface level of the compiler so I don't really know. I just figured I should make sure it's considered.

[burdges]

Yes, but only for types defined downstream of both traits, by Rule 1. If you have a &dyn A coming from a &u64 under the hood, then you cannot cast it to a &dyn B. And Box was not directly addressed.

how separate compilation works

There are degrees of separate compilation here: We've large projects using codegen-units=1 for various reasons, by far the worst failure of separate compilation, but really quite standard. By comparison, intertrait only needs some append only structure that's compiled late. Imho intertrait does not go far enough, and should reprocess those append only structures into efficently indexable families.

Around this, I wonder if intertrait failed because it depends upon lto or another linker flag?

Anyways, all these crates for casting dyn Trait track the vtables for the different pointer types seperately. As they check the underlying type, their reason for doing would be unstability of the Pointer layout, aka soundness bitrot. I'd suggest two preliminary changes:

First, Rust should commit to all smart pointer types using the same vtables. This is probably already the case. Second, Rust should exposes unsafe methods that manipulate the vtable pointer, optionally safe ones too. This would prevent soundness bitrot in external crates that cast dyn Traits, and allow wider adoption.

[arielb1]

So the current proposal does not involve linker tricks?

The non-linker-trick version would essentially allow doing error provides by adding subtraits of Error.

[arielb1]

cc https://rust-lang.zulipchat.com/#narrow/channel/219381-t-libs/topic/Pushing.20error_generic_member_access.20forward/with/567603494

[arielb1]

@rustbot label +I-lang-nominated

I am nominating this for lang team discussion.

Me and the libs team are trying to close our story for Error generic member access. It has come to our attention that this sort of trait downcasting essentially duplicates the functionality of Provide in a way that is easier to observe via rustdoc.

We would prefer not to have 2 ways of doing Provide, and for that we need to be aware whether this looks like a reasonable feature lang-wise.

[jackh726]

@traviscross @joshtriplett and I had a brief discussion about this RFC. We all seemed excited about this work, but all felt that it likely should start as a lang experiment rather than as an RFC.

On that note, we would need to find a lang champion for this. Probably the easiest/best path forward on that is to open a channel in #t-lang on zulip.

[Keith-Cancel]

@izagawd Regarding on of the Unresolved questions:

• Could #[exhaustive] be a keyword rather than an attribute?

I was having a short conversation with @programmerjake on zulip about just kinda an idea I had initially unrelated to this RFC. I have quoted part of the conversion below. Anyways he brought up this RFC, and it seems like it could also be useful. Essentially the short of it is some special bound I am just assuming it was called crate.

trait MyExhaustive: crate {
    // items...
}

@Keith-Cancel

You seem like you might know, I did search around a bit. However, did not find anything or maybe it was called something else. However, do you know if any sort of crate level bound was ever thought about.

Just for example assume it was called crate

impl<T: MyTrait + crate> CrateExtrenalTrait for T {
   // impl...
}

Something like that blanket impl becomes possible at least for types defined in a crate. This keep the principles of the orphan rule since we are saying this applies only to types defined in the crate. The alternative is generally like a macro or manual repetition.

It was more of just a thought, but more was trying to see if anything like that had been discussed anywhere?

@programmerjake

i'm not aware of anything that restricts a blanket impl to crate-local types, but there is a RFC that proposes traits that error if they're implemented for types that aren't in the same crate as the impls: #3885

@Keith-Cancel

Hmm I had not run into that, but I suppose instead of attribute like they proposed, the bound idea I am talking about would work. Assuming crate again for the name, but this would allow similar a concept.

trait MyExhaustive: crate {
   // items...
}

Nothing can implement this but types defined in the same crate. While something similar can be achieved with a sealed trait. I do suppose something like this would allow the compiler to reason about the trait in ways like the author of that rfc is wanting compared to a sealed trait which is more just a visibility trick.

It also feels more in line with the language than attribute idea since it avoid caveats like no blanket impls. Since those would be allowed, but such impls would have to include crate as one of their bounds for the type.

[burdges]

Imho, rust should seriously consider "mild" linker tricks, not only here, but reducing repeated codegen, etc. At least to me, Rule 1 seems way too heavy handed when seemingly simple alternatives exist.

Also, rust should remain open & friendly to linker tricks, so if one adds features like this then one should think about how they interact with better alternatives. At the same time, if this gives a way to resolve common situations before linker phase, then okay whatever.

[izagawd]

@Keith-Cancel the blanket implementation, but only for within the defining crate sounds interesting, but how about this scenario?

impl<T: MyTrait + crate, U> CrateExtrenalTrait<U> for T {
   // impl...
}

this could lead to concrete types implementing an infinite amount of CrateExternalTrait<T> generic variants, which would cause issues for exhaustive_traits. Though it should be possible if done with traits with no generics, or their generic parameters are concrete ((ie Trait<i32> rather than Trait<T>)

as for using:

trait MyExhaustive: crate {
   // items...
}

this implies that if I make a trait say:

trait Foo: MyExhaustive{}

that Foo is also exhaustive, and gains the restrictions and benefits of being an exhaustive trait as well, which may not be as flexible, as there may be some traits that want an exhaustive trait to be their supertrait, but not be exhaustive traits themselves. Though it could be argued that it should be the desired behavior

[Keith-Cancel]

@Keith-Cancel the blanket implementation, but only for within the defining crate sounds interesting, but how about this scenario?

this could lead to concrete types implementing an infinite amount of CrateExternalTrait<T> generic variants, which would cause issues for exhaustive_traits.

@izagawd Like your already allowed to do that with an instance of a crate local type and an external trait with a generic parameter: Rust Playground

as for using:

trait MyExhaustive: crate {
   // items...
}

this implies that if I make a trait say:

trait Foo: MyExhaustive{}

that Foo is also exhaustive, and gains the restrictions and benefits of being an exhaustive trait as well, which may not be as flexible, as there may be some traits that want an exhaustive trait to be their supertrait, but not be exhaustive traits themselves. Though it could be argued that it should be the desired behavior

Like whether the crate local restriction is expressed as a bound or your attribute, if Foo has MyExhaustive as a super trait the only possible way to implement Foo is if the type is crate local, otherwise you can't implement MyExhaustive I would say that is expected.

Conceptually to me there are two parts to this RFC, basically a crate local trait bound, and the cross crate casting.

[izagawd]

@izagawd Like your already allowed to do that with an instance of a crate local type and an external trait with a generic parameter: Rust Playground

I am aware that we are allowed to do that, but how exhaustive_traits works is that it stores all its implementations (as typeid vtable mappings) in it's metadata to allow cross trait casting. If it enabled infinite implementations, it will be impossible to do that

[Keith-Cancel]

@izagawd Like your already allowed to do that with an instance of a crate local type and an external trait with a generic parameter: Rust Playground

I am aware that we are allowed to do that, but how exhaustive_traits works is that it stores all its implementations (as typeid vtable mappings) in it's metadata to allow cross trait casting. If it enabled infinite implementations, it will be impossible to do that

@izagawd This can be worked around instead of treating the trait type ids as a flat id in your exhaustive map, if they composed of two parts. Basically one half identifies the trait, the other half the generic, and instead of just plain equality we change equality to: lhs == (lhs & rhs).

Might be easier to explain with just some code: Rust Play Ground

--EDIT-
Slight issue with that logic in general, but the idea still is the same.
Here it is updated with slightly different eq logic: Play Ground

[programmerjake]

@izagawd This can be worked around instead of treating the trait type ids as a flat id in your exhaustive map, if they composed of two parts. Basically one half identifies the trait, the other half the generic, and instead of just plain equality we change equality to: lhs == (lhs & rhs).

Might be easier to explain with just some code: Rust Play Ground

that still doesn't work for:

impl<T: MyTrait + crate, U> SomeTrait<U> for T {
   // impl...
}

because U isn't bounded by crate so it can be any arbitrary type, e.g. for every N you can have MyType: SomeTrait<[(); N]> which is enough different N values that you can't possibly store all of them in memory assuming you need more than 1 byte for each trait impl.

[Keith-Cancel]

@izagawd This can be worked around instead of treating the trait type ids as a flat id in your exhaustive map, if they composed of two parts. Basically one half identifies the trait, the other half the generic, and instead of just plain equality we change equality to: lhs == (lhs & rhs).
Might be easier to explain with just some code: Rust Play Ground

that still doesn't work for:

impl<T: MyTrait + crate, U> SomeTrait<U> for T {
   // impl...
}

because U isn't bounded by crate so it can be any arbitrary type, e.g. for every N you can have MyType: SomeTrait<[(); N]> which is enough different N values that you can't possibly store all of them in memory assuming you need more than 1 byte for each trait impl.

@programmerjake I don't see what you mean though this non-flat id does not work for impl<T: MyTrait + crate, U> SomeTrait<U>? There will always be only on V-Table regardless of U for each T in the crate? So if there is a... I guess you could it like a wild card ID in the exhaustive match table it will match any other type ID for U to that vtable.

That aside, I was busy earlier today after posting that, but was still thinking about it. I realized a couple things first my eq logic would need changed these are not bit-field ids so that trick does not quite work. The BIGGER issue I noticed was deeper levels of generics:

impl<T: MyTrait + crate, U> SomeTrait<(u32, U)>  for T {}
impl<T: MyTrait + crate, U> SomeTrait<(U, u32)>  for T {}
impl<T: MyTrait + crate, U, V> SomeTrait<(U, V)> for T {}

This can get arbitrarily wide or deep. It also made wonder how we would even generate IDs deterministically in the CONCRETE case. Since even if every type was explicitly spelled the type expression of that generic could arbitrarily long or deep. So would need like IDs of unbounded length or a hard limit on length and nesting.

--Edit--
@programmerjake That aside the thing that interests me most here is just some form of crate level bound, since it makes writing certain types of trait impls much nicer or possible.

[programmerjake]

@programmerjake I don't see what you mean though this non-flat id does not work for impl<T: MyTrait + crate, U> SomeTrait<U>? There will always be only on V-Table regardless of U for each T in the crate? So if there is a... I guess you could it like a wild card ID in the exhaustive match table it will match any other type ID for U.

the problem is that to support trait casting there has to be a finite known set of possible dyn types that you can cast to, so the compiler can generate the table of vtables, but dyn SomeTrait<U> is an infinite set of dyn types, one for each different type U, so for any possible table of vtables that the compiler could generate, there will always be some type U that isn't in that table even though it should be. [(); N] is just an easy way to get a very large set of possible U types where there's at least one type for every byte of memory that could possibly exist in the compiled program.

[Keith-Cancel]

the problem is that to support trait casting there has to be a finite known set of possible dyn types that you can cast to, so the compiler can generate the table of vtables, but dyn SomeTrait<U> is an infinite set of dyn types, one for each different type U, so for any possible table of vtables that the compiler could generate, there will always be some type U that isn't in that table even though it should be. [(); N] is just an easy way to get a very large set of possible U types where there's at least one type for every byte of memory that could possibly exist in the compiled program.

@programmerjake What's the V-Table for this then leaving casting aside?

impl <T> MyTrait<T> for MyType { ... }

It's just same V-Table regardless of of T correct?

This would just be doing the same but for every type in the crate. Naively it could just thought of copying pasting basically the above.

impl<T: crate, U> MyTrait<U> for T { ... }

Now back to casting let just look at this case. In the exhaustive table all we care is that we match the trait, the generic part could be treated as a wild card/any ect... since we have an impl that applies to all T. Hence why I split it into two parts a trait_id and generic_id. If the generic_id is zero in the table it's basically a wild card otherwise it's a concrete type.

impl <T> MyTrait<T> for MyType { ... }

The bigger issue which I noted above is how to handle arbitrarily wide or nested types which I noted above.

[programmerjake]

@Keith-Cancel I replied here: #3885 (comment)

[programmerjake]

actually, we should move this conversation to a separate thread to avoid clogging up the main PR comments, so I'm creating one here. was originally here #3885 (comment)

@programmerjake What's the V-Table for this then leaving casting aside?

impl <T> MyTrait<T> for MyType { ... }

It's just same V-Table regardless of of T correct?

nope, it generates a different vtable for each T that's used anywhere for unsizing to dyn MyTrait<T> (ignoring some lifetimes here).

e.g. in https://rust.godbolt.org/z/WsrxzKf5W
each of f1, f2, and f3 generate a different vtable:

pub trait MyTrait<T> {
    fn f(&self) -> &'static str;
}

pub struct MyType;

impl<T> MyTrait<T> for MyType {
    fn f(&self) -> &'static str {
        std::any::type_name::<dyn MyTrait<T>>()
    }
}

#[unsafe(no_mangle)]
pub fn f1(v: &MyType) -> &dyn MyTrait<[(); 1]> {
    v
}

#[unsafe(no_mangle)]
pub fn f2(v: &MyType) -> &dyn MyTrait<[(); 2]> {
    v
}

#[unsafe(no_mangle)]
pub fn f3(v: &MyType) -> &dyn MyTrait<[(); 3]> {
    v
}

[Keith-Cancel]

@programmerjake Hmm, Okay I see what you are saying. That does make things trickier for the casting and having it work without as many restrictions as in the RFC. Before I looked at this I kinda thinking about some kinda TraitID(id=X, generic=PATTERN) thing to handle nesting/wide types, but was not really coming up with any good ways to flatten a pattern to an integer, but looking at that Assembly kinda nixes that.

So the first 2 probably could be handled.

// Finite
impl<T: crate> MyTrait T { ... }
impl<T: crate, U: crate> MyTrait2<U> { ... } 

// Not Finite
impl<T: crate, U> MyTrait2<U> { ... } 

Although I wonder if there is a way around that since to do the cross_trait_cast a concrete type has to specified in someway for the generic part of the trait. We could use that concrete type to materialize/actualize such v-tables during monomorphization.

trait MyTrait2<T>: crate { }

fn cast(a: &dyn Other) {
    // We know that MyTrait2 has a finite number of implementers because
    // it's crate local only bound.
    // So it would seem possible to ensure during compilation, since we specify
    // MyTrait2 with a concrete type to check for MyTrait2 impls with a generic.

    // Then any such generic implementation like:
    // `impl<T> MyTrait2<T> Foo { ... } ` gets monomorphized with `[u32; 2]`
    // and as part of monomorphization that vtable type gets added to the Foo's
    // exhaustive table.
    let b: &dyn MyTrait2<[u32; 2]> = cross_trait_cast_ref(a).unwrap();
}

fn cast_gen<T>(a: &dyn Other) {
    // Even with a generic function like this `T` will have to
    // be a concrete type at some point to monomorphize this function.
    // At which point we can check `MyTrait2` implementers
    let b: &dyn MyTrait2<T> = cross_trait_cast_ref(a).unwrap();
}

Although, I am not sure that would work well with how the compiler currently compiles crate by crate. There also might be something else I am over-looking that could make doing that hard.

if that's the case it might be sensible have an attribute like the RFC's #[exhaustive] that further restricts how a trait can be implemented, and something like crate bound since it seems generally useful for more than just the cross trait cast feature.

[programmerjake]

if that's the case it might be sensible have an attribute like the RFC's #[exhaustive] that further restricts how a trait can be implemented, and something like crate bound since it seems generally useful for more than just the cross trait cast feature.

sounds good to me! if cross-trait casting can be made to efficiently work with generics like you proposed, that can always be added as a future extension, though I think it should still have some opt-in syntax on the trait that's more than just trait Trait: crate since it has a significant code-size cost from all the vtables especially on embedded systems where you only have a few kB of memory.

[Keith-Cancel]

I think it should still have some opt-in syntax on the trait that's more than just trait Trait: crate since it has a significant code-size cost from

True those exhaustive tables could/may add a fair bit of static data to a binary, so even if it's possible to create them based off bounds alone, it might not best to create them automatically in all cases of a crate bound unless requested, and just have those pointers in the vtable be an Option/Nullable or point to same single empty stub unless specified.

An attribute for opting into that seems fine since a lot the of other std attributes are for memory and memory layout purposes.

[Keith-Cancel]

@izagawd what's your thoughts on something like this:

#[cross_cast]
trait MyTrait: crate { ... } // again just assuming `crate` for now.

I like it because it separates the two main concepts #[exhaustive] as proposed does. Allowing one to be used independently or both in combination.