Hierarchy of Sized traits

[davidtwco]

All of Rust's types are either sized, which implement the Sized trait and have a statically known size during compilation, or unsized, which do not implement the Sized trait and are assumed to have a size which can be computed at runtime. However, this dichotomy misses two categories of type - types whose size is unknown during compilation but is a runtime constant, and types whose size can never be known. Supporting the former is a prerequisite to stable scalable vector types and supporting the latter is a prerequisite to unblocking extern types. This RFC proposes a hierarchy of Sized traits in order to be able to support these use cases.

This RFC relies on experimental, yet-to-be-RFC'd const traits, so this is blocked on that. I haven't squashed any of the previous revisions but can do so if/when this is approved. Already discussed in the 2024-11-13 t-lang design meeting with feedback incorporated.

See this comment for the most recent summary of changes to this RFC since it was opened.

Rendered

[programmerjake]

• Use std::ptr::Pointee
• Do the hack to avoid new ambiguity in the current addition
• Remove the hack in the next addition

imo having an associated type on practically every type will be annoying, so I think it might be useful to require writing <T as Pointee>::Metadata instead of T::Metadata on all editions.

[workingjubilee]

the trait's unstable, if it's just a matter of bikeshedding we can rename the type PtrMetadata or sth

[davidtwco]

I think std::ptr::Pointee should definitely be reused. I am not sure which alternative is, but it seems to me like the right thing to do is:

• Use std::ptr::Pointee
• Do the hack to avoid new ambiguity in the current addition
• Remove the hack in the next addition

I've added this as one of the unresolved questions.

In particular, the "Why have Pointee?" is a bit misleading in that semantically there currently is indeed no reason, it is just there for syntactic purposes.

I've rewritten this section, it should have been updated earlier with the addition of the alternatives which keep the ?Sized syntax and clarified that it's just a syntactic difference.

At the point I still think it is good education to talk about the empty trait set as:

• You can't use it behind a pointer
• You can't use it as a value
• You could use it in PhantomData, but probably this is left as future work
• Maybe you can't quantify over it all (as opposed to you can, but then you can't use the variable anywhere, and then its an unused type parameter error)

Oh and a final thing on semantics, I might also also present the back compat / editions story this way:

• Old semantic hierarchy
• New semantic hierarchy
• mapping from old to new semantics, (old {} (no traits) is mapped not to new {} but to const MetaSized) --- in particular, do not speak of ?Size when describing this semantic mapping.
• mapping from new syntax in next edition maps to new semantics.

In particular, the existing old syntax maps to the new semantics via the old semantics; we can cleanly factor that into those two steps, and I think this brings conceptual clarity.

I haven't made these changes at the moment - I've had good feedback about the explanations in the RFC and would prefer not to make major changes to that until the proposal itself needs changing.

[davidtwco]

I've updated the previous summary comment so it is still accurate.

[davidtwco]

I've pushed one small change with a reference to the now-upstream const traits RFC in #3762.

[safinaskar]

Linux has at its syscall boundary struct statmount. See here for details: https://brauner.io/2024/12/16/list-all-mounts.html . Its first field is size of whole struct itself. And it is u32 on all platforms! So, yes, I think we should support such structs. Note: this is not size of DST in its end, it is size of whole struct.

There is even more exotic case: struct dirent ( https://www.man7.org/linux/man-pages/man3/readdir.3.html ). (Yes, I already talked about it in previous comment, but I want to add more.) It embeds its own size, too. But not as a first field, but as a third. :) And that field (d_reclen) is not usize, it is u16! And again, this is not size of last DST field, this is size of struct itself.

Note: manpage says that size of last field is 256. This is a lie. In fact, this is DST, and it can be both bigger and smaller than 256.

Also, the string in the last field is always proper null-terminated C string, so we have two different ways of determining struct size: by calling strlen on d_name (with necessary calculations) and by reading d_reclen. So, technically d_reclen field is redundant. And to properly support this struct in Rust, Rust should have at least one of the following two features:

• ThinCStr we all dream about. It should be allowed to be located in the end of a struct (this will support this d_name field)
• Embedding size of a struct (of a whole struct!) inside struct itself. Not necessary as a first field. And not necessary as a usize. u16 should be supported

Not supporting both features will lead to horrible hacks, such as this: https://github.com/rust-lang/rust/blob/c1132470a6986b12503e8000e322e9164c1f03ac/library/std/src/sys/pal/unix/fs.rs#L730-L768 (this is from standard library!)

And again: to support statmount we need possibility to embed size of struct inside struct itself. As a first field of size u32 (even on 64-bit platforms). In this case calling strlen on last field will not help, because it is allowed to contain many embedded null bytes

[RalfJung]

Not supporting both features will lead to horrible hacks, such as this:

In the latest version, this hack is mostly gone. What remains is not so bad: https://github.com/rust-lang/rust/blob/58d6301cad7ac20407e708cd9d1ec499d1804015/library/std/src/sys/pal/unix/fs.rs#L726-L753
(rust-lang/rust#136479 cleans this up further by removing the macro.)

IMO it is not worth contorting the language to natively express esoteric types such as those, as long as we have some way of interfacing with C code using such types (and we do).

[davidtwco]

Another summary comment!

Here are the previous update summaries copied below so you don't need to uncollapse GitHub comments to find it:

2024-11-25

Summary comment: #3729 (comment)

For those following along or catching up, these are the notable the changes to the RFC since this was posted:

• Clarify proposed behaviour for ?Trait syntax for non-Sized, which is currently accepted
  • 23cb8a2
• Stop re-using std::ptr::Pointee and make Pointee its own new marker trait to avoid backwards incompatibility
  • 0c2f6e6, 2831cca
• Clarify backwards compatibility implications of ?Sized syntax and add alternatives to removing the default bound using positive bounds which continue to use ?Sized
  • These are particularly valuable additions to the RFC as some of these alternatives are quite viable and have significant advantages
  • 5f4b8ff, ab8e7f6, b050c2b, 432a77a, 6e124e5, 4faeb83
• Add that relaxing existing bounds in trait methods would be backwards incompatible
  • b4258c7
• Elaborate on necessity of implicit const ValueSized bound on Self type of traits
  • 9158ebb, b71401c, 5573be7
• Add MetaSized alternative to ValueSized which would resolve interactions with mutexes
  • c9e71bd, ef2f5cd
• Clarified that bounds on return types can never be relaxed.
  • 145446d

And these are all the other smaller changes that don't materially impact what is being proposed:

• Fixed some minor wording errors where supertrait/subtrait were used backwards
  • a840d8b, 260c980
• Removed HackMD's rust= syntax from codeblocks
  • 95a1b7d
• Fixed referring to the introduction of a const Sized trait, but rather adding a const modifier to the existing Sized trait
  • b670517
• Added some background/context on dynamic stack allocation
  • b8ea881
• Use current experimental const trait syntax
  • a703feb
• Corrected incorrect syntax for traits
  • fad04b0
• Listed all alternate bounds (adding ~const ValueSized to a list of bounds that it was missing from)
  • 2fb3aaa
• Fixed bound in description of size_of_val changes
  • 397e474
• Corrected description of current size_of_val and align_of_val behaviour
  • 6c1482b
• Corrected description of extern type usage in structs
  • 1f85898
• Mention Rust for Linux's interest in extern type
  • 98af888
• Weaken language in the externref future possibility to make it clear this proposal would not be sufficient on its own to support these
  • 60eab1a
• Re-write Aligned future possibility so that it is clear Aligned couldn't be added to the proposed hierarchy
  • 61f8d6d

At the moment, I prefer the following alternatives to the primary proposal of the RFC, and may re-write to incorporate these as the primary proposal:

• Adding MetaSized (either replacing ValueSized or in addition to it), as per the "What about MetaSized instead of or in addition to ValueSized?" section.
• Using ?Trait syntax instead of positive bounds as per "Adding ?ValueSized"

2024-12-04

Summary comment: #3729 (comment)

These are the major changes since the last summary:

• Change from ValueSized to MetaSized
  • 72743b6, 57b9a85
• Add unresolved question about whether re-using std::ptr::Pointee is important.
  • d6d73b2

These are the minor changes since the last summary:

• Clarify that bounds on parameters used as return types can never be relaxed (return types still need to implement Sized)
  • 145446d
• Clarify that non-const Sized is not nameable in the current edition due to the backwards compatibility migration, only in the next edition
  • b620534
• Clarify which Pointee (marker trait Pointee or std::ptr::Pointee) is referenced in the Custom DSTs future possibility
  • 02d1759
• Improve clarity of the table describing alternative syntax proposals for relaxing bounds
  • 8cbaef9
• Strengthen wording describing use of runtime-sized types in a const context as unsound
  • 9fd3dff
• Expand on motivation for MetaSized
  • 462d016
• Specify that methods of the new traits would be backwards-incompatible too, not just associated types
  • aa21450
• Use the same indentation in code blocks throughout
  • c0ee97d
• Fix typo in list of when each trait gets implemented
  • 7a11fe9
• Add a concrete example of code that would break if a supertrait of Clone were relaxed
  • 762c6d8
• Added note about potential reasons why delaying relaxation of size_of's bound might be desirable
  • 52b776f
• Fixed typo, replacing "alignment" with "size"
  • 5d602fb
• Rewrote "Why have a Pointee trait?" section to clarify that the trait is only necessary due to the proposed removal of ?Sized syntax
  • f26784b
• Updated reference to const traits now that it has an open RFC
  • edec592
• Fixed a broken link
  • 19469dc
• Fixed broken indentation on a bullet list
  • 0f9ec84

These are the alternatives described in the RFC that I think are worth consideration as the primary proposal:

• Using ?Trait syntax instead of positive bounds as per "Adding ?ValueSized"

---

We had a design meeting with the language team yesterday (Feb 5th) which led to the changes described below, a rough summary:

• No significant concerns were raised with the proposal itself (i.e. the specific traits proposed, the edition migration proposed, etc)
• There was a clear consensus on some of the unresolved questions and no strong opinion on others (see 5806017)
• t-lang would be okay with a experiment proceeding with this, but the preference of the const traits implementors is that no experiment take place while const traits' syntax is undecided, as this would increase the work they have to do when making any potential syntax changes requested by t-lang
• It is still unclear how to proceed with regards to the const traits dependency
  • Further experimentation following the meeting (see ) has reinforced the necessity of the const traits dependency
  • There is disagreement within the language team whether the not-yet-accepted const traits dependency blocks this from being accepted (without a possibility stabilisation w/out const traits stabilisation)

I've started an implementation of this - I've more-or-less finished implementing the non-const parts of the proposal, and will proceed with the const parts. It won't land upstream until const traits implementors are okay with it.

These are links to the compare with previous versions of the document (switch to the "Files changed" tab and then the rich diff):

• Changes since the pre-RFC version of this discussed at the first language team design meeting
• Changes since the version of this discussed at the second language team design meeting
• Changes since the 2024-11-25 summary comment
• Changes since the 2024-12-04 summary comment

These are the major changes since the last summary:

• Add note describing a niche but unavoidable backwards incompatibility
  • 6533e10
• Note that const traits isn't a strict dependency
  • 7b99ac6 (superseded by below)
  • 4ba5b07
• Describe the perspective of the language team following our meeting on the unresolved questions (leaving them unresolved but expressing a bias in a direction)
  • 5806017

These are the minor changes since the last summary:

• Describe another alternative which would enable re-using std::ptr::Pointer
  • bef7459
• Note that these traits can be stabilised independently of each other
  • 96a70f9

[Skepfyr]

Thank you for the summary comments they're incredibly useful. I've re-read this and looked through the updates and this proposal feels closer, however I still have a few comments...

Positive trait bounds are growing on me though, I like that they stop us having to worry about what "the empty set of traits" means, however I'm not convinced that's possible, because of supertrait bounds (as described in a comment).

[Skepfyr]

I feel like this RFC is implicitly dependent on unsized locals here, the only way I can see the Clone or Copy traits supporting Sized types is with it. Consider <svint8_t as Clone>::clone that returns an svint8_t but that doesn't have a stack size known at compile time, so you need a bunch of the unsized locals machinery to support it right?

[Skepfyr]

I just chatted at Rust Nation about this. My understanding of the answer to this is: in some sense yes, because locals/arguments/return types are no longer const Sized, but it doesn't depend on the full power of unsized locals (as they're Sized) and it's mostly a case of punting the problem to the backend and LLVM already handles this just fine.

[davidtwco]

To clarify further - these types don't need unsized_locals machinery - not the skipped Sized constraints in the front of the compiler, nor the MIR/codegen changes in the back of the compiler - they just emit different LLVM types and LLVM handles the rest.

[Skepfyr]

This feels like surprisingly weak motivation for all the const stuff that's making this hierarchy complicated. I wonder if we can replace Clone with:

trait Clone {
    fn clone(&self) -> Self;
    fn clone_from(&mut self, other: &Self) where Self: Sized { ... }
}

and replace Clone bounds with Clone + Sized, admittedly this is more edition shenanigans that feels a bit dirty (and I haven't completely thought it through).

I do agree that having const trait stuff would give you the ability to do some stuff: size_of on non-const Sized types, and have arrays&slices of non-const Sized types (although this would require some complier work). I'm just not clear how useful those are.

[davidtwco]

I'm not sure that I agree the const stuff is making the hierarchy complicated, just the nature of these types. Const traits are a natural fit for this and if they were already an accepted and stable part of the language then I don't think there would be any concerns about using them for this proposal.

At a first glance, changing Clone like you've suggested seems plausible but I don't think I'll add it as an alternative because of the additional advantages that using const traits give us that changing Clone wouldn't, I think those are worth it - these types can't be used in const contexts and we need to be able to prevent that from happening.

[Skepfyr]

I think that's reasonable, I am worried that all of these const traits add too many variations here but you're right that they are correctly representing the types. I was mostly just trying to keep alive the notion that we don't have to perfectly represent all possible variations and if that makes the proposal significantly simpler then it might be a good idea.

[Skepfyr]

I'm still very nervous about this, I believe this is done solely for backwards compatibility reasons but I want to make sure it's clear this is sacrificing forwards compatibility.

If traits gain (debatably keep) an implicit const MetaSized bound then:

• Everything keeps working in a nice way
• People writing libraries will just write trait Foo {} rather than trait Foo: Pointee {} unless they're specifically thinking about unsized types for some reason making them a second class citizen in the ecosystem (and relaxing the bound when asked is a breaking change)
• If we ever add another layer below Pointee (eg externref) then no existing traits will work with them and it would be breaking to relax them. I think this means no std traits could ever support them.

If traits have no implicit bound (also debatably keep, depending on how you imagine this) then:

• You'd have to do this over an edition, and add in : const MetaSized everywhere so that default method implementations keep working. (removing that bound is safe for std, and dubious for crates).
• You'd need to write : Pointee etc if you need to write default method impls, but I think that's a good thing.
• This does force us to work out what you can do with "the empty set of traits", because that what these types have in default trait impls (with no supertraits) (this makes me less favourable towards positive bounds)

[davidtwco]

You'd have to do this over an edition, and add in : const MetaSized everywhere so that default method implementations keep working. (removing that bound is safe for std, and dubious for crates).

I'm open to this as an alternative to be considered.

[Skepfyr]

I think this was straw-manned slightly in the design meeting, I disagree with the phrasing:

'I don't want to think of the "next most precise thing"'

and

"one level below" in the hierarchy

Neither proposal makes you think like that (ignoring the ?MetaSized below), the ?Sized variant requires you to write the positive bound always (and I'd argue that the T: Pointee case should be written T: ?(const Sized) + Pointee) so the only negative thinking you have to do is "do I need to disable the default const Sized bound?".

However, I agree this discussion can be punted until stabilisation given it has essentially no impact on the semantics of what's going on.

P.S The idea of negative trait aliases (trait Foo = ?Sized;) terrifies me, it feels like it'd be incredibly confusing at the use-site.

[davidtwco]

Neither proposal makes you think like that (ignoring the ?MetaSized below), the ?Sized variant requires you to write the positive bound always (and I'd argue that the T: Pointee case should be written T: ?(const Sized) + Pointee) so the only negative thinking you have to do is "do I need to disable the default const Sized bound?".

I don't think there's a realistic chance of the "Keeping only ?Sized" variant being accepted, it's very verbose and I don't think it is being seriously considered.

"Adding ?MetaSized" is feasible but it does have the "opt-out of the next strictest thing" behaviour. This approach also has confusing behaviour with respect to stability, as you don't name the trait that you want and refer to it by another trait that has different stability. ?Trait bounds are also just seen as confusing in general.

I obviously can't speak for the language team, but I'm not aware that anyone on the team currently prefers a variant with negative bounds.

[nikomatsakis]

Thanks for the pushback @Skepfyr. It's true that we didn't explicitly discuss the idea of writing ?Sized to "reset" and then a positive bound in combination with it. It's also true that this proposal has many of the advantages of positive bounds — but it's also true that it's very verbose and (in my judgment) not the right way forward for that reason. There is definitely a judgment call to be made here, my judgment is that having a small set of "well known" traits that, instead of declared your intent to invoke methods wind up declaring your intent to use it in a more limited way will "wear better" in the long run.

[Skepfyr]

I guess my worry is that "small set of "well known" traits" won't be true, with trait aliases and potentially supertrait bounds relaxing the default bound. I agree that T: ?(const Sized) + Pointee is horrifically verbose, but if it's just syntax then maybe we could just fix that?

There's nothing stopping us from doing something like one of these:

fn foo<T: ? + Pointee>() {} // Just drop the (const Sized)
fn foo<T: {} + Pointee>() {} // {} to imply the empty set
fn foo<T: unsized + Pointee>() {} // unsized is a keyword! (probably my favourite)
fn foo<unsized T: Pointee>() {} // Might be nicer here

[traviscross]

This is being altered by:

• Reject ?Trait bounds in various places where we unconditionally warned since 1.0 rust#135841

[davidtwco]

Thanks - I'll update the wording here.

[traviscross]

Suggested change

	Traits which are a supertrait of any of the proposed traits will not
	automatically imply the proposed trait in any bounds where the trait is
	used, e.g.
	```rust
	trait NewTrait: MetaSized {}
	Traits which are a subtrait of any of the proposed traits will not
	automatically imply the proposed trait in any bounds where the trait is
	used, e.g.
	```rust
	trait NewTrait: MetaSized {}

[traviscross]

Is this true in just the subtractive direction or in the additive direction as well? I.e., this works today:

trait Tr: Sized {}
fn f<T: ?Sized + Tr>() {
    _ = const { size_of::<T>() }
}

[davidtwco]

It would only apply in the subtractive direction (at least w/out the changes you're proposing in Zulip and referencing here). I can clarify this.

[traviscross]

Is it? I mean, we accept this today:

trait Tr /* : ?Sized */ {}
trait Id {
    type Ty: Tr;
}
fn f<T: Id>() {
    _ = const { size_of::<T::Ty>() }
}

If we were to use the supertrait's ?Sized bound to imply the removal of the default Sized bound on the associated type, then this would break. It would seem the same could be said of the new things added here.

[davidtwco]

It could still be done over an edition, adding + Sized to T: Id when we make this change so existing code doesn't break. It's certainly nicer to do it all at once and avoid that, I agree, but I do think it's technically forward compatible. I'm not opposed to making a change like this one as a pre-requisite to this RFC's proposed changes being stabilised, I just think it should be a separate proposal to this one.

[traviscross]

As a matter of terminology, we generally don't consider a change that requires an edition to be a backward compatible change. We just say, e.g., that it is a "change we could make over an edition". Probably the language should be updated here to reflect this.

Specifically, I think the outcome we arrived at is still that we only get one shot, when we stabilize these new bounds, to do this in a way that would work consistently across all editions. While we could later make things work differently in a new edition, and do a migration dance, that's something we'd generally avoid doing if we already knew we wanted a different behavior.

In terms of the dependencies here, I'm increasingly not a fan of us accepting RFCs with big open questions like this. Doing so ends up causing us to later stall the stabilization or the work toward it, causing pain. It's better if we can really nail down the important details, especially those big ones that meaningfully affect how the user would experience the feature. That way there's a clear path from RFC acceptance to stabilization.

So, it's OK if here we don't want to include this in this RFC, but I'd prefer to see us decide this question ahead of accepting this RFC, and if we decide that we'd like a behavior that isn't fully specified in this RFC, that we then accept another RFC for that ahead of accepting this one, so this one can be updated accordingly and reference it.

[traviscross]

As a note for my fellow @rust-lang/lang members, I'd prefer if we did not leave this question (and perhaps some of the other bikeshed questions in this list) unresolved.

This is in the context of our ongoing discussions about streamlining our feature development processes.

It's always tempting for us to defer these sort of things at RFC time to let work go ahead, but this turns into a false economy, because we then end up getting overloaded and blocking it later, at a worse time. I'd propose that we just commit to the bikeshed now.

We can always later change our minds, of course, with good cause. But we should settle on a presumptive set of choices as part of accepting this RFC so that it has an unencumbered path to stabilization.

[cramertj]

In light of the strong preference towards positive bounds, I'd like to leave one last comment that I believe it would be very helpful to have a visual indicator for bounds that relax requirements rather than adding them.

Moving from <T> to <T: Pointee> (or where T: {many bounds} to T: {many bounds} + Pointee) appears to be a breaking change. IMO this will cause significant confusion for library owners, especially as there are several different traits that have this new behavior. This issue could be resolved by introducing a keyword like <T: keyword Pointee>, e.g. only Pointee, just Pointee.

[programmerjake]

This issue could be resolved by introducing a keyword like <T: keyword Pointee>, e.g. only Pointee, just Pointee.

an existing keyword that might work: <T: become Pointee>
as or = or maybe @ could also work