mut generics

[programmerjake]

imho it'd be useful for code to be able to be generic over mutability.

related:
rust-lang/rfcs#414

[yoshuawuyts]

From chatting with @tmandry a while back, we figured something like this might be possible:

impl str {
    fn get(&mut<M> self, idx: usize) -> Option<&mut<M> char> {
        ...
    }
}

impl<T> Vec<T> {
    fn iterate(own<O> self) -> VecIter<O, T> { .. }
}

There is a real question of how to implement VecIter at that point though, since the internals need to look something like this:

// Shouldn't this be a slice iter in the non-owned case?
// Want to select `IntoIter` for owned, `slice::Iter(Mut)` or unowned.
struct VecIter<own O, T> {
    _marker: PhantomData<own<O> Vec<T>>,
    // How to mark NonNull?
    ptr: *mut<O> T,
}

But assuming we can specialize implementations of structs, I imagine it might be possible to define three distinct VecIter types - one for borrowed, mut borrowed, and owned. We need a similar mechanism to specialize between async and !async structs too.

[clarfonthey]

One (not very) lofty goal I'd have for a mut generic syntax would be to be backwards-compatible with existing immutable methods, so we could potentially replace old APIs without the need to rename them.

[ghost]

One (not very) lofty goal I'd have for a mut generic syntax would be to be backwards-compatible with existing immutable methods, so we could potentially replace old APIs without the need to rename them.

I can not imagine that it would be backwards compatible to make any function generic regarding the references, if it were not already generic in this regard.

Take str::split_once. If we make it generic, it would yield mutable references for a mutable reference input. Right now, it references the input as immutable, then yields borrowed immutable references.

[dhardy]

Owning variant?

First, is it desirable that a method written to be generic over &self and &mut self automatically have an owning self variant too? No: for example MutexGuard supports DerefMut but cannot support into_inner; hash_map::OccupiedEntry supports get(&self) -> &V and get_mut(&mut self) -> &mut V but into_mut(self) has return type &'a mut V not V. Thus it should be possible to use keyword generics for method impls without owning variants.

Second, is ownership orthogonal to mutability? E.g. slices support as_ptr(&self) -> *const T and as_mut_ptr(&mut self) -> *mut T. As with the Vec::iterate example above, this implies a need to map mutability across reference/pointer types, and there is no clear way to do this if the receiver is self (owning).

Choice of variants?

If calling as_ptr on a slice, it's clear that the receiver is &self. But calling a mut-generic method on a mutable value/reference/pointer then using the result as non-mut does not make it clear which variant is called.

Issue 1: let p: *const T = v.as_ptr(); may imply a mutable borrow on v when v is &mut S (type inference usually considers input types only or at least before output types).

Issue 2: method resolution may even depend on whether v was declared let v = ... or let mut v = ....

This becomes worse if mut-generics includes owning variants.

Explicit choice of variant

This will be required sometimes. How will it be achieved?

One answer is to cast a mut type to a non-mut type. Syntax will not always be ideal, especially with repeated usage.

Mut-generic method over non-generic impls?

Say I want to write some abstraction over HashMap or some third party container which provides both get and get_mut methods but no mut-generic equivalent, and that I want to write mut-generic methods in my API. Can I do that?

Or, say, I want to write some method with a mut-generic API but with subtly different behaviour in mut and non-mut impls (e.g. a mut-generic version of Rc::make_mut).

Can something appear in the API as one mut-generic method but be implemented via two separate methods? (This sounds more like C++'s function overloading.)

If the answer is yes, then expect much confusion due to the in-obvious rules regarding method resolution (above). But if the answer is no then:

1. Mut-generic methods with subtly different behaviour are impossible. For the most part this is a good thing, but likely there will be some annoying edge-cases or missed optimisations.
2. Writing mut-generic code first requires making all dependencies sufficiently mut-generic. This could be a major impediment to usage of this feature.

Last word

Although I have a strong dislike for having to write both mut and non-mut variants of methods with identical implementations, there are enough issues here to make me question whether this feature could be worth the cost. Especially since adding the feature now may require collections have get, get_mut and get_gen_mut methods for compatibility.

A light-weight alternative might be some form of macro that does not change the API at all, but allows expansion. Tricky, since it affects method names:

fn foo<_mut>(&<mut> self) -> &<mut> X {
    let <mut> x = &<mut> self.x;
    x.bar<_mut>()
}

[dewert99]

Choice of variants?

I'm not sure how much I like this, but one possibility is to always instantiate mutability parameters with "shared" (not mut) unless they explicitly passed in. eg.

fn first<T, mut M>(x: &mut<M> [T]) -> &mut<M> T {
  & mut<M> x[0]
}

fn test() {
   let mut x = &mut [0, 1];
   let a = first(x)
   assert_eq!(type_name_of_val(&a), type_name::<&i32>());
   let b = first::<i32>(x)
   assert_eq!(type_name_of_val(&a), type_name::<&i32>());
   let c = first::<_, mut>(x);
   assert_eq!(type_name_of_val(&a), type_name::<&mut i32>());
}

This has the advantage that it would be backwards compatible with methods that worked with shared references, and makes uses of mutability more explicit, but is quite verbose.
Since mut is a keyword we could potentially allow first::<mut>(x) or even first::mut(x) to infer other (type/lifetime/...) parameters, and instantiate the (first) mutability parameter to mut.
When instantiating mutability parameters to other mutability parameters things get somewhat trickier.
Having first::<M>(x) desugar to first::<_, M> would make the desugaring depend on whether M was declared as a mutability parameter or a type parameter which seems too non-local. An alternative would be to have first::<mut M> desugar to first::<_, M>, while this might not even save characters it does avoid needing to remember how many type/lifetime parameters are required.

This desugaring could potentially be taken even further by allowing ::mut and ::<mut M> to instantiate trait mutability parameters if there aren't any method mutability parameters eg:

trait AsRef<T, mut M> {
  fn as_ref(&mut<M> self) -> &mut<M> T;
}

fn test<mut M>(v: &mut<M> Vec<u32>) {
  let s: &mut<M> [u32] = x.as_ref::<mut M>();
}

Mut-generic method over non-generic impls?

One possibility here would be not to let the type-system know that there are only two different mutability, in other words to distinguish between implementing something for all mutability vs only implementing it for "shared" and "mutable". This could potentially even allow the ability to reflect on a mutability parameter to be a library feature.

// simulate a higher kinded type using a trait
trait HKTMut {
  type This<mut M>;
}


trait MutReflect<mut M> {
  fn reflect<T: HKTMut, U: HKTMut>(x: T::This<M>, f_shr: FnOnce(T::This) -> U::This, f_mut: FnOnce(T::This<mut>) -> U::This<mut>) -> U::This<M>;
}

// self can't be a mutability so we use () for self and make the mutability a trait parameter
impl MutReflect for () {
  fn reflect<T: HKTMut, U: HKTMut>(x: T::This, f_shr: FnOnce(T::This) -> U::This, _: FnOnce(T::This<mut>) -> U::This<mut>) -> U::This {
    f_shr(x)
  }
}

impl MutReflect<mut> for () {
  fn reflect<T: HKTMut, U: HKTMut>(x: T::This<mut>, _: FnOnce(T::This) -> U::This, f_mut: FnOnce(T::This<mut>) -> U::This<mut>) -> U::This<mut> {
    f_mut(x)
  }
}


// Even if reflect_mutability was a compiler-builtin it would probably still need the same signature other than the where bound
fn reflect_mutability<T: HKTMut, U: HKTMut, mut M>(x: T::This<M>, f_shr: FnOnce(T::This) -> U::This, f_mut: FnOnce(T::This<mut>) -> U::This<mut>) -> U::This<M>
  where (): MutReflect<M> {
  <() as MutRefelect<M>>::reflect(x, f_shr, f_mut)
}

This would then allow any function to reflect on a mutability parameter as long as it adds a (): MutReflect<M> bound, while also guaranteeing functions without this bound won't have subtly different behaviour. I'm not sure, but it may even be possible that the compiler could erase mutability parameters that are not used in where clauses to reduce code-gen.

For convenience, we could also have a simple version of reflect_mutability for the common case where we are just dealing with references.

struct RefHKTMut<T>(PhantomData<T>);

impl HKTMut for RefHKTMut {
  type This<mut M> = &mut<M> T;
}

fn reflect_mutability_ref<T, U, mut M>(x: &mut<M> T, f_shr: FnOnce(&T) -> &U, f_mut: FnOnce(&mut T) -> &mut U) -> &mut<M> U
  where (): MutReflect<M> {
  reflect_mutability::<RefHKTMut<T>, RefHKTMut<U>, M>(x, f_shr, f_mut)
}

fn deref_rc<T, mut M>(x: &mut<M> Rc<T>) -> &mut<M> T 
  where (): MutReflect<M> {
  reflect_mutablity_ref::<_, _, M>(x, |x| &*x, |x| x.make_mut())
}