When, if ever, are `extern "Rust"` ABIs compatible with each other?

[joshlf]

This question was originally discussed on Twitter here.

I'm trying to write a library to stack-allocate dyn Future objects. The core of this code is the following:

const N: usize = 8;

pub struct DynamicFuture<O> {
    data: [u8; N], // NOTE: Alignment is handled, but I've elided it here for simplicity
    poll_fn: for<'r, 's, 't0> fn(Pin<&'r mut ()>, &'s mut Context<'t0>) -> Poll<_>,
    _marker: PhantomData<O>,
}

When a DynamicFuture is created, the static type of the future is known, and poll_fn is set to <F as Future>::poll. Then, in the implementation of Future for DynamicFuture, the data field is converted to a Pin<&mut ()>, and poll_fn is called on it.

My question is: Is the ABI of <F as Future>::poll (type signature: for<'r, 's, 't0> fn(Pin<&'r mut F>, &'s mut Context<'t0>) -> Poll<_> guaranteed to be the same as the ABI of poll_fn (identical type signature except for Pin<&'r mut ()> instead of Pin<&'r mut F>)? If so, is it sound to call poll_fn by constructing a Pin<&mut ()> from the data field?

[comex]

See also: When can function pointers be transmuted?

But in general, this is not guaranteed. The default extern "Rust" ABI makes basically no guarantees at all, and even with extern "C", there are control flow integrity schemes that prohibit calling function pointers with the wrong signature.

Instead, I suggest creating a wrapper function:

unsafe fn poll_wrap<'r, 's, 't0, F: Future + 'r>(this: Pin<&'r mut ()>, ctx: &'s mut Context<'t0>) -> Poll<F::Output> {
    let f_raw: *mut F = Pin::into_inner_unchecked(this) as *mut _ as *mut _;
    let f_pin: Pin<&'r mut F> = Pin::new_unchecked(&mut *f_raw);
    f_pin.poll(ctx)
}

Then you can convert that into a function pointer, since its signature doesn't depend on the future type.

[bjorn3]

Is the ABI of ::poll (type signature: for<'r, 's, 't0> fn(Pin<&'r mut F>, &'s mut Context<'t0>) -> Poll<_> guaranteed to be the same as the ABI of poll_fn (identical type signature except for Pin<&'r mut ()> instead of Pin<&'r mut F>)?

If F is an unsized type, no Pin<&'r mut F> would be a fat pointer while Pin<&'r mut ()> is a thin pointer.

If so, is it sound to call poll_fn by constructing a Pin<&mut ()> from the data field?

No, &mut () gives access to exactly zero bytes behind the pointer under the stacked borrows memory model, which is less than the size of F for all non-ZST futures.

[comex]

I don't think that's quite right. If F has a field field of type () which is at the beginning of F, then &mut f.field would be an &mut () with permission to zero bytes. But casting f to &mut (), with &mut *(f as *mut _ as *mut _), doesn't reduce the permissions. Or at least, miri complains about one but not the other.

[bjorn3]

I am pretty sure that the moment you write &mut * your permission gets reduced to zero bytes. That miri doesn't intermediately may be because of a lack of an actual access to the referenced data.

[Lokathor]

yeah that sure sounds like a miri error. you shouldn't have a reference with access to more span than the size of its type (as far as i know).

[joshlf]

See also: When can function pointers be transmuted?

But in general, this is not guaranteed. The default extern "Rust" ABI makes basically no guarantees at all, and even with extern "C", there are control flow integrity schemes that prohibit calling function pointers with the wrong signature.

Instead, I suggest creating a wrapper function:

unsafe fn poll_wrap<'r, 's, 't0, F: Future + 'r>(this: Pin<&'r mut ()>, ctx: &'s mut Context<'t0>) -> Poll<F::Output> {
    let f_raw: *mut F = Pin::into_inner_unchecked(this) as *mut _ as *mut _;
    let f_pin: Pin<&'r mut F> = Pin::new_unchecked(&mut *f_raw);
    f_pin.poll(ctx)
}

Then you can convert that into a function pointer, since its signature doesn't depend on the future type.

Great idea! Ran into some lifetime issues, so we ended up getting rid of 'r; do you think this is sound as written?

unsafe fn poll_wrap<'s, 't0, F: Future>(this: Pin<&mut Buffer>, ctx: &'s mut Context<'t0>) -> Poll<F::Output> {
    let pin : Pin<&mut F> = this.map_unchecked_mut(|slf| transmute::<&mut Buffer, &mut F>(slf));
    pin.poll(ctx)
}

To address @bjorn3 's concern, we do type Buffer = [MaybeUninit<u8>; N].

(note: this is joint work with @jswrenn)

Appendix

We use our own custom transmute that works in a generic context:

#[inline(always)]
unsafe fn transmute<Src, Dst>(src: Src) -> Dst
{
    use core::mem::ManuallyDrop;

    #[repr(C)]
    union Transmute<Src, Dst> {
        src: ManuallyDrop<Src>,
        dst: ManuallyDrop<Dst>,
    }

    ManuallyDrop::into_inner(Transmute { src: ManuallyDrop::new(src) }.dst)
}

[comex]

Well, you shouldn't need transmute… if you're using Buffer instead of (), so you're going down in size instead of up, &mut *(x as *mut _ as *mut _) should work.

Edit: But I think it is sound with or without transmute.

[joshlf]

OK great, thanks!

[RalfJung]

yeah that sure sounds like a miri error. you shouldn't have a reference with access to more span than the size of its type (as far as i know).

What concrete example is a Miri error?

When you transmute a fn ptr, what effectively happens is that on a call, the argument is transmuted from the caller type to the callee type.

In this case the argument seem to not be a reference but a Pin<&mut T>; only references are retagged on function entry and this does not recurse into struct fields. Doing so would make RefCell unsound -- but maybe RefCell indeed is buggy here. See #125 for more discussion of this subject.