rust_for_linux: -Zregparm=<N> commandline flag for X86 (#116972)

[azhogin]

Command line flag -Zregparm=<N> for X86 (32-bit) for rust-for-linux: #116972
Implemented in the similar way as fastcall/vectorcall support (args are marked InReg if fit).

[rustbot]

r? @pnkfelix

rustbot has assigned @pnkfelix.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[bors]

☔ The latest upstream changes (presumably #129970) made this pull request unmergeable. Please resolve the merge conflicts.

[rustbot]

These commits modify compiler targets.
(See the Target Tier Policy.)

Some changes occurred in compiler/rustc_codegen_gcc

cc @antoyo, @GuillaumeGomez

[nikic]

As this affects call ABI, doesn't this need to be a target option rather than compiler flag? Otherwise @RalfJung will be very sad.

[jieyouxu]

As this affects call ABI, doesn't this need to be a target option rather than compiler flag? Otherwise @RalfJung will be very sad.

Am I understanding correctly that like -C soft-float, the problem that if this is a compiler flag, then it's easy to have code compiled with -Z regparam call code that is not -Z regparam even though they appear to be of the same concrete target, then dragons get summoned?

[nikic]

Yes, exactly.

[RalfJung]

As a nightly flag I don't mind having this experimentally, but the docs should call out very clearly that all code that is linked together needs to use the same value for this flag.

But this can't be stabilized in that form.

[jieyouxu]

As a nightly flag I don't mind having this experimentally, but the docs should call out very clearly that all code that is linked together needs to use the same value for this flag.

Although IIRC target options don't have a stable format either, so we may as well make this part of target options and implement this the "correct" way to make using it correctly less footgunny from the get-go instead of having the flag that's asking for fireworks? Since I imagine if RfL wants this flag then surely they'll want to actually use it. Especially if this is limited to a specific target architecture(?).

[RalfJung]

Yeah if this can be made a target option that would probably be better. I just didn't want to block experimentation on these concerns.

[jieyouxu]

Right, that's fair and fine by me as well.

[azhogin]

Sorry about the rebases, I hope they were relatively easy. 😅

Np. I have added __m128 & __m256 test cases (vector types are skipped in the same way as float types).

[ojeda]

I get an ICE building core, e.g.

rustc --edition=2021 --target i686-unknown-linux-gnu -Zregparm=3 --crate-type rlib library/core/src/lib.rs --sysroot=/dev/null

query stack during panic:
#0 [fn_abi_of_instance] computing call ABI of `slice::ascii::<impl at library/core/src/slice/ascii.rs:9:1: 9:10>::is_ascii`
#1 [eval_to_allocation_raw] const-evaluating + checking `ascii::ascii_char::<impl at library/core/src/ascii/ascii_char.rs:588:1: 588:30>::fmt::HEX_DIGITS`
end of query stack

The new tests pass though (by the way, since we pass --target, do we need only-x86? I noticed one of the sets was ignored)

[workingjubilee]

Backtrace gives this:

  17:     0x7a29058f5943 - fill_inregs<rustc_middle::ty::Ty, rustc_middle::ty::layout::LayoutCx>
                               at /home/jubilee/rust/rustc/compiler/rustc_target/src/callconv/x86.rs:178:17

Panic is here, it seems:

unreachable!("x86 shouldn't be passing arguments by {:?}", arg.mode)

Error:

internal error: entered unreachable code: x86 shouldn't be passing arguments by Pair(ArgAttributes { regular: NonNull | NoUndef, arg_ext: None, pointee_size: Size(0 bytes), pointee_align: Some(Align(1 bytes)) }, ArgAttributes { regular: NoUndef, arg_ext: None, pointee_size: Size(0 bytes), pointee_align: None })

I had a feeling that was unwarrantedly confident, but the error branch was preexisting, so it wasn't obvious how we'd start to hit it. Hmm...

[workingjubilee]

oh, because we're now making the Rust ABI go through it.

[workingjubilee]

@azhogin I have pushed a fix for the test omission and reverted extern "Rust" handling from the flag, you may wish to pull before you continue working. And feel free to drop/edit/squash/whatever those commits, naturally (though you may wish to branch off before you do so, as to have the extern "Rust" code around).

@ojeda Good catch for the only-x86! Yeah, it's completely unnecessary.

Let's try again, without the scope-creep into extern "Rust"? It builds library/core now with -Zregparm=3, at least.

[workingjubilee]

Thank you for your work on this, @azhogin. This needs a slight doc update if we keep it in its current state, but I'm fully content with this if it satisfies the extern "C" ABI requirements of the kernel.

It now seems obvious that @nbdd0121 was right: the goals of ABI compatibility and performance optimization are different. I realize it might be confusing, but c'est la vie. We shouldn't land this PR with extern "Rust" support for -Zregparm, unless we really want to be here for a few more weeks.

We also haven't even started to discuss other possibilities... like making extern "Rust" always be "fastcall-like" on x86.

[azhogin]

Thank you for your work on this, @azhogin. This needs a slight doc update if we keep it in its current state, but I'm fully content with this if it satisfies the extern "C" ABI requirements of the kernel.

Could it be better to just skip signatures (for Rust cc) with PassMode:: (Pair/Cast/Indirect with meta_attrs) ? Yes, it is still a temporary solution, but allows to support most of "Rust" calling conv signatures. I am just not sure if -Zregparm flag without Rust cc has valuable meaning for RfL.
I tested core compilation with this change (and its ok):

    // For types generating PassMode::Cast, PassMode::Indirect(meta_attrs) and PassMode::Pair,
    // InRegs will not be set.
    // Maybe, this is a FIXME
    let has_incompatibles = fn_abi.args.iter().any(
        |arg| matches!(arg.mode,
            PassMode::Pair { .. }
            | PassMode::Indirect { attrs: _, meta_attrs: Some(_), on_stack: _ }
            | PassMode::Cast { .. })
    );
    if has_incompatibles && rust_abi {
        return;
    }

[workingjubilee]

Could it be better to just skip signatures (for Rust cc) with PassMode:: (Pair/Cast/Indirect with meta_attrs) ? Yes, it is still a temporary solution, but allows to support most of "Rust" calling conv signatures. I am just not sure if -Zregparm flag without Rust cc has valuable meaning for RfL.

The most core need here is to be able to match the ABI of the kernel's C code as it is normally built, so that Rust's presence does not force deoptimizing that C code. This patch should address that in its current state, even if it does not affect the Rust calling convention at all.

Landing this patch first will allow us to still implement whatever Rust-ABI optimizations we want. But there is no need to rush such, and we should in fact scrutinize it more closely. This is not the first time the Rust and C ABIs have had the oddity of the C ABI being actually preferable for something, and it won't be the last. Especially not as a temporary situation.

[ojeda]

I am just not sure if -Zregparm flag without Rust cc has valuable meaning for RfL.

I think it has (well, assuming it actually has a positive effect for codegen for the Rust side too -- we will need to see if that is true or not), but like @workingjubilee says, it is more important to at least be able to build with the right ABI. With the docs that you added now clarifying where it applies, I think it is clear enough for users.

[workingjubilee]

Cool.

I believe this will serialize with

• rust_for_linux: -Zreg-struct-return commandline flag for X86 (#116973) #130777
• Set signext or zeroext for integer arguments on RISC-V and LoongArch64 #131955

So let's unblock those.

@bors r=workingjubilee,pnkfelix

[bors]

📌 Commit 37dc4ec has been approved by workingjubilee,pnkfelix

It is now in the queue for this repository.