clang, sparc: missing compiler command line flags for linux kernel build

[arndb]

Bugzilla Link	41447
Version	8.0
OS	Linux
Blocks	#4440
CC	@arndb,@nickdesaulniers

Extended Description

I tried to build the Linux kernel for sparc32 and sparc64. Both work with few changes, but the Makefiles pass a number of options to the compiler that clang does not understand.

In particular, linux-5.1 passes

-mno-fpu
-fcall-used-g5
-fcall-used-g7
-mcpu=ultrasparc
-mcmodel=medlow
-ffixed-g4
-ffixed-g5
-fcall-used-g7
-mv8plus

Removing all of these from the Makefile gets it to compile almost all files, but I'm fairly sure this will not boot because some of the options are in fact required.

[brad0]

For OpenBSD/sparc64 we ran into -mno-fpu being missing when building the kernel.

[llvmbot]

@llvm/issue-subscribers-clang-driver

[ihiasi]

I see in the Sparc GCC options document that :

-mno-fpu
-msoft-float
Generate output containing library calls for floating point. Warning: the requisite libraries are not available for all SPARC targets. Normally the facilities of the machine’s usual C compiler are used, but this cannot be done directly in cross-compilation. You must make your own arrangements to provide suitable library functions for cross-compilation. The embedded targets ‘sparc-*-aout’ and ‘sparclite-*-*’ do provide software floating-point support.

I changed arch/sparc/Makefile to use soft-float (supported by LLVM) and seems to compile (fails on the other flags you listed). FYI.

[ihiasi]

@arndb, @nickdesaulniers - I'm hitting the following after I turn off the -ffixed/-fcall flags :

+ ld.lld -m elf64_sparc -z noexecstack --build-id=sha1 -X --script=./arch/sparc/kernel/vmlinux.lds --strip-debug -o .tmp_vmlinux.kallsyms1 --whole-archive arch/sparc/kernel/head_64.o init/built-in.a usr/built-in.a arch/sparc/built-in.a kernel/built-in.a certs/built-in.a mm/built-in.a fs/built-in.a ipc/built-in.a security/built-in.a crypto/built-in.a block/built-in.a io_uring/built-in.a lib/built-in.a arch/sparc/prom/built-in.a arch/sparc/lib/built-in.a lib/lib.a arch/sparc/prom/lib.a arch/sparc/lib/lib.a drivers/built-in.a sound/built-in.a net/built-in.a virt/built-in.a arch/sparc/power/built-in.a .vmlinux.export.o --no-whole-archive --start-group --end-group
ld.lld: error: arch/sparc/kernel/head_64.o:(.text+0xccc): unknown relocation (41) against symbol user_rtt_fill_fixup_common
ld.lld: error: arch/sparc/kernel/head_64.o:(.text+0xcd4): unknown relocation (41) against symbol user_rtt_fill_fixup_common
ld.lld: error: arch/sparc/kernel/head_64.o:(.text+0xcdc): unknown relocation (41) against symbol user_rtt_fill_fixup_common
ld.lld: error: arch/sparc/kernel/head_64.o:(.text+0xe04): unknown relocation (41) against symbol etrap
ld.lld: error: arch/sparc/kernel/head_64.o:(.text+0xe14): unknown relocation (41) against symbol rtrap
ld.lld: error: arch/sparc/kernel/head_64.o:(.text+0xf04): unknown relocation (41) against symbol etrap
ld.lld: error: arch/sparc/kernel/head_64.o:(.text+0xf14): unknown relocation (41) against symbol rtrap
ld.lld: error: arch/sparc/kernel/head_64.o:(.text+0xf34): unknown relocation (41) against symbol etrap
ld.lld: error: arch/sparc/kernel/head_64.o:(.text+0xf5c): unknown relocation (41) against symbol rtrap
ld.lld: error: arch/sparc/kernel/head_64.o:(.text+0xf70): unknown relocation (41) against symbol rtrap
ld.lld: error: arch/sparc/kernel/head_64.o:(.text+0xf90): unknown relocation (41) against symbol etrap
ld.lld: error: arch/sparc/kernel/head_64.o:(.text+0xfbc): unknown relocation (41) against symbol rtrap
ld.lld: error: arch/sparc/kernel/head_64.o:(.text+0xfcc): unknown relocation (41) against symbol rtrap
ld.lld: error: arch/sparc/kernel/head_64.o:(function do_fpdis: .text+0xffc): unknown relocation (41) against symbol etrap
ld.lld: error: arch/sparc/kernel/head_64.o:(function do_fpdis: .text+0x1008): unknown relocation (41) against symbol rtrap
ld.lld: error: arch/sparc/kernel/head_64.o:(function fp_other_bounce: .text+0x1228): unknown relocation (41) against symbol rtrap
ld.lld: error: arch/sparc/kernel/head_64.o:(function do_fptrap: .text+0x147c): unknown relocation (41) against symbol etrap
ld.lld: error: arch/sparc/kernel/head_64.o:(function utrap_trap: .text+0x157c): unknown relocation (41) against symbol etrap
ld.lld: error: arch/sparc/kernel/head_64.o:(function utrap_trap: .text+0x1590): unknown relocation (41) against symbol rtrap
ld.lld: error: arch/sparc/kernel/head_64.o:(function __spitfire_access_error: .text+0x1668): unknown relocation (41) against symbol etraptl1
ld.lld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors)
make: *** [../Makefile:1168: vmlinux] Error 1

Any ideas what might cause this ? I'd like to get the Linux SPARC LLVM port to completion.

[arndb]

I would recommend you try ld.bfd instead of ld.lld to reduce the number of variables in your build setup compared to the gcc setup.

For https://github.com/ClangBuiltLinux, we pretty much ignore sparc now, as there is very little interest in the architecture upstream compared to the other clang supported targets.

[MaskRay]

The lld support state is similar: the port is quite incomplete. I am also concerned of sparc doing things too differently and lld support would introduce maintenance burden.

[ihiasi]

actually, if I compile ONLY for 64-bit :

make -j 20 CC=clang LD=ld.bfd ARCH=sparc64 CROSS_COMPILE=sparc64-linux-gnu- O=./build_clang_ldbfd_64 sparc64_defconfig
cd build_clang_ldbfd_64
make -j 20 CC=clang LD=ld.bfd ARCH=sparc64 CROSS_COMPILE=sparc64-linux-gnu- V=1 vmlinux modules zImage

I'm getting different errors (regardless of if I use LLVM=1 or CC=clang LD=ld.bfd - since the errors are in the LLVM assembly reader/parser) :

../arch/sparc/include/asm/irqflags_64.h:33:3: error: invalid operand for instruction
                "wrpr   %0, %%pil"
                ^
<inline asm>:1:12: note: instantiated into assembly here
        wrpr    %i0, %pil
                     ^
...
...
../arch/sparc/include/asm/irqflags_64.h:53:3: error: invalid operand for instruction
                "wrpr   0, %%pil"
                ^
<inline asm>:1:7: note: instantiated into assembly here
        wrpr    0, %pil

This is happening because SparcInstrInfo.td declares wrpr as :

// Section A.62 - Write Privileged Register Instructions
let Predicates = [HasV9] in {
  def WRPRrr : F3_1<2, 0b110010,
                   (outs PRRegs:$rd), (ins IntRegs:$rs1, IntRegs:$rs2),
                   "wrpr $rs1, $rs2, $rd", []>;
  def WRPRri : F3_2<2, 0b110010,
                   (outs PRRegs:$rd), (ins IntRegs:$rs1, simm13Op:$simm13),
                   "wrpr $rs1, $simm13, $rd", []>;
}

ie, no wrpr <register>, <register> or wrpr <immediate>, <register>.
Both the original SparcV9 spec and the Oracle updated SparcV9 spec do not mention wrpr imm, %reg - but ld.bfd seem to accept it and I see the same in the mtrace Sparc code, for example.

I'll fix that and update here.

[ihiasi]

The lld support state is similar: the port is quite incomplete. I am also concerned of sparc doing things too differently and lld support would introduce maintenance burden.

The hope is that we won't introduce maintenance burden since Sparc is in many ways very similar to Mips for example and Mips support is already in and pretty strong.

[brad0]

@koachan

[koachan]

From a quick look about the various issues in the report:

-mfpu/-mno-fpu

This is already implemented in clang as -mhard-float/-msoft-float, but I guess an alias could be made for those.

-mcmodel=medlow

I don't think clang has any option to set the 64-bit SPARC code model now, but from the assembly output I'm guessing that currently it compiles in a mode corresponding to GCC's medmid. If Linux has a hard requirement on medlow code model then yeah, this is a problem.
(Though from my understanding medmid is a strict superset of medlow so this should not be a problem in practice)

-mv8plus

32-bit V8+ ABI extensions is also unsupported by clang right now :(

-fcall-used-g5
-fcall-used-g7
-ffixed-g4
-ffixed-g5
-fcall-used-g7

I have no idea what do those options do and I can't find it in GCC's documentation, so no comment on those.
As an aside, I find it very weird that Linux passes both -mcmodel (which is a 64-bit only option) and -mv8plus (which is a 32-bit only option). Here on my local GCC 12.1, passing the option for the wrong target bitness results in it throwing an error, at least.

As for the instructions, seems like GNU assembler has those two nonstandard aliases for WRPR:

wrpr %reg, %rd -> wrpr %reg, %g0, %rd
wrpr  imm, %rd -> wrpr  %g0, imm, %rd

God knows why did they decide to place the %g0 in separate places in the expansion but I guess we just have to put up with it.

About LLD, I'm unfamiliar with it and its internals but at least I can confirm that as of now it can only work on very basic use cases, so binutils' LD is still the way to go for SPARC linking needs.

[brad0]

As an aside, I find it very weird that Linux passes both -mcmodel (which is a 64-bit only option) and -mv8plus (which is a 32-bit only option). Here on my local GCC 12.1, passing the option for the wrong target bitness results in it throwing an error, at least.

The OP said above he was building both sparc and sparc64 kernels. So I think the confusion is the fact the options are all clumped together instead of being separated into the options being passed to a sparc kernel vs a sparc64 kernel.

[brad0]

From a quick look about the various issues in the report:

-mfpu/-mno-fpu

This is already implemented in clang as -mhard-float/-msoft-float, but I guess an alias could be made for those.

Please provide a diff. Both *BSD and Linux use this.