Message ID | 20211026200346.3371750-1-adhemerval.zanella@linaro.org |
---|---|
Headers |
Return-Path: <libc-alpha-bounces+patchwork=sourceware.org@sourceware.org> X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id EC94E385800A for <patchwork@sourceware.org>; Tue, 26 Oct 2021 20:06:24 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org EC94E385800A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1635278785; bh=SM2Banl76FDsfLoMDCQtKXjxL+CvKsvONf55XXgLJig=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=wiCeVOvwhyt53lT/TC62gcnUp2UvguAcoFn8m/2j0gn1nhuX7M/AHPZXQl89sp17G 5g6/h8+dT1iAnwh/RRiEY6FpGXrVDOmPOs+gqZN349Zq9YBQWw2U7LaDFqhFrdhXNw UGgUOpSUt9YIACfhG3iamQMM5kHXNK03GowC5l0A= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-oi1-x234.google.com (mail-oi1-x234.google.com [IPv6:2607:f8b0:4864:20::234]) by sourceware.org (Postfix) with ESMTPS id 245763858036 for <libc-alpha@sourceware.org>; Tue, 26 Oct 2021 20:03:51 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 245763858036 Received: by mail-oi1-x234.google.com with SMTP id t4so260112oie.5 for <libc-alpha@sourceware.org>; Tue, 26 Oct 2021 13:03:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=SM2Banl76FDsfLoMDCQtKXjxL+CvKsvONf55XXgLJig=; b=sadOlMttbqtvtVh/ANzmDKTztRYmKTVZBBl7YGN4FlLiOxFbysKjXino87d0/+e8xh EoiHFyFfsWe2hDEVtps512+kt6ywit8hDCmTEYGqlm0sDcoBpGhQWixIcuG8aAHSXX47 oW+l5G1ya1gyoN312sOt9XkEkpD24NnB5T/6oOQC7C71nflS9kgKPujNPDyMLiN6krJV QWT1+gPD246JaCGAADSekJ2U1iaZPckW9X+F/URIMg68o0uEnwKIvE+FG5qcbCyK/Dlu ifwO9RkwETacvljJ1S7I+ZSHW4Jok/SFvjojD68KYrNS9ZDlWf3r0MqZviafvR0i1c05 QGMg== X-Gm-Message-State: AOAM533MzQq+05nilSQSMApNzPb5CgA+zwjyIc4GXzHcdQEy6m50dQvI B0owUCwFL+eRFtReph4yX5LTTx7B2cIZbw== X-Google-Smtp-Source: ABdhPJySzOQ6wC2TsIfwUJipgPoVc+eye7/JuO+gC/GcGqUlt8a6841LRmxY+4cwBpDfInSXTT2LfA== X-Received: by 2002:aca:5d07:: with SMTP id r7mr684097oib.138.1635278630292; Tue, 26 Oct 2021 13:03:50 -0700 (PDT) Received: from birita.. ([2804:431:c7cb:6672:308c:d549:2f1f:5bb6]) by smtp.gmail.com with ESMTPSA id l3sm2689062otu.6.2021.10.26.13.03.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Oct 2021 13:03:49 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH 0/3] Improve lld support and current status Date: Tue, 26 Oct 2021 17:03:43 -0300 Message-Id: <20211026200346.3371750-1-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-5.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, URIBL_BLACK autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list <libc-alpha.sourceware.org> List-Unsubscribe: <https://sourceware.org/mailman/options/libc-alpha>, <mailto:libc-alpha-request@sourceware.org?subject=unsubscribe> List-Archive: <https://sourceware.org/pipermail/libc-alpha/> List-Post: <mailto:libc-alpha@sourceware.org> List-Help: <mailto:libc-alpha-request@sourceware.org?subject=help> List-Subscribe: <https://sourceware.org/mailman/listinfo/libc-alpha>, <mailto:libc-alpha-request@sourceware.org?subject=subscribe> From: Adhemerval Zanella via Libc-alpha <libc-alpha@sourceware.org> Reply-To: Adhemerval Zanella <adhemerval.zanella@linaro.org> Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" <libc-alpha-bounces+patchwork=sourceware.org@sourceware.org> |
Series |
Improve lld support and current status
|
|
Message
Adhemerval Zanella Netto
Oct. 26, 2021, 8:03 p.m. UTC
Ths patch allows x86_64, i686, aarch64, powerpc64le, powerpc64, and powerpc to build, including the tests, with LLD packed in LLVM 13 [1]. For x86_64 and aarch64 there is no regression in testssuite. On i686 there is only one: FAIL: elf/ifuncmain6pie Which segfaults calling the 'foo' functions: Program received signal SIGSEGV, Segmentation fault. 0xfb010101 in ?? () (gdb) bt #0 0xfb010101 in ?? () #1 0xf7fba620 in call_foo () at ifuncmod6.c:18 #2 0xf7fc0857 in ?? () #3 0xf7e1b195 in __libc_start_call_main (main=main@entry=0xf7fc0830, argc=argc@entry=2, argv=argv@entry=0xffffc610) at ../sysdeps/nptl/libc_start_call_main.h:58 #4 0xf7e1b286 in __libc_start_main_impl (main=0xf7fc0830, argc=2, argv=0xffffc610, init=0x0, fini=0x0, rtld_fini=0xf7fe79b0 <_dl_fini>, stack_end=0xffffc60c) at ../csu/libc-start.c:409 #5 0xf7fc06d7 in ?? () The powerpc64le fails at libc.so start: (gdb) bt #0 0x00007ffff7f2a980 in __gep_setup___vmx__sigjmp_save () from /home/azanella/glibc/build/powerpc64le-linux-gnu-power9-lld/libc.so.6 #1 0x00007ffff7d7ac5c in __libc_start_call_main () from /home/azanella/glibc/build/powerpc64le-linux-gnu-power9-lld/libc.so.6 #2 0x00007ffff7d7ae90 in __libc_start_main_impl () from /home/azanella/glibc/build/powerpc64le-linux-gnu-power9-lld/libc.so.6 #3 0x0000000000000000 in ?? () (gdb) disas Dump of assembler code for function __gep_setup___vmx__sigjmp_save: => 0x00007ffff7f2a980 <+0>: .long 0x613ffe6 0x00007ffff7f2a984 <+4>: li r12,-1280 0x00007ffff7f2a988 <+8>: mtctr r12 0x00007ffff7f2a98c <+12>: bctr And powerpc64-linux-gnu fails on dynamic loaders start: (gdb) bt #0 0x00007ffff7ffa9a8 in _start () (gdb) disas Dump of assembler code for function _start: => 0x00007ffff7ffa9a8 <+0>: .long 0x0 0x00007ffff7ffa9ac <+4>: .long 0x0 0x00007ffff7ffa9b0 <+8>: .long 0x0 0x00007ffff7ffa9b4 <+12>: .long 0x0 The arm, sparcv9, mips, and riscv fail to build due different issues. On arm the loader fails to build: ld.so fails with ld.lld: error: relocation R_ARM_GOTOFF32 cannot be used against symbol _dl_argv; recompile with -fPIC >>> defined in >>> /home/azanella/Projects/glibc/build/arm-linux-gnueabihf-lld/elf/librtld.os >>> referenced by rtld.c:164 >>> /home/azanella/Projects/glibc/build/arm-linux-gnueabihf-lld/elf/librtld.os:(.text+0xA8) On sparcv9, lld handles --relax option different than ld.bfd: it throws an error instead of silent ignoring it: $ sparc64-glibc-linux-gnu-gcc -mcpu=niagara -fuse-ld=lld -Bclang+llvm-13.0.0-x86_64-linux-gnu-ubuntu-20.04/bin -g -O2 -fPIC -shared -fno-stack-protector -o conftest.so conftest.c -nostdlib -nostartfiles -Wl,-z,combreloc ld.lld: error: unknown argument '-relax' And even when -mno-relax is explicit add, lld does not support some relocations generated by GCC: $ sparc64-glibc-linux-gnu-readelf -Wr elf/librtld.os | grep _dl_skip_args | head -n2 00000000000000a0 0000012200000052 R_SPARC_GOTDATA_OP_HIX22 0000000000000058 _dl_skip_args + 0 00000000000000a4 0000012200000053 R_SPARC_GOTDATA_OP_LOX10 0000000000000058 _dl_skip_args + 0 The mips/mipsel also fails to build the loader: ld.lld: error: can't create dynamic relocation R_MIPS_32 against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output >>> defined in >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os >>> referenced by /home/azanella/toolchain/src/gcc/libgcc/libgcc2.c >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x1C) ld.lld: error: can't create dynamic relocation R_MIPS_32 against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output >>> defined in >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os >>> referenced by /home/azanella/toolchain/src/gcc/libgcc/libgcc2.c >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x54) Similar to mips64/mips64el: ld.lld: warning: ../sysdeps/unix/sysv/linux/setitimer.c:(function _dl_runtime_resolve: .text+0x18954): found R_MIPS_JALR relocation against non-function symbol . This is invalid and most likely a compiler bug. ld.lld: error: can't create dynamic relocation R_MIPS_64 against local symbol in readonly segment; recompile object files with -fPIC or pass '-Wl,-z,notext' to allow text relocations in the output >>> defined in >>> /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os >>> referenced by ../sysdeps/unix/sysv/linux/setitimer.c >>> /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x20) The riscv32/riscv64 fails to the loader, but it is due missing support to relaxation: ld.lld: error: /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2584:(.text+0xEC84): relocation R_RISCV_ALIGN requires unimplemented linker relaxation; recompile with -mno-relax ld.lld: error: /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2589:(.text+0x13DEC): relocation R_RISCV_ALIGN requires unimplemented linker relaxation; recompile with -mno-relax ld.lld: error: /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2589:(.text+0x13DEE): relocation R_RISCV_ALIGN requires unimplemented linker relaxation; recompile with -mno-rela Adding -mno-relax to sysdep-CFLAGS seems to be ineffective. [1] https://releases.llvm.org/download.html#13.0.0 Adhemerval Zanella (3): elf: Disable ifuncmain{1,5,5pic,5pie} when using LLD Fix LIBC_PROG_BINUTILS for -fuse-ld=lld Check if linker also support -mtls-dialect=gnu2 aclocal.m4 | 6 +++++- configure | 10 +++++++--- configure.ac | 4 ++-- elf/Makefile | 17 +++++++++++++---- 4 files changed, 27 insertions(+), 10 deletions(-)
Comments
On 2021-10-26, Adhemerval Zanella wrote: >Ths patch allows x86_64, i686, aarch64, powerpc64le, powerpc64, and >powerpc to build, including the tests, with LLD packed in LLVM 13 [1]. Thanks for checking! >For x86_64 and aarch64 there is no regression in testssuite. On >i686 there is only one: > > FAIL: elf/ifuncmain6pie > >Which segfaults calling the 'foo' functions: > > Program received signal SIGSEGV, Segmentation fault. > 0xfb010101 in ?? () > (gdb) bt > #0 0xfb010101 in ?? () > #1 0xf7fba620 in call_foo () at ifuncmod6.c:18 > #2 0xf7fc0857 in ?? () > #3 0xf7e1b195 in __libc_start_call_main (main=main@entry=0xf7fc0830, > argc=argc@entry=2, argv=argv@entry=0xffffc610) at > ../sysdeps/nptl/libc_start_call_main.h:58 > #4 0xf7e1b286 in __libc_start_main_impl (main=0xf7fc0830, argc=2, > argv=0xffffc610, init=0x0, fini=0x0, rtld_fini=0xf7fe79b0 <_dl_fini>, > stack_end=0xffffc60c) at ../csu/libc-start.c:409 > #5 0xf7fc06d7 in ?? () > >The powerpc64le fails at libc.so start: > > (gdb) bt > #0 0x00007ffff7f2a980 in __gep_setup___vmx__sigjmp_save () > from > /home/azanella/glibc/build/powerpc64le-linux-gnu-power9-lld/libc.so.6 > #1 0x00007ffff7d7ac5c in __libc_start_call_main () > from > /home/azanella/glibc/build/powerpc64le-linux-gnu-power9-lld/libc.so.6 > #2 0x00007ffff7d7ae90 in __libc_start_main_impl () > from > /home/azanella/glibc/build/powerpc64le-linux-gnu-power9-lld/libc.so.6 > #3 0x0000000000000000 in ?? () > (gdb) disas > Dump of assembler code for function __gep_setup___vmx__sigjmp_save: > => 0x00007ffff7f2a980 <+0>: .long 0x613ffe6 > 0x00007ffff7f2a984 <+4>: li r12,-1280 > 0x00007ffff7f2a988 <+8>: mtctr r12 > 0x00007ffff7f2a98c <+12>: bctr > >And powerpc64-linux-gnu fails on dynamic loaders start: > > (gdb) bt > #0 0x00007ffff7ffa9a8 in _start () > (gdb) disas > Dump of assembler code for function _start: > => 0x00007ffff7ffa9a8 <+0>: .long 0x0 > 0x00007ffff7ffa9ac <+4>: .long 0x0 > 0x00007ffff7ffa9b0 <+8>: .long 0x0 > 0x00007ffff7ffa9b4 <+12>: .long 0x0 CCed the powerpc maintainer. I am not familiar with debugging rtld for powerpc64... > >The arm, sparcv9, mips, and riscv fail to build due different issues. On >arm the loader fails to build: > > ld.so fails with > ld.lld: error: relocation R_ARM_GOTOFF32 cannot be used against symbol > _dl_argv; recompile with -fPIC > >>> defined in > >>> /home/azanella/Projects/glibc/build/arm-linux-gnueabihf-lld/elf/librtld.os > >>> referenced by rtld.c:164 > >>> /home/azanella/Projects/glibc/build/arm-linux-gnueabihf-lld/elf/librtld.os:(.text+0xA8) R_ARM_GOTOFF32 = S + A - GOT_ORG . Such a relocation referencing a preemptible symbol cannot be used. Filed GNU ld bug: https://sourceware.org/bugzilla/show_bug.cgi?id=28501 _dl_argv needs to be made non-preemptible. There are many ways: --dynamic-list, -Bsymbolic, hidden visibility. >On sparcv9, lld handles --relax option different than ld.bfd: it throws >an error instead of silent ignoring it: > > $ sparc64-glibc-linux-gnu-gcc -mcpu=niagara -fuse-ld=lld -Bclang+llvm-13.0.0-x86_64-linux-gnu-ubuntu-20.04/bin > -g -O2 -fPIC -shared -fno-stack-protector -o conftest.so conftest.c -nostdlib -nostartfiles -Wl,-z,combreloc > ld.lld: error: unknown argument '-relax' LLD's sparcv9 port is for retrocomputing fans:) and is far from usable (see https://lld.llvm.org/ "production quality"). The GOT/PLT support has quite a few issues (https://reviews.llvm.org/D102985). >And even when -mno-relax is explicit add, lld does not support some relocations >generated by GCC: > > $ sparc64-glibc-linux-gnu-readelf -Wr elf/librtld.os | grep _dl_skip_args > | head -n2 > 00000000000000a0 0000012200000052 R_SPARC_GOTDATA_OP_HIX22 > 0000000000000058 _dl_skip_args + 0 > 00000000000000a4 0000012200000053 R_SPARC_GOTDATA_OP_LOX10 > 0000000000000058 _dl_skip_args + 0 > >The mips/mipsel also fails to build the loader: > > ld.lld: error: can't create dynamic relocation R_MIPS_32 against local > symbol in readonly segment; recompile object files with -fPIC or pass > '-Wl,-z,notext' to allow text relocations in the output > >>> defined in > >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os > >>> referenced by /home/azanella/toolchain/src/gcc/libgcc/libgcc2.c > >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x1C) > > ld.lld: error: can't create dynamic relocation R_MIPS_32 against local > symbol in readonly segment; recompile object files with -fPIC or pass > '-Wl,-z,notext' to allow text relocations in the output > >>> defined in > >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os > >>> referenced by /home/azanella/toolchain/src/gcc/libgcc/libgcc2.c > >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x54) lld.llvm.org says "MIPS seems decent too." and I personally know really little about MIPS. However, I think this is likely a genuine issue somewhere in sysdeps/mips/ . Relocation processing is rigid and consistent in LLD. GNU ld may miss some error checking for some relocation types. >Similar to mips64/mips64el: > > ld.lld: warning: ../sysdeps/unix/sysv/linux/setitimer.c:(function > _dl_runtime_resolve: .text+0x18954): found R_MIPS_JALR relocation > against non-function symbol . This is invalid and most likely a compiler > bug. > ld.lld: error: can't create dynamic relocation R_MIPS_64 against local > symbol in readonly segment; recompile object files with -fPIC or pass > '-Wl,-z,notext' to allow text relocations in the output > >>> defined in > >>> /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os > >>> referenced by ../sysdeps/unix/sysv/linux/setitimer.c > >>> /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x20) I took a look at LLD's R_MIPS_JALR code and I am inclined to trust it reporting a genuine issue. >The riscv32/riscv64 fails to the loader, but it is due missing support >to relaxation: > > ld.lld: error: > /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2584:(.text+0xEC84): > relocation R_RISCV_ALIGN requires unimplemented linker relaxation; > recompile with -mno-relax > ld.lld: error: > /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2589:(.text+0x13DEC): > relocation R_RISCV_ALIGN requires unimplemented linker relaxation; > recompile with -mno-relax > ld.lld: error: > /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2589:(.text+0x13DEE): > relocation R_RISCV_ALIGN requires unimplemented linker relaxation; > recompile with -mno-rela LLD does not implement RISC-V linker relaxation. R_RISCV_ALIGN has weird linker unfriendly semantics: simply ignoring the relocation type can break semantics (https://github.com/riscv-non-isa/riscv-elf-psabi-doc/issues/183). >Adding -mno-relax to sysdep-CFLAGS seems to be ineffective. So sysdep-CFLAGS is not a catch-all option affecting all C compiles... > >[1] https://releases.llvm.org/download.html#13.0.0 > >Adhemerval Zanella (3): > elf: Disable ifuncmain{1,5,5pic,5pie} when using LLD > Fix LIBC_PROG_BINUTILS for -fuse-ld=lld > Check if linker also support -mtls-dialect=gnu2 > > aclocal.m4 | 6 +++++- > configure | 10 +++++++--- > configure.ac | 4 ++-- > elf/Makefile | 17 +++++++++++++---- > 4 files changed, 27 insertions(+), 10 deletions(-) > >-- >2.32.0 >
On 26/10/2021 17:33, Fangrui Song wrote: >> The arm, sparcv9, mips, and riscv fail to build due different issues. On >> arm the loader fails to build: >> >> ld.so fails with >> ld.lld: error: relocation R_ARM_GOTOFF32 cannot be used against symbol >> _dl_argv; recompile with -fPIC >> >>> defined in >> >>> /home/azanella/Projects/glibc/build/arm-linux-gnueabihf-lld/elf/librtld.os >> >>> referenced by rtld.c:164 >> >>> /home/azanella/Projects/glibc/build/arm-linux-gnueabihf-lld/elf/librtld.os:(.text+0xA8) > > R_ARM_GOTOFF32 = S + A - GOT_ORG . > Such a relocation referencing a preemptible symbol cannot be used. > > Filed GNU ld bug: https://sourceware.org/bugzilla/show_bug.cgi?id=28501 > > _dl_argv needs to be made non-preemptible. There are many ways: --dynamic-list, -Bsymbolic, hidden visibility. I think it is something else, the symbols is defined as hidden (output of preprocessor): extern __typeof (_dl_argv) _dl_argv __asm__ ("" "__GI__dl_argv") __attribute__ ((visibility ("hidden"))); However, hidden is set only the internal symbol: $ arm-glibc-linux-gnueabi-readelf -Ws elf/librtld.os | grep _dl_argv 1378: 00000000 4 OBJECT GLOBAL DEFAULT 6 _dl_argv 1470: 00000000 4 OBJECT GLOBAL HIDDEN 6 __GI__dl_argv And on rtld.c assembly we have the directive to use the internal definition: .set _dl_argv,__GI__dl_argv So why lld is not binding the usage to internal to the local hidden alias? > >> On sparcv9, lld handles --relax option different than ld.bfd: it throws >> an error instead of silent ignoring it: >> >> $ sparc64-glibc-linux-gnu-gcc -mcpu=niagara -fuse-ld=lld -Bclang+llvm-13.0.0-x86_64-linux-gnu-ubuntu-20.04/bin >> -g -O2 -fPIC -shared -fno-stack-protector -o conftest.so conftest.c -nostdlib -nostartfiles -Wl,-z,combreloc >> ld.lld: error: unknown argument '-relax' > > LLD's sparcv9 port is for retrocomputing fans:) and is far from usable > (see https://lld.llvm.org/ "production quality"). > The GOT/PLT support has quite a few issues (https://reviews.llvm.org/D102985). Right, but lld is still showing a different ld.bfd semantic that might trigger other issues where on platforms where neither --relax nor -no-relax it supported ld.bfd accepts both argument but ignore it [1]. At least on sparc, gcc does pass --relax and expects that static linker just ignore if it is no supported. [1] https://sourceware.org/binutils/docs-2.37/ld.html > >> And even when -mno-relax is explicit add, lld does not support some relocations >> generated by GCC: >> >> $ sparc64-glibc-linux-gnu-readelf -Wr elf/librtld.os | grep _dl_skip_args >> | head -n2 >> 00000000000000a0 0000012200000052 R_SPARC_GOTDATA_OP_HIX22 >> 0000000000000058 _dl_skip_args + 0 >> 00000000000000a4 0000012200000053 R_SPARC_GOTDATA_OP_LOX10 >> 0000000000000058 _dl_skip_args + 0 >> >> The mips/mipsel also fails to build the loader: >> >> ld.lld: error: can't create dynamic relocation R_MIPS_32 against local >> symbol in readonly segment; recompile object files with -fPIC or pass >> '-Wl,-z,notext' to allow text relocations in the output >> >>> defined in >> >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os >> >>> referenced by /home/azanella/toolchain/src/gcc/libgcc/libgcc2.c >> >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x1C) >> >> ld.lld: error: can't create dynamic relocation R_MIPS_32 against local >> symbol in readonly segment; recompile object files with -fPIC or pass >> '-Wl,-z,notext' to allow text relocations in the output >> >>> defined in >> >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os >> >>> referenced by /home/azanella/toolchain/src/gcc/libgcc/libgcc2.c >> >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x54) > > lld.llvm.org says "MIPS seems decent too." and I personally know really > little about MIPS. > > However, I think this is likely a genuine issue somewhere in sysdeps/mips/ . > Relocation processing is rigid and consistent in LLD. > GNU ld may miss some error checking for some relocation types. > >> Similar to mips64/mips64el: >> >> ld.lld: warning: ../sysdeps/unix/sysv/linux/setitimer.c:(function >> _dl_runtime_resolve: .text+0x18954): found R_MIPS_JALR relocation >> against non-function symbol . This is invalid and most likely a compiler >> bug. >> ld.lld: error: can't create dynamic relocation R_MIPS_64 against local >> symbol in readonly segment; recompile object files with -fPIC or pass >> '-Wl,-z,notext' to allow text relocations in the output >> >>> defined in >> >>> /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os >> >>> referenced by ../sysdeps/unix/sysv/linux/setitimer.c >> >>> /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x20) > > I took a look at LLD's R_MIPS_JALR code and I am inclined to trust it > reporting a genuine issue. I give you that loader code is tricky, but it does not really explain why lld is failing to link the eh_frame. It *might* due some internal assembly routines, but at least from logs it does not seem so. > >> The riscv32/riscv64 fails to the loader, but it is due missing support >> to relaxation: >> >> ld.lld: error: >> /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2584:(.text+0xEC84): >> relocation R_RISCV_ALIGN requires unimplemented linker relaxation; >> recompile with -mno-relax >> ld.lld: error: >> /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2589:(.text+0x13DEC): >> relocation R_RISCV_ALIGN requires unimplemented linker relaxation; >> recompile with -mno-relax >> ld.lld: error: >> /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2589:(.text+0x13DEE): >> relocation R_RISCV_ALIGN requires unimplemented linker relaxation; >> recompile with -mno-rela > > LLD does not implement RISC-V linker relaxation. > R_RISCV_ALIGN has weird linker unfriendly semantics: > simply ignoring the relocation type can break semantics > (https://github.com/riscv-non-isa/riscv-elf-psabi-doc/issues/183). > >> Adding -mno-relax to sysdep-CFLAGS seems to be ineffective. > > So sysdep-CFLAGS is not a catch-all option affecting all C compiles... In fact it is, the problem was the autogenerated syscalls built from the assembly. I could avoid the R_RISCV_ALIGN with: diff --git a/sysdeps/riscv/Makefile b/sysdeps/riscv/Makefile index 20a9968106..28dbd89cb0 100644 --- a/sysdeps/riscv/Makefile +++ b/sysdeps/riscv/Makefile @@ -5,3 +5,9 @@ endif # RISC-V's assembler also needs to know about PIC as it changes the definition # of some assembler macros. ASFLAGS-.os += $(pic-ccflag) + +# lld does not implement R_RISCV_ALIGN relaxation optimization. +ifeq (yes,$(with-lld)) +ASFLAGS-.os += -Wa,-mno-relax +sysdep-CFLAGS += -mno-relax +endif However it does fail later for the libc.so with: ld.lld: warning: attempt to reassign symbol 'fcntl' of version 'GLIBC_2.27' to version 'GLIBC_2.28' ld.lld: warning: attempt to reassign symbol '__sched_get_priority_min' of version 'GLIBC_2.27' to version 'GLIBC_PRIVATE' ld.lld: warning: attempt to reassign symbol '__sched_get_priority_max' of version 'GLIBC_2.27' to version 'GLIBC_PRIVATE' ld.lld: error: relocation R_RISCV_RVC_JUMP cannot be used against symbol __sigsetjmp; recompile with -fPIC >>> defined in /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean >>> referenced by iconv.c:62 (/home/azanella/Projects/glibc/glibc-git/iconv/iconv.c:62) >>> /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean:(__GI__setjmp) ld.lld: error: relocation R_RISCV_JAL cannot be used against symbol exit; recompile with -fPIC >>> defined in /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean >>> referenced by gconv_open.c:89 (/home/azanella/Projects/glibc/glibc-git/iconv/gconv_open.c:89) >>> /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean:(__start_context) collect2: error: ld returned 1 exit status
Fangrui Song via Libc-alpha <libc-alpha@sourceware.org> writes: > On 2021-10-26, Adhemerval Zanella wrote: >>The powerpc64le fails at libc.so start: >> >> (gdb) bt >> #0 0x00007ffff7f2a980 in __gep_setup___vmx__sigjmp_save () >> from >> /home/azanella/glibc/build/powerpc64le-linux-gnu-power9-lld/libc.so.6 >> #1 0x00007ffff7d7ac5c in __libc_start_call_main () >> from >> /home/azanella/glibc/build/powerpc64le-linux-gnu-power9-lld/libc.so.6 This is a local call from libc.so to libc.so. It should be using the local entry point for __vmx__sigjmp_save. >> #2 0x00007ffff7d7ae90 in __libc_start_main_impl () >> from >> /home/azanella/glibc/build/powerpc64le-linux-gnu-power9-lld/libc.so.6 >> #3 0x0000000000000000 in ?? () >> (gdb) disas >> Dump of assembler code for function __gep_setup___vmx__sigjmp_save: >> => 0x00007ffff7f2a980 <+0>: .long 0x613ffe6 >> 0x00007ffff7f2a984 <+4>: li r12,-1280 This is a pla, but this GDB isn't able to disassemble it. This instruction shouldn't be used unless when configuring glibc using --with-cpu=power10. >>And powerpc64-linux-gnu fails on dynamic loaders start: >> >> (gdb) bt >> #0 0x00007ffff7ffa9a8 in _start () >> (gdb) disas >> Dump of assembler code for function _start: >> => 0x00007ffff7ffa9a8 <+0>: .long 0x0 >> 0x00007ffff7ffa9ac <+4>: .long 0x0 >> 0x00007ffff7ffa9b0 <+8>: .long 0x0 >> 0x00007ffff7ffa9b4 <+12>: .long 0x0 I've been told that LLD doesn't support the powerpc ELFv1 yet. > CCed the powerpc maintainer. Thanks!
Tulio Magno Quites Machado Filho via Libc-alpha <libc-alpha@sourceware.org> writes: > Fangrui Song via Libc-alpha <libc-alpha@sourceware.org> writes: > >> On 2021-10-26, Adhemerval Zanella wrote: >>> #2 0x00007ffff7d7ae90 in __libc_start_main_impl () >>> from >>> /home/azanella/glibc/build/powerpc64le-linux-gnu-power9-lld/libc.so.6 >>> #3 0x0000000000000000 in ?? () >>> (gdb) disas >>> Dump of assembler code for function __gep_setup___vmx__sigjmp_save: >>> => 0x00007ffff7f2a980 <+0>: .long 0x613ffe6 >>> 0x00007ffff7f2a984 <+4>: li r12,-1280 > > This is a pla, but this GDB isn't able to disassemble it. This instruction > shouldn't be used unless when configuring glibc using --with-cpu=power10. I can reproduce this issue even when configuring glibc with --with-cpu=power9 --disable-multi-arch, which means the build should not have any Power10 instructions.
On 2021-10-27, Tulio Magno Quites Machado Filho wrote: >Fangrui Song via Libc-alpha <libc-alpha@sourceware.org> writes: > >> On 2021-10-26, Adhemerval Zanella wrote: >>>The powerpc64le fails at libc.so start: >>> >>> (gdb) bt >>> #0 0x00007ffff7f2a980 in __gep_setup___vmx__sigjmp_save () >>> from >>> /home/azanella/glibc/build/powerpc64le-linux-gnu-power9-lld/libc.so.6 >>> #1 0x00007ffff7d7ac5c in __libc_start_call_main () >>> from >>> /home/azanella/glibc/build/powerpc64le-linux-gnu-power9-lld/libc.so.6 > >This is a local call from libc.so to libc.so. >It should be using the local entry point for __vmx__sigjmp_save. > >>> #2 0x00007ffff7d7ae90 in __libc_start_main_impl () >>> from >>> /home/azanella/glibc/build/powerpc64le-linux-gnu-power9-lld/libc.so.6 >>> #3 0x0000000000000000 in ?? () >>> (gdb) disas >>> Dump of assembler code for function __gep_setup___vmx__sigjmp_save: >>> => 0x00007ffff7f2a980 <+0>: .long 0x613ffe6 >>> 0x00007ffff7f2a984 <+4>: li r12,-1280 > >This is a pla, but this GDB isn't able to disassemble it. This instruction >shouldn't be used unless when configuring glibc using --with-cpu=power10. > >>>And powerpc64-linux-gnu fails on dynamic loaders start: >>> >>> (gdb) bt >>> #0 0x00007ffff7ffa9a8 in _start () >>> (gdb) disas >>> Dump of assembler code for function _start: >>> => 0x00007ffff7ffa9a8 <+0>: .long 0x0 >>> 0x00007ffff7ffa9ac <+4>: .long 0x0 >>> 0x00007ffff7ffa9b0 <+8>: .long 0x0 >>> 0x00007ffff7ffa9b4 <+12>: .long 0x0 > >I've been told that LLD doesn't support the powerpc ELFv1 yet. You are right:) The incomplete PowerPC64 ELF v1 support was removed by https://reviews.llvm.org/D46316 (2018-05). Is there remaining ELF v1 usage for powerpc64le in glibc? (I only know some 64-bit ELF v2 and some 32-bit, but know really little about 64-bit ELF v1.) Legacy ELF v1 is also the issue that powerpc64 big-endian Linux kernel cannot be linked with LLD (https://github.com/ClangBuiltLinux/linux/issues/602). There is little benefit retroactively adding ELF v1 support to LLD, so migration to ELF v2 will be nice :) >> CCed the powerpc maintainer. > >Thanks! > >-- >Tulio Magno
On 2021-10-27, Adhemerval Zanella wrote: > > >On 26/10/2021 17:33, Fangrui Song wrote: >>> The arm, sparcv9, mips, and riscv fail to build due different issues. On >>> arm the loader fails to build: >>> >>> ld.so fails with >>> ld.lld: error: relocation R_ARM_GOTOFF32 cannot be used against symbol >>> _dl_argv; recompile with -fPIC >>> >>> defined in >>> >>> /home/azanella/Projects/glibc/build/arm-linux-gnueabihf-lld/elf/librtld.os >>> >>> referenced by rtld.c:164 >>> >>> /home/azanella/Projects/glibc/build/arm-linux-gnueabihf-lld/elf/librtld.os:(.text+0xA8) >> >> R_ARM_GOTOFF32 = S + A - GOT_ORG . >> Such a relocation referencing a preemptible symbol cannot be used. >> >> Filed GNU ld bug: https://sourceware.org/bugzilla/show_bug.cgi?id=28501 >> >> _dl_argv needs to be made non-preemptible. There are many ways: --dynamic-list, -Bsymbolic, hidden visibility. > >I think it is something else, the symbols is defined as hidden (output of >preprocessor): > > extern __typeof (_dl_argv) _dl_argv __asm__ ("" "__GI__dl_argv") __attribute__ ((visibility ("hidden"))); > >However, hidden is set only the internal symbol: > > $ arm-glibc-linux-gnueabi-readelf -Ws elf/librtld.os | grep _dl_argv > 1378: 00000000 4 OBJECT GLOBAL DEFAULT 6 _dl_argv > 1470: 00000000 4 OBJECT GLOBAL HIDDEN 6 __GI__dl_argv > >And on rtld.c assembly we have the directive to use the internal definition: > > .set _dl_argv,__GI__dl_argv > >So why lld is not binding the usage to internal to the local hidden alias? I use the Debian package g++-arm-linux-gnueabihf (nowadays glibc build needs a C++ compiler even if I don't build tests...). mkdir -p out/arm; cd out/arm ../../configure --prefix=/tmp/glibc/arm --host=arm-linux-gnueabihf make -j 50 Here is GNU ld produced elf/librtld.os: % readelf -Ws elf/librtld.os | grep _dl_argv 1582: 00000000 4 OBJECT GLOBAL HIDDEN 13 __GI__dl_argv 1603: 00000000 4 OBJECT GLOBAL DEFAULT 13 _dl_argv So LLD's librtld.os matches GNU ld for the two symbols. The .set directive sets value/type but not binding/visibility. Perhaps sysdeps/arm/dl-machine.h should use .word __GI__dl_argv(GOTOFF) instead of .word _dl_argv(GOTOFF) >> >>> On sparcv9, lld handles --relax option different than ld.bfd: it throws >>> an error instead of silent ignoring it: >>> >>> $ sparc64-glibc-linux-gnu-gcc -mcpu=niagara -fuse-ld=lld -Bclang+llvm-13.0.0-x86_64-linux-gnu-ubuntu-20.04/bin >>> -g -O2 -fPIC -shared -fno-stack-protector -o conftest.so conftest.c -nostdlib -nostartfiles -Wl,-z,combreloc >>> ld.lld: error: unknown argument '-relax' >> >> LLD's sparcv9 port is for retrocomputing fans:) and is far from usable >> (see https://lld.llvm.org/ "production quality"). >> The GOT/PLT support has quite a few issues (https://reviews.llvm.org/D102985). > >Right, but lld is still showing a different ld.bfd semantic that might trigger >other issues where on platforms where neither --relax nor -no-relax it supported >ld.bfd accepts both argument but ignore it [1]. > >At least on sparc, gcc does pass --relax and expects that static linker just >ignore if it is no supported. > >[1] https://sourceware.org/binutils/docs-2.37/ld.html LLD's sparcv9 port is quite immature, so I cannot even advertise it. A large portion should be rewritten but the sparc port is only cared by very few retrocomputing fans now.. >> >>> And even when -mno-relax is explicit add, lld does not support some relocations >>> generated by GCC: >>> >>> $ sparc64-glibc-linux-gnu-readelf -Wr elf/librtld.os | grep _dl_skip_args >>> | head -n2 >>> 00000000000000a0 0000012200000052 R_SPARC_GOTDATA_OP_HIX22 >>> 0000000000000058 _dl_skip_args + 0 >>> 00000000000000a4 0000012200000053 R_SPARC_GOTDATA_OP_LOX10 >>> 0000000000000058 _dl_skip_args + 0 >>> >>> The mips/mipsel also fails to build the loader: >>> >>> ld.lld: error: can't create dynamic relocation R_MIPS_32 against local >>> symbol in readonly segment; recompile object files with -fPIC or pass >>> '-Wl,-z,notext' to allow text relocations in the output >>> >>> defined in >>> >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os >>> >>> referenced by /home/azanella/toolchain/src/gcc/libgcc/libgcc2.c >>> >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x1C) >>> >>> ld.lld: error: can't create dynamic relocation R_MIPS_32 against local >>> symbol in readonly segment; recompile object files with -fPIC or pass >>> '-Wl,-z,notext' to allow text relocations in the output >>> >>> defined in >>> >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os >>> >>> referenced by /home/azanella/toolchain/src/gcc/libgcc/libgcc2.c >>> >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x54) >> >> lld.llvm.org says "MIPS seems decent too." and I personally know really >> little about MIPS. >> >> However, I think this is likely a genuine issue somewhere in sysdeps/mips/ . >> Relocation processing is rigid and consistent in LLD. >> GNU ld may miss some error checking for some relocation types. >> >>> Similar to mips64/mips64el: >>> >>> ld.lld: warning: ../sysdeps/unix/sysv/linux/setitimer.c:(function >>> _dl_runtime_resolve: .text+0x18954): found R_MIPS_JALR relocation >>> against non-function symbol . This is invalid and most likely a compiler >>> bug. >>> ld.lld: error: can't create dynamic relocation R_MIPS_64 against local >>> symbol in readonly segment; recompile object files with -fPIC or pass >>> '-Wl,-z,notext' to allow text relocations in the output >>> >>> defined in >>> >>> /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os >>> >>> referenced by ../sysdeps/unix/sysv/linux/setitimer.c >>> >>> /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x20) >> >> I took a look at LLD's R_MIPS_JALR code and I am inclined to trust it >> reporting a genuine issue. > >I give you that loader code is tricky, but it does not really explain why >lld is failing to link the eh_frame. It *might* due some internal assembly >routines, but at least from logs it does not seem so. Hope Joseph can answer the question. If we have easy-to-follow instructions building glibc mips with LLD, I can forward it to Simon Atanasyan (MIPS code owner of llvm-project and the major contributor of LLD's MIPS port)... But see below (for riscv/), I suspect this is a glibc sysdeps/ problem. >> >>> The riscv32/riscv64 fails to the loader, but it is due missing support >>> to relaxation: >>> >>> ld.lld: error: >>> /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2584:(.text+0xEC84): >>> relocation R_RISCV_ALIGN requires unimplemented linker relaxation; >>> recompile with -mno-relax >>> ld.lld: error: >>> /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2589:(.text+0x13DEC): >>> relocation R_RISCV_ALIGN requires unimplemented linker relaxation; >>> recompile with -mno-relax >>> ld.lld: error: >>> /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2589:(.text+0x13DEE): >>> relocation R_RISCV_ALIGN requires unimplemented linker relaxation; >>> recompile with -mno-rela >> >> LLD does not implement RISC-V linker relaxation. >> R_RISCV_ALIGN has weird linker unfriendly semantics: >> simply ignoring the relocation type can break semantics >> (https://github.com/riscv-non-isa/riscv-elf-psabi-doc/issues/183). >> >>> Adding -mno-relax to sysdep-CFLAGS seems to be ineffective. >> >> So sysdep-CFLAGS is not a catch-all option affecting all C compiles... > >In fact it is, the problem was the autogenerated syscalls built from >the assembly. I could avoid the R_RISCV_ALIGN with: > >diff --git a/sysdeps/riscv/Makefile b/sysdeps/riscv/Makefile >index 20a9968106..28dbd89cb0 100644 >--- a/sysdeps/riscv/Makefile >+++ b/sysdeps/riscv/Makefile >@@ -5,3 +5,9 @@ endif > # RISC-V's assembler also needs to know about PIC as it changes the definition > # of some assembler macros. > ASFLAGS-.os += $(pic-ccflag) >+ >+# lld does not implement R_RISCV_ALIGN relaxation optimization. >+ifeq (yes,$(with-lld)) >+ASFLAGS-.os += -Wa,-mno-relax >+sysdep-CFLAGS += -mno-relax >+endif > > >However it does fail later for the libc.so with: > >ld.lld: warning: attempt to reassign symbol 'fcntl' of version 'GLIBC_2.27' to version 'GLIBC_2.28' >ld.lld: warning: attempt to reassign symbol '__sched_get_priority_min' of version 'GLIBC_2.27' to version 'GLIBC_PRIVATE' >ld.lld: warning: attempt to reassign symbol '__sched_get_priority_max' of version 'GLIBC_2.27' to version 'GLIBC_PRIVATE' >ld.lld: error: relocation R_RISCV_RVC_JUMP cannot be used against symbol __sigsetjmp; recompile with -fPIC >>>> defined in /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean >>>> referenced by iconv.c:62 (/home/azanella/Projects/glibc/glibc-git/iconv/iconv.c:62) >>>> /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean:(__GI__setjmp) > >ld.lld: error: relocation R_RISCV_JAL cannot be used against symbol exit; recompile with -fPIC >>>> defined in /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean >>>> referenced by gconv_open.c:89 (/home/azanella/Projects/glibc/glibc-git/iconv/gconv_open.c:89) >>>> /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean:(__start_context) There is a GNU ld issue: https://sourceware.org/bugzilla/show_bug.cgi?id=28509 It incorrectly allows non-PLT non-GOT relocation referencing a preemptible symbol. LLD uses consistent categories (e.g. R_PC / R_PLT_PC) to describe relocation types. It has consistent diagnostic across all ports it supports. GNU ld uses dispatches for each relocation type and may miss some diagnostics. >collect2: error: ld returned 1 exit status
On 2021-10-27, Fangrui Song wrote: > >On 2021-10-27, Adhemerval Zanella wrote: >>>>Similar to mips64/mips64el: >>>> >>>> ld.lld: warning: ../sysdeps/unix/sysv/linux/setitimer.c:(function >>>> _dl_runtime_resolve: .text+0x18954): found R_MIPS_JALR relocation >>>> against non-function symbol . This is invalid and most likely a compiler >>>> bug. >>>> ld.lld: error: can't create dynamic relocation R_MIPS_64 against local >>>> symbol in readonly segment; recompile object files with -fPIC or pass >>>> '-Wl,-z,notext' to allow text relocations in the output >>>> >>> defined in >>>> >>> /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os >>>> >>> referenced by ../sysdeps/unix/sysv/linux/setitimer.c >>>> >>> /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x20) >>> >>>I took a look at LLD's R_MIPS_JALR code and I am inclined to trust it >>>reporting a genuine issue. >> >>I give you that loader code is tricky, but it does not really explain why >>lld is failing to link the eh_frame. It *might* due some internal assembly >>routines, but at least from logs it does not seem so. > >Hope Joseph can answer the question. > >If we have easy-to-follow instructions building glibc mips with LLD, I can >forward it to Simon Atanasyan (MIPS code owner of llvm-project and the major >contributor of LLD's MIPS port)... > >But see below (for riscv/), I suspect this is a glibc sysdeps/ problem. jrtc27 told me that BFD mips has a hack to rewrite R_MIPS_64 in .eh_frame to avoid such a R_MIPS_64 diagnostic. LLD doesn't implement it. Clang's mips port emits relative relocations in .eh_frame (DW_EH_PE_pcrel | dwarf::DW_EH_PE_sdata4) to avoid the linker issue: https://reviews.llvm.org/D72228 Lack of 64-bit PC-relative relocation is a longstanding problem for mips. LLVM integrated assembler and linker actually support `.quad foo - .` via composed relocations, but it is still unsupported in GNU as (feature request: https://sourceware.org/bugzilla/show_bug.cgi?id=26222) I know really little about MIPS but I'd wish that GNU as implements .quad foo-. and GCC migrates to PC-relative relocations like Clang.
On 27/10/2021 22:06, Fangrui Song wrote: > > On 2021-10-27, Adhemerval Zanella wrote: >> >> >> On 26/10/2021 17:33, Fangrui Song wrote: >>>> The arm, sparcv9, mips, and riscv fail to build due different issues. On >>>> arm the loader fails to build: >>>> >>>> ld.so fails with >>>> ld.lld: error: relocation R_ARM_GOTOFF32 cannot be used against symbol >>>> _dl_argv; recompile with -fPIC >>>> >>> defined in >>>> >>> /home/azanella/Projects/glibc/build/arm-linux-gnueabihf-lld/elf/librtld.os >>>> >>> referenced by rtld.c:164 >>>> >>> /home/azanella/Projects/glibc/build/arm-linux-gnueabihf-lld/elf/librtld.os:(.text+0xA8) >>> >>> R_ARM_GOTOFF32 = S + A - GOT_ORG . >>> Such a relocation referencing a preemptible symbol cannot be used. >>> >>> Filed GNU ld bug: https://sourceware.org/bugzilla/show_bug.cgi?id=28501 >>> >>> _dl_argv needs to be made non-preemptible. There are many ways: --dynamic-list, -Bsymbolic, hidden visibility. >> >> I think it is something else, the symbols is defined as hidden (output of >> preprocessor): >> >> extern __typeof (_dl_argv) _dl_argv __asm__ ("" "__GI__dl_argv") __attribute__ ((visibility ("hidden"))); >> >> However, hidden is set only the internal symbol: >> >> $ arm-glibc-linux-gnueabi-readelf -Ws elf/librtld.os | grep _dl_argv >> 1378: 00000000 4 OBJECT GLOBAL DEFAULT 6 _dl_argv >> 1470: 00000000 4 OBJECT GLOBAL HIDDEN 6 __GI__dl_argv >> >> And on rtld.c assembly we have the directive to use the internal definition: >> >> .set _dl_argv,__GI__dl_argv >> >> So why lld is not binding the usage to internal to the local hidden alias? > > I use the Debian package g++-arm-linux-gnueabihf (nowadays glibc build needs a C++ compiler even if I don't build tests...). You can set an empty CXX to avoid it ("CXX=") at configure time. > > mkdir -p out/arm; cd out/arm > ../../configure --prefix=/tmp/glibc/arm --host=arm-linux-gnueabihf > make -j 50 > > Here is GNU ld produced elf/librtld.os: > > % readelf -Ws elf/librtld.os | grep _dl_argv > 1582: 00000000 4 OBJECT GLOBAL HIDDEN 13 __GI__dl_argv > 1603: 00000000 4 OBJECT GLOBAL DEFAULT 13 _dl_argv > > So LLD's librtld.os matches GNU ld for the two symbols. > > > > The .set directive sets value/type but not binding/visibility. > Perhaps sysdeps/arm/dl-machine.h should use > > .word __GI__dl_argv(GOTOFF) > > instead of > .word _dl_argv(GOTOFF) > It think it should be ok. So it seems that ld.bfd is being forgiving here and binding to the internal alias. >>> >>>> On sparcv9, lld handles --relax option different than ld.bfd: it throws >>>> an error instead of silent ignoring it: >>>> >>>> $ sparc64-glibc-linux-gnu-gcc -mcpu=niagara -fuse-ld=lld -Bclang+llvm-13.0.0-x86_64-linux-gnu-ubuntu-20.04/bin >>>> -g -O2 -fPIC -shared -fno-stack-protector -o conftest.so conftest.c -nostdlib -nostartfiles -Wl,-z,combreloc >>>> ld.lld: error: unknown argument '-relax' >>> >>> LLD's sparcv9 port is for retrocomputing fans:) and is far from usable >>> (see https://lld.llvm.org/ "production quality"). >>> The GOT/PLT support has quite a few issues (https://reviews.llvm.org/D102985). >> >> Right, but lld is still showing a different ld.bfd semantic that might trigger >> other issues where on platforms where neither --relax nor -no-relax it supported >> ld.bfd accepts both argument but ignore it [1]. >> >> At least on sparc, gcc does pass --relax and expects that static linker just >> ignore if it is no supported. >> >> [1] https://sourceware.org/binutils/docs-2.37/ld.html > > LLD's sparcv9 port is quite immature, so I cannot even advertise it. > A large portion should be rewritten but the sparc port is only cared by very few > retrocomputing fans now.. It would be good if we could detect it at configure time and just warn that linker is not sufficient to build glibc. > >>> >>>> And even when -mno-relax is explicit add, lld does not support some relocations >>>> generated by GCC: >>>> >>>> $ sparc64-glibc-linux-gnu-readelf -Wr elf/librtld.os | grep _dl_skip_args >>>> | head -n2 >>>> 00000000000000a0 0000012200000052 R_SPARC_GOTDATA_OP_HIX22 >>>> 0000000000000058 _dl_skip_args + 0 >>>> 00000000000000a4 0000012200000053 R_SPARC_GOTDATA_OP_LOX10 >>>> 0000000000000058 _dl_skip_args + 0 >>>> >>>> The mips/mipsel also fails to build the loader: >>>> >>>> ld.lld: error: can't create dynamic relocation R_MIPS_32 against local >>>> symbol in readonly segment; recompile object files with -fPIC or pass >>>> '-Wl,-z,notext' to allow text relocations in the output >>>> >>> defined in >>>> >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os >>>> >>> referenced by /home/azanella/toolchain/src/gcc/libgcc/libgcc2.c >>>> >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x1C) >>>> >>>> ld.lld: error: can't create dynamic relocation R_MIPS_32 against local >>>> symbol in readonly segment; recompile object files with -fPIC or pass >>>> '-Wl,-z,notext' to allow text relocations in the output >>>> >>> defined in >>>> >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os >>>> >>> referenced by /home/azanella/toolchain/src/gcc/libgcc/libgcc2.c >>>> >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x54) >>> >>> lld.llvm.org says "MIPS seems decent too." and I personally know really >>> little about MIPS. >>> >>> However, I think this is likely a genuine issue somewhere in sysdeps/mips/ . >>> Relocation processing is rigid and consistent in LLD. >>> GNU ld may miss some error checking for some relocation types. >>> >>>> Similar to mips64/mips64el: >>>> >>>> ld.lld: warning: ../sysdeps/unix/sysv/linux/setitimer.c:(function >>>> _dl_runtime_resolve: .text+0x18954): found R_MIPS_JALR relocation >>>> against non-function symbol . This is invalid and most likely a compiler >>>> bug. >>>> ld.lld: error: can't create dynamic relocation R_MIPS_64 against local >>>> symbol in readonly segment; recompile object files with -fPIC or pass >>>> '-Wl,-z,notext' to allow text relocations in the output >>>> >>> defined in >>>> >>> /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os >>>> >>> referenced by ../sysdeps/unix/sysv/linux/setitimer.c >>>> >>> /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x20) >>> >>> I took a look at LLD's R_MIPS_JALR code and I am inclined to trust it >>> reporting a genuine issue. >> >> I give you that loader code is tricky, but it does not really explain why >> lld is failing to link the eh_frame. It *might* due some internal assembly >> routines, but at least from logs it does not seem so. > > Hope Joseph can answer the question. > > If we have easy-to-follow instructions building glibc mips with LLD, I can > forward it to Simon Atanasyan (MIPS code owner of llvm-project and the major > contributor of LLD's MIPS port)... > > But see below (for riscv/), I suspect this is a glibc sysdeps/ problem. > >>> >>>> The riscv32/riscv64 fails to the loader, but it is due missing support >>>> to relaxation: >>>> >>>> ld.lld: error: >>>> /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2584:(.text+0xEC84): >>>> relocation R_RISCV_ALIGN requires unimplemented linker relaxation; >>>> recompile with -mno-relax >>>> ld.lld: error: >>>> /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2589:(.text+0x13DEC): >>>> relocation R_RISCV_ALIGN requires unimplemented linker relaxation; >>>> recompile with -mno-relax >>>> ld.lld: error: >>>> /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2589:(.text+0x13DEE): >>>> relocation R_RISCV_ALIGN requires unimplemented linker relaxation; >>>> recompile with -mno-rela >>> >>> LLD does not implement RISC-V linker relaxation. >>> R_RISCV_ALIGN has weird linker unfriendly semantics: >>> simply ignoring the relocation type can break semantics >>> (https://github.com/riscv-non-isa/riscv-elf-psabi-doc/issues/183). >>> >>>> Adding -mno-relax to sysdep-CFLAGS seems to be ineffective. >>> >>> So sysdep-CFLAGS is not a catch-all option affecting all C compiles... >> >> In fact it is, the problem was the autogenerated syscalls built from >> the assembly. I could avoid the R_RISCV_ALIGN with: >> >> diff --git a/sysdeps/riscv/Makefile b/sysdeps/riscv/Makefile >> index 20a9968106..28dbd89cb0 100644 >> --- a/sysdeps/riscv/Makefile >> +++ b/sysdeps/riscv/Makefile >> @@ -5,3 +5,9 @@ endif >> # RISC-V's assembler also needs to know about PIC as it changes the definition >> # of some assembler macros. >> ASFLAGS-.os += $(pic-ccflag) >> + >> +# lld does not implement R_RISCV_ALIGN relaxation optimization. >> +ifeq (yes,$(with-lld)) >> +ASFLAGS-.os += -Wa,-mno-relax >> +sysdep-CFLAGS += -mno-relax >> +endif >> >> >> However it does fail later for the libc.so with: >> >> ld.lld: warning: attempt to reassign symbol 'fcntl' of version 'GLIBC_2.27' to version 'GLIBC_2.28' >> ld.lld: warning: attempt to reassign symbol '__sched_get_priority_min' of version 'GLIBC_2.27' to version 'GLIBC_PRIVATE' >> ld.lld: warning: attempt to reassign symbol '__sched_get_priority_max' of version 'GLIBC_2.27' to version 'GLIBC_PRIVATE' >> ld.lld: error: relocation R_RISCV_RVC_JUMP cannot be used against symbol __sigsetjmp; recompile with -fPIC >>>>> defined in /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean >>>>> referenced by iconv.c:62 (/home/azanella/Projects/glibc/glibc-git/iconv/iconv.c:62) >>>>> /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean:(__GI__setjmp) >> >> ld.lld: error: relocation R_RISCV_JAL cannot be used against symbol exit; recompile with -fPIC >>>>> defined in /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean >>>>> referenced by gconv_open.c:89 (/home/azanella/Projects/glibc/glibc-git/iconv/gconv_open.c:89) >>>>> /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean:(__start_context) > > There is a GNU ld issue: https://sourceware.org/bugzilla/show_bug.cgi?id=28509 > It incorrectly allows non-PLT non-GOT relocation referencing a preemptible symbol. > > > LLD uses consistent categories (e.g. R_PC / R_PLT_PC) to describe relocation types. > It has consistent diagnostic across all ports it supports. > GNU ld uses dispatches for each relocation type and may miss some diagnostics. > >> collect2: error: ld returned 1 exit status Indeed these are the same ARM issues, we will need to use internal symbols.
Fangrui Song via Libc-alpha <libc-alpha@sourceware.org> writes: > You are right:) The incomplete PowerPC64 ELF v1 support was removed by > https://reviews.llvm.org/D46316 (2018-05). Is there remaining ELF v1 > usage for powerpc64le in glibc? No. We reuse code whenever possible, but that's all. However, without this support we can't build glibc for powerpc64 big-endian (aka. powerpc64-linux).
On 27/10/2021 22:18, Fangrui Song wrote: > On 2021-10-27, Fangrui Song wrote: >> >> On 2021-10-27, Adhemerval Zanella wrote: >>>>> Similar to mips64/mips64el: >>>>> >>>>> ld.lld: warning: ../sysdeps/unix/sysv/linux/setitimer.c:(function >>>>> _dl_runtime_resolve: .text+0x18954): found R_MIPS_JALR relocation >>>>> against non-function symbol . This is invalid and most likely a compiler >>>>> bug. >>>>> ld.lld: error: can't create dynamic relocation R_MIPS_64 against local >>>>> symbol in readonly segment; recompile object files with -fPIC or pass >>>>> '-Wl,-z,notext' to allow text relocations in the output >>>>> >>> defined in >>>>> >>> /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os >>>>> >>> referenced by ../sysdeps/unix/sysv/linux/setitimer.c >>>>> >>> /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x20) >>>> >>>> I took a look at LLD's R_MIPS_JALR code and I am inclined to trust it >>>> reporting a genuine issue. >>> >>> I give you that loader code is tricky, but it does not really explain why >>> lld is failing to link the eh_frame. It *might* due some internal assembly >>> routines, but at least from logs it does not seem so. >> >> Hope Joseph can answer the question. >> >> If we have easy-to-follow instructions building glibc mips with LLD, I can >> forward it to Simon Atanasyan (MIPS code owner of llvm-project and the major >> contributor of LLD's MIPS port)... >> >> But see below (for riscv/), I suspect this is a glibc sysdeps/ problem. > > jrtc27 told me that BFD mips has a hack to rewrite R_MIPS_64 in > .eh_frame to avoid such a R_MIPS_64 diagnostic. LLD doesn't implement it. > > Clang's mips port emits relative relocations in .eh_frame (DW_EH_PE_pcrel | dwarf::DW_EH_PE_sdata4) > to avoid the linker issue: https://reviews.llvm.org/D72228 > > Lack of 64-bit PC-relative relocation is a longstanding problem for mips. > LLVM integrated assembler and linker actually support `.quad foo - .` > via composed relocations, but it is still unsupported in GNU as > (feature request: https://sourceware.org/bugzilla/show_bug.cgi?id=26222) > > I know really little about MIPS but I'd wish that GNU as implements > .quad foo-. and GCC migrates to PC-relative relocations like Clang. In any case, it would be good to a configure check to warn uses that R_MIPS_64 on .eh_frame might no be supported and bail out if lld idea is to not implement any workaround.