[RFC] Introduce pt-compat-stubs and use it to replace pt-vfork. (Architecture maintainer feedback wanted.)
Commit Message
I am looking into the possibility of eliminating all of the duplicate
function definitions from libpthread, replacing them with
properly-tagged weak compatibility symbols that just call the
definition in libc. Because one of the duplicated functions is vfork,
the calls to libc absolutely must reuse the stack frame (a "sibcall"
in GCC internals jargon), and on several important targets, GCC does
not implement sibcalls, or only implements them for intra-module
calls. But we only need to implement a single special case,
sibcalling a function with exactly the same signature, from
immediately after the caller's own entry point; so doing it by hand in
assembly language is not a crazy notion. I believe I have managed to
turn the trick for all currently-supported targets *except* powerpc64,
where I cannot find a way around a linker sanity check. The object
file contains
0000000000000000 <.__pstub_vfork>:
0: 48 00 00 00 b 0 <.__pstub_vfork>
0: R_PPC64_REL24 __libc_vfork
4: 60 00 00 00 nop
and the linker throws this error:
nptl/libpthread_pic.a(pt-compat-stubs.os): In function `__pstub_vfork':
.../pt-compat-stubs.S:56:(.text+0x0): call to `__libc_vfork@@GLIBC_PRIVATE'
lacks nop, can't restore toc; recompile with -fPIC
You can see that there *is* a nop after the call. GNU ld lumps
several different circumstances into this error message, one of which
is just that R_PPC64_REL24 to a symbol outside the current module has
been applied to a plain branch (as opposed to branch-and-link)
instruction. *Normally* this would be unsafe, but in this specific
case, I believe it would work fine: __pstub_vfork is never called from
within libpthread, so __libc_vfork will ultimately return to a
TOC-restoration shim within __pstub_vfork's own caller. There's
some code in bfd/elf64-ppc.c that makes it sound like there might be a
way to bypass the sanity check,
if (stub_entry->stub_type == ppc_stub_plt_call
&& !htab->opd_abi
&& htab->params->plt_localentry0 != 0
&& is_elfv2_localentry0 (&h->elf))
{
/* The function doesn't use or change r2. */
can_plt_call = TRUE;
}
but I can't figure out how to make that condition be true. Any help
would be appreciated.
Anyhow, I thought I'd get some feedback on the changes to the other
architectures as well, before I go any further. This patch just
converts the existing vfork stub, so that review can focus on the new
sysdep.h SIBCALL macros.
* sysdeps/generic/pt-compat-stubs.S: New file.
* nptl/pt-vfork.c, sysdeps/unix/sysv/linux/aarch64/pt-vfork.c
* sysdeps/unix/sysv/linux/m68k/pt-vfork.c
* sysdeps/unix/sysv/linux/tile/pt-vfork.c
Remove file.
* nptl/Makefile (libpthread-routines): Remove pt-vfork, add
pt-compat-stubs.
(libpthread-shared-only-routines): Add pt-compat-stubs.
* posix/vfork.c: Define __libc_vfork as well as __vfork and vfork.
* sysdeps/generic/sysdep.h (SIBCALL): New macro to perform
sibling calls; the generic definition errors out if used.
* sysdeps/aarch64/sysdep.h, sysdeps/arm/sysdep.h
* sysdeps/hppa/sysdep.h, sysdeps/ia64/sysdep.h
* sysdeps/m68k/sysdep.h, sysdeps/microblaze/sysdep.h
* sysdeps/nios2/sysdep.h, sysdeps/powerpc/powerpc32/sysdep.h
* sysdeps/powerpc/powerpc64/sysdep.h, sysdeps/s390/s390-32/sysdep.h
* sysdeps/s390/s390-64/sysdep.h, sysdeps/tile/sysdep.h
* sysdeps/unix/alpha/sysdep.h, sysdeps/unix/mips/mips32/sysdep.h
* sysdeps/unix/mips/mips64/n32/sysdep.h
* sysdeps/unix/mips/mips64/n64/sysdep.h
* sysdeps/unix/sysv/linux/riscv/sysdep.h
* sysdeps/x86/sysdep.h
Provide appropriate architecture-specific definition of SIBCALL.
---
nptl/Makefile | 5 ++-
nptl/pt-vfork.c | 65 ------------------------------
posix/vfork.c | 6 +--
sysdeps/aarch64/sysdep.h | 9 +++++
sysdeps/arm/sysdep.h | 9 +++++
sysdeps/generic/pt-compat-stubs.S | 64 +++++++++++++++++++++++++++++
sysdeps/generic/sysdep.h | 13 +++++-
sysdeps/hppa/sysdep.h | 10 +++++
sysdeps/ia64/sysdep.h | 9 +++++
sysdeps/m68k/sysdep.h | 9 +++++
sysdeps/microblaze/sysdep.h | 9 +++++
sysdeps/nios2/sysdep.h | 23 ++++++++++-
sysdeps/powerpc/powerpc32/sysdep.h | 8 ++++
sysdeps/powerpc/powerpc64/sysdep.h | 9 +++++
sysdeps/s390/s390-32/sysdep.h | 9 +++++
sysdeps/s390/s390-64/sysdep.h | 9 +++++
sysdeps/sh/sysdep.h | 14 +++++++
sysdeps/sparc/sysdep.h | 9 +++++
sysdeps/tile/sysdep.h | 9 +++++
sysdeps/unix/alpha/sysdep.h | 10 +++++
sysdeps/unix/mips/mips32/sysdep.h | 25 ++++++++++++
sysdeps/unix/mips/mips64/n32/sysdep.h | 31 ++++++++++++++
sysdeps/unix/mips/mips64/n64/sysdep.h | 23 +++++++++++
sysdeps/unix/sysv/linux/aarch64/pt-vfork.c | 54 -------------------------
sysdeps/unix/sysv/linux/m68k/pt-vfork.c | 1 -
sysdeps/unix/sysv/linux/riscv/sysdep.h | 9 +++++
sysdeps/unix/sysv/linux/tile/pt-vfork.c | 1 -
sysdeps/x86/sysdep.h | 9 +++++
28 files changed, 333 insertions(+), 128 deletions(-)
delete mode 100644 nptl/pt-vfork.c
create mode 100644 sysdeps/generic/pt-compat-stubs.S
delete mode 100644 sysdeps/unix/sysv/linux/aarch64/pt-vfork.c
delete mode 100644 sysdeps/unix/sysv/linux/m68k/pt-vfork.c
delete mode 100644 sysdeps/unix/sysv/linux/tile/pt-vfork.c
Comments
On Mär 20 2018, Zack Weinberg <zackw@panix.com> wrote:
> if (stub_entry->stub_type == ppc_stub_plt_call
> && !htab->opd_abi
> && htab->params->plt_localentry0 != 0
> && is_elfv2_localentry0 (&h->elf))
> {
> /* The function doesn't use or change r2. */
> can_plt_call = TRUE;
> }
>
> but I can't figure out how to make that condition be true. Any help
> would be appreciated.
This is about ELFv2, which is powerpc64le only.
Andreas.
On Wed, Mar 21, 2018 at 4:53 AM, Andreas Schwab <schwab@suse.de> wrote:
> On Mär 20 2018, Zack Weinberg <zackw@panix.com> wrote:
>> but I can't figure out how to make that condition be true. Any help
>> would be appreciated.
>
> This is about ELFv2, which is powerpc64le only.
I should have been more specific: I get the "call lacks nop, can't
restore toc" failure on _both_ big- and little-endian PowerPC 64-bit
configurations. Do you know how to work around the problem on either,
or, ideally, both configurations?
zw
I'd suggest to file a binutils bug.
Andreas.
On Wed, Mar 21, 2018 at 9:59 AM, Andreas Schwab <schwab@suse.de> wrote:
> I'd suggest to file a binutils bug.
I suppose that's an option, but I'd rather look for approaches
entirely within libc first.
On Mär 21 2018, Zack Weinberg <zackw@panix.com> wrote:
> On Wed, Mar 21, 2018 at 9:59 AM, Andreas Schwab <schwab@suse.de> wrote:
>> I'd suggest to file a binutils bug.
>
> I suppose that's an option, but I'd rather look for approaches
> entirely within libc first.
Alan will be happy to help you there, I'm sure.
Andreas.
On Tue, Mar 20, 2018 at 09:05:51PM -0400, Zack Weinberg wrote:
> I believe I have managed to
> turn the trick for all currently-supported targets *except* powerpc64,
> where I cannot find a way around a linker sanity check. The object
> file contains
>
> 0000000000000000 <.__pstub_vfork>:
> 0: 48 00 00 00 b 0 <.__pstub_vfork>
> 0: R_PPC64_REL24 __libc_vfork
> 4: 60 00 00 00 nop
>
> and the linker throws this error:
>
> nptl/libpthread_pic.a(pt-compat-stubs.os): In function `__pstub_vfork':
> .../pt-compat-stubs.S:56:(.text+0x0): call to `__libc_vfork@@GLIBC_PRIVATE'
> lacks nop, can't restore toc; recompile with -fPIC
>
> You can see that there *is* a nop after the call.
Yes, the error message isn't the best in this case. At one stage GNU
ld did allow code like you're writing. ie. a useless nop to be
replaced with a toc restore insn that won't ever be executed. It
was a way for an assembly programmer to say they knew what they were
doing, please don't complain "can't restore toc". I took that feature
out after getting reports of crashes due to bad toc pointer values..
> GNU ld lumps
> several different circumstances into this error message, one of which
> is just that R_PPC64_REL24 to a symbol outside the current module has
> been applied to a plain branch (as opposed to branch-and-link)
> instruction. *Normally* this would be unsafe, but in this specific
> case, I believe it would work fine: __pstub_vfork is never called from
> within libpthread, so __libc_vfork will ultimately return to a
> TOC-restoration shim within __pstub_vfork's own caller.
Yes, if you can guarantee no local calls to __pstub_vfork then the
trick should work OK.
> There's
> some code in bfd/elf64-ppc.c that makes it sound like there might be a
> way to bypass the sanity check,
>
> if (stub_entry->stub_type == ppc_stub_plt_call
> && !htab->opd_abi
> && htab->params->plt_localentry0 != 0
> && is_elfv2_localentry0 (&h->elf))
> {
> /* The function doesn't use or change r2. */
> can_plt_call = TRUE;
> }
>
> but I can't figure out how to make that condition be true. Any help
> would be appreciated.
No, that code won't apply. You'd need to link with --plt-localentry
and __libc_vfork isn't localentry:0 anyway.
937: 00000000000e5f64 52 FUNC GLOBAL DEFAULT [<localentry>: 8] 10 __libc_vfork@@GLIBC_PRIVATE
Do you need lazy linking of the __libc_vfork PLT entry? If not, you
could just implement the stub as an indirect call to __libc_vfork.
For ELFv2, something like:
.text
.global __pstub_vfork
.type __pstub_vfork,@function
__pstub_vfork:
0:
addis 12,12,1f-0b@ha
addi 12,12,1f-0b@l
ld 12,0(12)
mtctr 12
bctr
.size __pstub_vfork,.-0b
.data
1:
.quad __libc_vfork
ELFv1 code would look like:
.section .opd,"aw",@progbits
.global __pstub_vfork
.type __pstub_vfork,@function
__pstub_vfork:
.quad 0f,.TOC.,0
.text
0:
addis 11,2,1f-0b@ha
addi 11,11,1f-0b@l
ld 11,0(11)
ld 12,0(11)
mtctr 12
ld 2,8(11)
ld 11,16(11)
bctr
.size __pstub_vfork,.-0b
.data
1:
.quad __libc_vfork
Note the addi and ld look like they ought to be combined into one
insn, but we lack a REL16_LO_DS reloc to make that kosher.
On Thu, Mar 22, 2018 at 7:47 PM, Alan Modra <amodra@gmail.com> wrote:
>
> Yes, the error message isn't the best in this case. At one stage GNU
> ld did allow code like you're writing. ie. a useless nop to be
> replaced with a toc restore insn that won't ever be executed. It
> was a way for an assembly programmer to say they knew what they were
> doing, please don't complain "can't restore toc". I took that feature
> out after getting reports of crashes due to bad toc pointer values..
Yeah, this is subtle enough that I think that was probably the right call...
...
> Do you need lazy linking of the __libc_vfork PLT entry? If not, you
> could just implement the stub as an indirect call to __libc_vfork.
Lazy linking would be _better_, since these stubs exist only for
compatibility with old executables and shouldn't get called very
often, but I think we can live with eager linking. It's probably not
a measurable difference in the time it takes to load libpthread.so
anyway.
> For ELFv2, something like:
...
Thanks, I was trying to work these out myself (see the message I sent
to binutils@) and got stuck because the ABI spec uses
pseudo-relocation notation that the assembler doesn't actually accept.
It was also not clear to me that ELFv2 scrapped function descriptors;
you wouldn't happen to know where there's a list of *changes* from v1
to v2, would you?
> .text
> 0:
> addis 11,2,1f-0b@ha
> addi 11,11,1f-0b@l
> ld 11,0(11)
> ld 12,0(11)
> mtctr 12
> ld 2,8(11)
> ld 11,16(11)
Is this third load (to r11) from the function descriptor required?
That's the static chain slot, which C doesn't use, I thought...
zw
On Thu, Mar 22, 2018 at 7:47 PM, Alan Modra <amodra@gmail.com> wrote:
> ELFv1 code would look like:
>
> .section .opd,"aw",@progbits
> .global __pstub_vfork
> .type __pstub_vfork,@function
> __pstub_vfork:
> .quad 0f,.TOC.,0
>
> .text
> 0:
> addis 11,2,1f-0b@ha
> addi 11,11,1f-0b@l
Is this really correct? 1f-0b here is the offset from the beginning
of the function to the global it wants to reference in .data, but r2
is the TOC pointer, not the beginning of the function.
zw
On Fri, Mar 23, 2018 at 03:32:09PM -0400, Zack Weinberg wrote:
> On Thu, Mar 22, 2018 at 7:47 PM, Alan Modra <amodra@gmail.com> wrote:
> > ELFv1 code would look like:
> >
> > .section .opd,"aw",@progbits
> > .global __pstub_vfork
> > .type __pstub_vfork,@function
> > __pstub_vfork:
> > .quad 0f,.TOC.,0
> >
> > .text
> > 0:
> > addis 11,2,1f-0b@ha
> > addi 11,11,1f-0b@l
>
> Is this really correct? 1f-0b here is the offset from the beginning
> of the function to the global it wants to reference in .data, but r2
> is the TOC pointer, not the beginning of the function.
No, it's broken, sorry. Not enough editing when copying the ELFv2
code..
addis 11,2,1f@toc@ha
addi 11,11,1f@toc@l
ld 11,0(11)
or better, since we do have the required relocs in this case
addis 11,2,1f@toc@ha
ld 11,1f@toc@l(11)
Hmm, I also missed a ".p2align 3" when emitting the address to .data,
and didn't write a proper OPD entry. :-( You could also put the
function pointer in .toc which would be better for relro and huge
ELFv1 shared libraries (not the case here but nicer example code for
others to copy), or in .data.rel.ro.
Revised ELFv1 code
.section .opd,"aw",@progbits
.global __pstub_vfork
.type __pstub_vfork,@function
__pstub_vfork:
.quad 0f,.TOC.@tocbase,0
.text
0:
addis 11,2,1f@toc@ha
ld 11,1f@toc@l(11)
ld 12,0(11)
mtctr 12
ld 2,8(11)
bctr
.size __pstub_vfork,.-0b
.section .toc,"aw",@progbits
.p2align 3
1:
.quad __libc_vfork
ELFv2 code
.text
.global __pstub_vfork
.type __pstub_vfork,@function
__pstub_vfork:
0:
addis 12,12,1f-0b@ha
addi 12,12,1f-0b@l
ld 12,0(12)
mtctr 12
bctr
.size __pstub_vfork,.-0b
.section .data.rel.ro,"aw",@progbits
.p2align 3
1:
.quad __libc_vfork
In reply to your other email, the third load is the static chain and
can be omitted for C. I also happen to have been working on inline
plt call support for powerpc ld, which will give you the ability to
write stubs that support lazy linking.
@@ -120,9 +120,10 @@ libpthread-routines = nptl-init vars events version pt-interp \
cancellation \
lowlevellock \
lll_timedlock_wait lll_timedwait_tid \
- pt-fork pt-vfork \
+ pt-fork \
$(pthread-compat-wrappers) \
pt-raise pt-system \
+ pt-compat-stubs \
flockfile ftrylockfile funlockfile \
sigaction \
herrno res pt-allocrtsig \
@@ -146,7 +147,7 @@ libpthread-routines = nptl-init vars events version pt-interp \
# pthread_setresgid
libpthread-shared-only-routines = version pt-interp pt-allocrtsig \
- unwind-forcedunwind
+ pt-compat-stubs unwind-forcedunwind
# Since cancellation handling is in large parts handled using exceptions
# we have to compile some files with exception handling enabled, some
deleted file mode 100644
@@ -1,65 +0,0 @@
-/* vfork ABI-compatibility entry points for libpthread.
- Copyright (C) 2014-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-#include <unistd.h>
-#include <shlib-compat.h>
-
-/* libpthread used to have its own vfork implementation that differed
- from libc's only in having a pointless micro-optimization. There
- is no longer any use to having a separate copy in libpthread, but
- the historical ABI requires it. For static linking, there is no
- need to provide anything here--the libc version will be linked in.
- For shared library ABI compatibility, there must be __vfork and
- vfork symbols in libpthread.so; so we define them using IFUNC to
- redirect to the libc function. */
-
-/* Note! If the architecture doesn't support IFUNC, then we need an
- alternate target-specific mechanism to implement this. So we just
- assume IFUNC here and require that the target override this file
- if necessary.
-
- If the architecture can assume all supported versions of gcc will
- produce a tail-call to __libc_vfork, consider including the version
- in sysdeps/unix/sysv/linux/aarch64/pt-vfork.c. */
-
-#if !HAVE_IFUNC
-# error "must write pt-vfork for this machine or get IFUNC support"
-#endif
-
-#if (SHLIB_COMPAT (libpthread, GLIBC_2_0, GLIBC_2_20) \
- || SHLIB_COMPAT (libpthread, GLIBC_2_1_2, GLIBC_2_20))
-
-extern __typeof (vfork) __libc_vfork; /* Defined in libc. */
-
-# undef INIT_ARCH
-# define INIT_ARCH()
-# define DEFINE_VFORK(name) libc_ifunc (name, &__libc_vfork)
-
-#endif
-
-#if SHLIB_COMPAT (libpthread, GLIBC_2_0, GLIBC_2_20)
-extern __typeof(vfork) vfork_ifunc;
-DEFINE_VFORK (vfork_ifunc)
-compat_symbol (libpthread, vfork_ifunc, vfork, GLIBC_2_0);
-#endif
-
-#if SHLIB_COMPAT (libpthread, GLIBC_2_1_2, GLIBC_2_20)
-extern __typeof(vfork) __vfork_ifunc;
-DEFINE_VFORK (__vfork_ifunc)
-compat_symbol (libpthread, __vfork_ifunc, __vfork, GLIBC_2_1_2);
-#endif
@@ -21,10 +21,10 @@
/* If we don't have vfork, fork is close enough. */
__pid_t
-__vfork (void)
+__libc_vfork (void)
{
return __fork ();
}
+strong_alias (__libc_vfork, __vfork)
libc_hidden_def (__vfork)
-
-weak_alias (__vfork, vfork)
+weak_alias (__libc_vfork, vfork)
@@ -151,6 +151,15 @@
movk PTR_REG (R), #:abs_g0_nc:NAME;
#endif
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#undef SIBCALL
+#define SIBCALL(dest) \
+ b JUMPTARGET(dest)
+
/* Since C identifiers are not normally prefixed with an underscore
on this system, the asm identifier `syscall_error' intrudes on the
C name space. Make sure we use an innocuous name. */
@@ -121,6 +121,15 @@
#define CALL_MCOUNT /* Do nothing. */
#endif
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#undef SIBCALL
+#define SIBCALL(dest) \
+ b PLTJMP(dest)
+
/* Since C identifiers are not normally prefixed with an underscore
on this system, the asm identifier `syscall_error' intrudes on the
C name space. Make sure we use an innocuous name. */
new file mode 100644
@@ -0,0 +1,64 @@
+/* Compatibility stubs for functions formerly exposed by libpthread.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+/* The functions defined by this file used to have two different
+ definitions, one in libc.so and one in libpthread.so. Nowadays,
+ only the version in libc is necessary, but libpthread must continue
+ to expose these symbols for compatibility's sake. The stubs just
+ jump to the definition in libc.
+
+ This file is written in heavily macro-ized assembly language
+ because one of the stubs that it needs to define is vfork, and the
+ implementation of vfork must not touch the stack. Having done the
+ work to handle that, we may as well reuse the mechanism for all of
+ the stubs. */
+
+#include <shlib-compat.h>
+#include <sysdep.h>
+
+ compat_text_section
+
+#define define_stub(name) \
+ define_stub_1(__pstub_##name, __libc_##name)
+#define define_stub_1(pstub_name, libc_name) \
+ ENTRY(pstub_name) ASM_LINE_SEP \
+ SIBCALL(libc_name) ASM_LINE_SEP \
+ END(pstub_name)
+
+#define compat_stub(base, sym, ver) \
+ compat_stub_1(base, sym, ver, __COUNTER__)
+#define compat_stub_1(base, sym, ver, tag) \
+ compat_stub_2(base, sym, ver, tag)
+#define compat_stub_2(base, sym, ver, tag) \
+ compat_stub_3(__pstub_##base, __pstub_##base##_##tag, sym, ver)
+#define compat_stub_3(base, nonce, sym, ver) \
+ weak_alias(base, nonce) ASM_LINE_SEP \
+ compat_symbol(libpthread, nonce, sym, ver)
+
+#if SHLIB_COMPAT (libpthread, GLIBC_2_0, GLIBC_2_20) \
+ || SHLIB_COMPAT (libpthread, GLIBC_2_1_2, GLIBC_2_20)
+
+ define_stub(vfork)
+# if SHLIB_COMPAT (libpthread, GLIBC_2_0, GLIBC_2_20)
+ compat_stub(vfork, vfork, GLIBC_2_0)
+# endif
+# if SHLIB_COMPAT (libpthread, GLIBC_2_1_2, GLIBC_2_20)
+ compat_stub(vfork, __vfork, GLIBC_2_1_2)
+# endif
+
+#endif
@@ -33,9 +33,20 @@
# ifndef JUMPTARGET
# define JUMPTARGET(sym) sym
# endif
+
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+# ifndef SIBCALL
+# define SIBCALL(dest) \
+ .error "Missing definition of SIBCALL"
+# endif
#endif
-/* Makros to generate eh_frame unwind information. */
+
+/* Macros to generate eh_frame unwind information. */
#ifdef __ASSEMBLER__
# define cfi_startproc .cfi_startproc
# define cfi_endproc .cfi_endproc
@@ -62,6 +62,16 @@
#define PSEUDO_END(name) \
END (name)
+
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#undef SIBCALL
+#define SIBCALL(dest) \
+ b JUMPTARGET(dest)
+
#undef JUMPTARGET
#define JUMPTARGET(name) name
#define SYSCALL_PIC_SETUP /* Nothing. */
@@ -59,4 +59,13 @@
#undef END
#define END(sym) .endp C_SYMBOL_NAME(sym)
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#undef SIBCALL
+#define SIBCALL(dest) \
+ .mib; nop 0; nop 0; br.sptk.many C_SYMBOL_NAME(dest);;
+
#endif /* ASSEMBLER */
@@ -71,4 +71,13 @@
# define JUMPTARGET(name) name
# endif
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#undef SIBCALL
+#define SIBCALL(dest) \
+ jbra JUMPTARGET(dest)
+
#endif /* __ASSEMBLER__ */
@@ -70,6 +70,15 @@
# define PSEUDO_END(name) \
END (name)
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#undef SIBCALL
+#define SIBCALL(dest) \
+ brid JUMPTARGET(dest); nop
+
# undef JUMPTARGET
# ifdef PIC
# define JUMPTARGET(name) name##@PLTPC
@@ -62,4 +62,25 @@
# define CALL_MCOUNT /* Do nothing. */
#endif
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#undef SIBCALL
+#ifdef __PIC__
+#define SIBCALL(dest) \
+ nextpc r2; \
+1: \
+ movhi r3, %hiadj(_gp_got - 1b); \
+ addi r3, r3, %lo(_gp_got - 1b); \
+ addi r2, r2, r3; \
+ ldw r2, %call(dest)(r2); \
+ jmp r2
+#else
+#define SIBCALL(dest) \
+ jmpi dest
+#endif
+
+
#endif /* __ASSEMBLER__ */
@@ -90,6 +90,14 @@ GOT_LABEL: ; \
cfi_endproc; \
ASM_SIZE_DIRECTIVE(name)
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#define SIBCALL(dest) \
+ b JUMPTARGET(dest)
+
#if !IS_IN(rtld) && !defined(__SPE__)
# define ABORT_TRANSACTION_IMPL \
cmpwi 2,0; \
@@ -359,6 +359,15 @@ LT_LABELSUFFIX(name,_name_end): ; \
#define PSEUDO_END_ERRVAL(name) \
END (name)
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#define SIBCALL(dest) \
+ b JUMPTARGET(dest); \
+ ori r0, r0, 0
+
#else /* !__ASSEMBLER__ */
#if _CALL_ELF != 2
@@ -59,6 +59,15 @@
#define CALL_MCOUNT /* Do nothing. */
#endif
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#undef SIBCALL
+#define SIBCALL(dest) \
+ jg JUMPTARGET(dest)
+
/* Since C identifiers are not normally prefixed with an underscore
on this system, the asm identifier `syscall_error' intrudes on the
C name space. Make sure we use an innocuous name. */
@@ -57,6 +57,15 @@
#define CALL_MCOUNT /* Do nothing. */
#endif
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#undef SIBCALL
+#define SIBCALL(dest) \
+ jg JUMPTARGET(dest)
+
/* Since C identifiers are not normally prefixed with an underscore
on this system, the asm identifier `syscall_error' intrudes on the
C name space. Make sure we use an innocuous name. */
@@ -66,6 +66,20 @@
#define CALL_MCOUNT /* Do nothing. */
#endif
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#undef SIBCALL
+#define SIBCALL(dest) \
+ mov.l 1f, r1; \
+ braf r1; \
+ nop; \
+0: .align 2; \
+1: .long dest@PLT+(.-0b)
+
+
/* Since C identifiers are not normally prefixed with an underscore
on this system, the asm identifier `syscall_error' intrudes on the
C name space. Make sure we use an innocuous name. */
@@ -79,4 +79,13 @@ C_LABEL(name) \
#undef LOC
#define LOC(name) .L##name
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#undef SIBCALL
+#define SIBCALL(dest) \
+ jmp dest
+
#endif /* __ASSEMBLER__ */
@@ -56,6 +56,15 @@
#define CALL_MCOUNT /* Do nothing. */
#endif
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#undef SIBCALL
+#define SIBCALL(dest) \
+ j plt(dest)
+
/* Local label name for asm code. */
#define L(name) .L##name
@@ -148,6 +148,16 @@ __LABEL(name) \
#undef PSEUDO_END_ERRVAL
#define PSEUDO_END_ERRVAL(sym) END(sym)
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#define SIBCALL(dest) \
+ ldgp gp, 0(pv); \
+ lda pv, dest; \
+ jmp (pv), dest
+
#define ret_ERRVAL ret
#define r0 v0
@@ -54,3 +54,28 @@ L(syse1):
bne a3, zero, 99b; \
L(syse1):
#endif
+
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#ifdef __PIC__
+#define SIBCALL(dest) \
+ .set nomips16; \
+ .set noreorder; \
+ .cpload $25; \
+ .set nomacro; \
+ lw $25, %call16(dest)($28); \
+ nop; \
+ .reloc 1f, R_MIPS_JALR, dest; \
+1: jr $25; \
+ nop
+#else
+#define SIBCALL(dest) \
+ .set nomips16; \
+ .set noreorder; \
+ .set nomacro; \
+ j dest; \
+ nop
+#endif
@@ -61,4 +61,35 @@ L(syse1):
L(syse1):
#endif
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#ifdef __PIC__
+/* There's no PLT on MIPS/n32, we are expected to load the address by
+ hand, but the usual gp register on MIPS ($28) is call-saved so we
+ can't use it. Use $at ($1) instead. */
+#define SIBCALL(dest) \
+ .set nomips16; \
+ .set noreorder; \
+ .set nomacro; \
+ .set noat; \
+0: lui $1, %hi(%neg(%gp_rel(0b))); \
+ addiu $1, $1, %lo(%neg(%gp_rel(0b))); \
+ addu $1, $1, $25; \
+ lw $1, %call16(dest)($1); \
+ nop; \
+ .reloc 1f, R_MIPS_JALR, dest; \
+1: jr $25; \
+ nop
+#else
+#define SIBCALL(dest) \
+ .set nomips16; \
+ .set noreorder; \
+ .set nomacro; \
+ j dest; \
+ nop
#endif
+
+#endif /* __ASSEMBLER__ */
@@ -61,4 +61,27 @@ L(syse1):
L(syse1):
#endif
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#define SIBCALL(dest) \
+/* There's no PLT on MIPS/n64, we are expected to load the address by
+ hand, but the usual gp register on MIPS ($28) is call-saved so we
+ can't use it. Use $at ($1) instead. */
+#define SIBCALL(dest) \
+ .set nomips16; \
+ .set noreorder; \
+ .set nomacro; \
+ .set noat; \
+0: lui $1, %hi(%neg(%gp_rel(0b))); \
+ daddiu $1, $1, %lo(%neg(%gp_rel(0b))); \
+ daddu $1, $1, $25; \
+ ld $1, %call16(dest)($1); \
+ nop; \
+ .reloc 1f, R_MIPS_JALR, dest; \
+1: jr $25; \
+ nop
+
#endif
deleted file mode 100644
@@ -1,54 +0,0 @@
-/* vfork ABI-compatibility entry points for libpthread.
- Copyright (C) 2014-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-#include <shlib-compat.h>
-
-/* libpthread used to have its own vfork implementation that differed
- from libc's only in having a pointless micro-optimization. There
- is no longer any use to having a separate copy in libpthread, but
- the historical ABI requires it. For static linking, there is no
- need to provide anything here--the libc version will be linked in.
- For shared library ABI compatibility, there must be __vfork and
- vfork symbols in libpthread.so. */
-
-#if HAVE_IFUNC
-# include <nptl/pt-vfork.c>
-#elif (SHLIB_COMPAT (libpthread, GLIBC_2_0, GLIBC_2_20) \
- || SHLIB_COMPAT (libpthread, GLIBC_2_1_2, GLIBC_2_20))
-
-/* Thankfully, on AArch64 we can rely on the compiler generating
- a tail call here. */
-
-extern void __libc_vfork (void);
-
-void
-vfork_compat (void)
-{
- __libc_vfork ();
-}
-
-# if SHLIB_COMPAT (libpthread, GLIBC_2_0, GLIBC_2_20)
-compat_symbol (libpthread, vfork_compat, vfork, GLIBC_2_0);
-# endif
-
-# if SHLIB_COMPAT (libpthread, GLIBC_2_1_2, GLIBC_2_20)
-strong_alias (vfork_compat, vfork_compat2)
-compat_symbol (libpthread, vfork_compat2, __vfork, GLIBC_2_1_2);
-# endif
-
-#endif
deleted file mode 100644
@@ -1 +0,0 @@
-#include <sysdeps/unix/sysv/linux/aarch64/pt-vfork.c>
@@ -107,6 +107,15 @@
# undef ret_ERRVAL
# define ret_ERRVAL ret
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#undef SIBCALL
+#define SIBCALL(dest) \
+ tail JUMPTARGET(dest)
+
#endif /* __ASSEMBLER__ */
/* In order to get __set_errno() definition in INLINE_SYSCALL. */
deleted file mode 100644
@@ -1 +0,0 @@
-#include <sysdeps/unix/sysv/linux/aarch64/pt-vfork.c>
@@ -46,6 +46,15 @@
#define ENTRY_CHK(name) ENTRY (name)
#define END_CHK(name) END (name)
+/* Make a "sibling call" to DEST -- that is, transfer control to DEST
+ as-if it had been the function called by the caller of this function.
+ DEST is likely to be defined in a different shared object. Only
+ ever used immediately after ENTRY. Must not touch the stack at
+ all, and must preserve all argument and call-saved registers. */
+#undef SIBCALL
+#define SIBCALL(dest) \
+ jmp JUMPTARGET(dest)
+
/* Since C identifiers are not normally prefixed with an underscore
on this system, the asm identifier `syscall_error' intrudes on the
C name space. Make sure we use an innocuous name. */