From patchwork Fri Aug 14 17:33:02 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 8210 Received: (qmail 27446 invoked by alias); 14 Aug 2015 17:33:10 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 27437 invoked by uid 89); 14 Aug 2015 17:33:09 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=1.5 required=5.0 tests=AWL, BAYES_99, BAYES_999, FREEMAIL_FROM, RCVD_IN_DNSWL_LOW, SPF_PASS autolearn=no version=3.3.2 X-HELO: mail-ob0-f180.google.com MIME-Version: 1.0 X-Received: by 10.182.86.39 with SMTP id m7mr42259543obz.18.1439573582590; Fri, 14 Aug 2015 10:33:02 -0700 (PDT) In-Reply-To: <20150814120309.GB28610@gmail.com> References: <20150812192001.GA12730@intel.com> <20150812221203.GA4224@intel.com> <20150814120309.GB28610@gmail.com> Date: Fri, 14 Aug 2015 10:33:02 -0700 Message-ID: Subject: Re: [PATCH 2/6] Optimize i386 syscall inlining From: "H.J. Lu" To: Zack Weinberg Cc: GNU C Library On Fri, Aug 14, 2015 at 5:03 AM, H.J. Lu wrote: > On Wed, Aug 12, 2015 at 07:37:20PM -0400, Zack Weinberg wrote: >> If I'm reading that right, it's still not quite optimal; there's an >> unnecessary register shuffle after the system call... better would be >> >> push %ebx >> mov $0x2d,%eax >> mov 0x8(%esp),%ebx >> call __x86.get_pc_thunk.cx >> add $_GLOBAL_OFFSET_TABLE_,%ecx >> call *%gs:0x10 >> mov __curbrk(%ecx),%edx >> mov %eax,(%edx) >> cmp %eax,%ebx >> ja 1f >> xor %eax,%eax >> pop %ebx >> ret >> 1: >> ; set errno and return -1 >> > > Here is the updated patch. OK for master? > > H.J. > -- > Define INLINE_SYSCALL_RETURN and INLINE_SYSCALL_ERROR_RETURN so > that i386 can optimize setting errno by branching to the internal > __syscall_error without PLT. > > Since GCC 5 and above can properly spill %ebx when needed, we can inline > syscalls with 6 arguments if GCC 5 or above is used to compile glibc. > This patch rewrites INTERNAL_SYSCALL macros and skips __libc_do_syscall > for GCC 5. > > For sysdeps/unix/sysv/linux/i386/brk.c, with -O2 -march=i686 > -mtune=generic, GCC 5.2 now generates: > > <__brk>: > 0: push %ebx > 1: mov $0x2d,%eax > 6: mov 0x8(%esp),%ebx > a: call b <__brk+0xb> b: R_386_PC32 __x86.get_pc_thunk.dx > f: add $0x2,%edx 11: R_386_GOTPC _GLOBAL_OFFSET_TABLE_ > 15: call *%gs:0x10 > 1c: mov 0x0(%edx),%edx 1e: R_386_GOT32 __curbrk > 22: cmp %eax,%ebx > 24: mov %eax,(%edx) > 26: ja 30 <__brk+0x30> > 28: xor %eax,%eax > 2a: pop %ebx > 2b: ret > > instead of > > <__brk>: > 0: push %ebx > 1: mov 0x8(%esp),%ecx > 5: call 6 <__brk+0x6> 6: R_386_PC32 __x86.get_pc_thunk.bx > a: add $0x2,%ebx c: R_386_GOTPC _GLOBAL_OFFSET_TABLE_ > 10: xchg %ecx,%ebx > 12: mov $0x2d,%eax > 17: call *%gs:0x10 > 1e: xchg %ecx,%ebx > 20: mov %eax,%edx > 22: mov 0x0(%ebx),%eax 24: R_386_GOT32 __curbrk > 28: mov %edx,(%eax) > 2a: xor %eax,%eax > 2c: cmp %edx,%ecx > 2e: ja 38 <__brk+0x38> > 30: pop %ebx > 31: ret > > The new one is shorter by 2 instructions. > > * sysdeps/unix/sysv/linux/i386/Makefile [$(subdir) == csu] > (sysdep-dl-routines): Add sysdep. > [$(subdir) == nptl] (libpthread-routines): Likewise. > [$(subdir) == rt] (librt-routines): Likewise. > * sysdeps/unix/sysv/linux/i386/brk.c (__brk): Add > INTERNAL_SYSCALL_DECL. Use INLINE_SYSCALL_ERROR_RETURN. > * sysdeps/unix/sysv/linux/i386/clone.S (__clone): Don't check > PIC when branching to SYSCALL_ERROR_LABEL. > * sysdeps/unix/sysv/linux/i386/fcntl.c (__fcntl_nocancel): Use > INLINE_SYSCALL_RETURN and INLINE_SYSCALL_ERROR_RETURN. > (__libc_fcntl): Likewise. > * sysdeps/unix/sysv/linux/i386/fxstat.c (__fxstat): Likewise. > * sysdeps/unix/sysv/linux/i386/fxstatat.c (__fxstatat): > Likewise. > * sysdeps/unix/sysv/linux/i386/getmsg.c (getmsg): Likewise. > * sysdeps/unix/sysv/linux/i386/lockf64.c (lockf64): Likewise. > * sysdeps/unix/sysv/linux/i386/lxstat.c (__lxstat): Likewise. > * sysdeps/unix/sysv/linux/i386/msgctl.c (__old_msgctl): > Likewise. > (__new_msgctl): Likewise. > * sysdeps/unix/sysv/linux/i386/putmsg.c (putmsg): Likewise. > * sysdeps/unix/sysv/linux/i386/semctl.c (__old_semctl): > Likewise. > (__new_semctl): Likewise. > * sysdeps/unix/sysv/linux/i386/setegid.c (setegid): Likewise. > * sysdeps/unix/sysv/linux/i386/seteuid.c (seteuid): Likewise. > * sysdeps/unix/sysv/linux/i386/shmctl.c (__old_shmctl): > Likewise. > (__new_shmctl): Likewise. > * sysdeps/unix/sysv/linux/i386/sigaction.c (__libc_sigaction): > Likewise. > * sysdeps/unix/sysv/linux/i386/xstat.c (__xstat): Likewise. > * sysdeps/unix/sysv/linux/i386/libc-do-syscall.S > (__libc_do_syscall): Defined only if !__GNUC_PREREQ (5,0). > * sysdeps/unix/sysv/linux/i386/sysdep.S: Removed. > * sysdeps/unix/sysv/linux/i386/sysdep.c: New file. > * sysdeps/unix/sysv/linux/i386/sysdep.h: Define assembler macros > only if !__GNUC_PREREQ (5,0). > (SYSCALL_ERROR_LABEL): Changed to __syscall_error. > (SYSCALL_ERROR_HANDLER): Changed to empty. > (SYSCALL_ERROR_ERRNO): Removed. > (SYSCALL_ERROR_HANDLER_TLS_STORE): Likewise. > (__syscall_error): New prototype. > (INLINE_SYSCALL_RETURN): New. > (INLINE_SYSCALL_ERROR_RETURN): Likewise. > (LOADREGS_0): Likewise. > (ASMARGS_0): Likewise. > (LOADREGS_1): Likewise. > (ASMARGS_1): Likewise. > (LOADREGS_2): Likewise. > (ASMARGS_2): Likewise. > (LOADREGS_3): Likewise. > (ASMARGS_3): Likewise. > (LOADREGS_4): Likewise. > (ASMARGS_4): Likewise. > (LOADREGS_5): Likewise. > (ASMARGS_5): Likewise. > (LOADREGS_6): Likewise. > (ASMARGS_6): Likewise. > (INTERNAL_SYSCALL_MAIN_6): Optimize for GCC 5. > (INTERNAL_SYSCALL_MAIN_INLINE): Likewise. > (INTERNAL_SYSCALL_NCS): Likewise. Here is the patch for updated INLINE_SYSCALL_RETURN and INLINE_SYSCALL_ERROR_RETURN. OK for master? From bb8940d0d841c4ae4a514d0110b72cfeca09f9b3 Mon Sep 17 00:00:00 2001 From: "H.J. Lu" Date: Wed, 12 Aug 2015 09:13:29 -0700 Subject: [PATCH 2/6] Optimize i386 syscall inlining Define INLINE_SYSCALL_RETURN and INLINE_SYSCALL_ERROR_RETURN so that i386 can optimize setting errno by branching to the internal __syscall_error without PLT. Since GCC 5 and above can properly spill %ebx when needed, we can inline syscalls with 6 arguments if GCC 5 or above is used to compile glibc. This patch rewrites INTERNAL_SYSCALL macros and skips __libc_do_syscall for GCC 5. For sysdeps/unix/sysv/linux/i386/brk.c, with -O2 -march=i686 -mtune=generic, GCC 5.2 now generates: <__brk>: 0: push %ebx 1: mov $0x2d,%eax 6: mov 0x8(%esp),%ebx a: call b <__brk+0xb> b: R_386_PC32 __x86.get_pc_thunk.dx f: add $0x2,%edx 11: R_386_GOTPC _GLOBAL_OFFSET_TABLE_ 15: call *%gs:0x10 1c: mov 0x0(%edx),%edx 1e: R_386_GOT32 __curbrk 22: cmp %eax,%ebx 24: mov %eax,(%edx) 26: ja 30 <__brk+0x30> 28: xor %eax,%eax 2a: pop %ebx 2b: ret instead of <__brk>: 0: push %ebx 1: mov 0x8(%esp),%ecx 5: call 6 <__brk+0x6> 6: R_386_PC32 __x86.get_pc_thunk.bx a: add $0x2,%ebx c: R_386_GOTPC _GLOBAL_OFFSET_TABLE_ 10: xchg %ecx,%ebx 12: mov $0x2d,%eax 17: call *%gs:0x10 1e: xchg %ecx,%ebx 20: mov %eax,%edx 22: mov 0x0(%ebx),%eax 24: R_386_GOT32 __curbrk 28: mov %edx,(%eax) 2a: xor %eax,%eax 2c: cmp %edx,%ecx 2e: ja 38 <__brk+0x38> 30: pop %ebx 31: ret The new one is shorter by 2 instructions. * sysdeps/unix/sysv/linux/i386/Makefile [$(subdir) == csu] (sysdep-dl-routines): Add sysdep. [$(subdir) == nptl] (libpthread-routines): Likewise. [$(subdir) == rt] (librt-routines): Likewise. * sysdeps/unix/sysv/linux/i386/brk.c (__brk): Add INTERNAL_SYSCALL_DECL. Use INLINE_SYSCALL_ERROR_RETURN. * sysdeps/unix/sysv/linux/i386/clone.S (__clone): Don't check PIC when branching to SYSCALL_ERROR_LABEL. * sysdeps/unix/sysv/linux/i386/fcntl.c (__fcntl_nocancel): Use INLINE_SYSCALL_RETURN and INLINE_SYSCALL_ERROR_RETURN. (__libc_fcntl): Likewise. * sysdeps/unix/sysv/linux/i386/fxstat.c (__fxstat): Likewise. * sysdeps/unix/sysv/linux/i386/fxstatat.c (__fxstatat): Likewise. * sysdeps/unix/sysv/linux/i386/getmsg.c (getmsg): Likewise. * sysdeps/unix/sysv/linux/i386/lockf64.c (lockf64): Likewise. * sysdeps/unix/sysv/linux/i386/lxstat.c (__lxstat): Likewise. * sysdeps/unix/sysv/linux/i386/msgctl.c (__old_msgctl): Likewise. (__new_msgctl): Likewise. * sysdeps/unix/sysv/linux/i386/putmsg.c (putmsg): Likewise. * sysdeps/unix/sysv/linux/i386/semctl.c (__old_semctl): Likewise. (__new_semctl): Likewise. * sysdeps/unix/sysv/linux/i386/setegid.c (setegid): Likewise. * sysdeps/unix/sysv/linux/i386/seteuid.c (seteuid): Likewise. * sysdeps/unix/sysv/linux/i386/setgid.c (__setgid): Likewise. * sysdeps/unix/sysv/linux/i386/setgroups.c (setgroups): Likewise. * sysdeps/unix/sysv/linux/i386/setregid.c (__setregid): Likewise. * sysdeps/unix/sysv/linux/i386/setresgid.c (__setresgid): Likewise. * sysdeps/unix/sysv/linux/i386/setresuid.c (__setresuid): Likewise. * sysdeps/unix/sysv/linux/i386/setreuid.c (__setreuid): Likewise. * sysdeps/unix/sysv/linux/i386/setuid.c (__setuid): Likewise. * sysdeps/unix/sysv/linux/i386/shmctl.c (__old_shmctl): Likewise. (__new_shmctl): Likewise. * sysdeps/unix/sysv/linux/i386/sigaction.c (__libc_sigaction): Likewise. * sysdeps/unix/sysv/linux/i386/xstat.c (__xstat): Likewise. * sysdeps/unix/sysv/linux/i386/libc-do-syscall.S (__libc_do_syscall): Defined only if !__GNUC_PREREQ (5,0). * sysdeps/unix/sysv/linux/i386/sysdep.S: Removed. * sysdeps/unix/sysv/linux/i386/sysdep.c: New file. * sysdeps/unix/sysv/linux/i386/sysdep.h: Define assembler macros only if !__GNUC_PREREQ (5,0). (SYSCALL_ERROR_LABEL): Changed to __syscall_error. (SYSCALL_ERROR_HANDLER): Changed to empty. (SYSCALL_ERROR_ERRNO): Removed. (SYSCALL_ERROR_HANDLER_TLS_STORE): Likewise. (__syscall_error): New prototype. (INLINE_SYSCALL_RETURN): New. (INLINE_SYSCALL_ERROR_RETURN): Likewise. (LOADREGS_0): Likewise. (ASMARGS_0): Likewise. (LOADREGS_1): Likewise. (ASMARGS_1): Likewise. (LOADREGS_2): Likewise. (ASMARGS_2): Likewise. (LOADREGS_3): Likewise. (ASMARGS_3): Likewise. (LOADREGS_4): Likewise. (ASMARGS_4): Likewise. (LOADREGS_5): Likewise. (ASMARGS_5): Likewise. (LOADREGS_6): Likewise. (ASMARGS_6): Likewise. (INTERNAL_SYSCALL_MAIN_6): Optimize for GCC 5. (INTERNAL_SYSCALL_MAIN_INLINE): Likewise. (INTERNAL_SYSCALL_NCS): Likewise. --- sysdeps/unix/sysv/linux/i386/Makefile | 14 ++ sysdeps/unix/sysv/linux/i386/brk.c | 12 +- sysdeps/unix/sysv/linux/i386/clone.S | 8 - sysdeps/unix/sysv/linux/i386/fcntl.c | 4 +- sysdeps/unix/sysv/linux/i386/fxstat.c | 12 +- sysdeps/unix/sysv/linux/i386/fxstatat.c | 9 +- sysdeps/unix/sysv/linux/i386/getmsg.c | 3 +- sysdeps/unix/sysv/linux/i386/libc-do-syscall.S | 3 + sysdeps/unix/sysv/linux/i386/lockf64.c | 15 +- sysdeps/unix/sysv/linux/i386/lxstat.c | 13 +- sysdeps/unix/sysv/linux/i386/msgctl.c | 6 +- sysdeps/unix/sysv/linux/i386/putmsg.c | 3 +- sysdeps/unix/sysv/linux/i386/semctl.c | 6 +- sysdeps/unix/sysv/linux/i386/setegid.c | 16 +- sysdeps/unix/sysv/linux/i386/seteuid.c | 13 +- sysdeps/unix/sysv/linux/i386/setgid.c | 7 +- sysdeps/unix/sysv/linux/i386/setgroups.c | 3 +- sysdeps/unix/sysv/linux/i386/setregid.c | 6 +- sysdeps/unix/sysv/linux/i386/setresgid.c | 6 +- sysdeps/unix/sysv/linux/i386/setresuid.c | 6 +- sysdeps/unix/sysv/linux/i386/setreuid.c | 6 +- sysdeps/unix/sysv/linux/i386/setuid.c | 6 +- sysdeps/unix/sysv/linux/i386/shmctl.c | 6 +- sysdeps/unix/sysv/linux/i386/sigaction.c | 12 +- sysdeps/unix/sysv/linux/i386/sysdep.S | 40 ----- sysdeps/unix/sysv/linux/i386/sysdep.c | 30 ++++ sysdeps/unix/sysv/linux/i386/sysdep.h | 196 +++++++++++++++++-------- sysdeps/unix/sysv/linux/i386/xstat.c | 13 +- 28 files changed, 248 insertions(+), 226 deletions(-) delete mode 100644 sysdeps/unix/sysv/linux/i386/sysdep.S create mode 100644 sysdeps/unix/sysv/linux/i386/sysdep.c diff --git a/sysdeps/unix/sysv/linux/i386/Makefile b/sysdeps/unix/sysv/linux/i386/Makefile index 80da593..e10d133 100644 --- a/sysdeps/unix/sysv/linux/i386/Makefile +++ b/sysdeps/unix/sysv/linux/i386/Makefile @@ -27,3 +27,17 @@ endif ifeq ($(subdir),stdlib) gen-as-const-headers += ucontext_i.sym endif + +ifeq ($(subdir),csu) +sysdep-dl-routines += sysdep +endif + +ifeq ($(subdir),nptl) +# pull in __syscall_error routine +libpthread-routines += sysdep +endif + +ifeq ($(subdir),rt) +# pull in __syscall_error routine +librt-routines += sysdep +endif diff --git a/sysdeps/unix/sysv/linux/i386/brk.c b/sysdeps/unix/sysv/linux/i386/brk.c index 5b9a0ce..e49905a 100644 --- a/sysdeps/unix/sysv/linux/i386/brk.c +++ b/sysdeps/unix/sysv/linux/i386/brk.c @@ -31,19 +31,11 @@ weak_alias (__curbrk, ___brk_addr) int __brk (void *addr) { - void *newbrk; - INTERNAL_SYSCALL_DECL (err); - newbrk = (void *) INTERNAL_SYSCALL (brk, err, 1, addr); - + void *newbrk = (void *) INTERNAL_SYSCALL (brk, err, 1, addr); __curbrk = newbrk; - if (newbrk < addr) - { - __set_errno (ENOMEM); - return -1; - } - + INLINE_SYSCALL_ERROR_RETURN (-ENOMEM, int, ); return 0; } weak_alias (__brk, brk) diff --git a/sysdeps/unix/sysv/linux/i386/clone.S b/sysdeps/unix/sysv/linux/i386/clone.S index 243dbfe..2aafb3a 100644 --- a/sysdeps/unix/sysv/linux/i386/clone.S +++ b/sysdeps/unix/sysv/linux/i386/clone.S @@ -47,19 +47,11 @@ ENTRY (__clone) /* Sanity check arguments. */ movl $-EINVAL,%eax movl FUNC(%esp),%ecx /* no NULL function pointers */ -#ifdef PIC - jecxz SYSCALL_ERROR_LABEL -#else testl %ecx,%ecx jz SYSCALL_ERROR_LABEL -#endif movl STACK(%esp),%ecx /* no NULL stack pointers */ -#ifdef PIC - jecxz SYSCALL_ERROR_LABEL -#else testl %ecx,%ecx jz SYSCALL_ERROR_LABEL -#endif /* Insert the argument onto the new stack. Make sure the new thread is started with an alignment of (mod 16). */ diff --git a/sysdeps/unix/sysv/linux/i386/fcntl.c b/sysdeps/unix/sysv/linux/i386/fcntl.c index 56f4bd1..eae84b5 100644 --- a/sysdeps/unix/sysv/linux/i386/fcntl.c +++ b/sysdeps/unix/sysv/linux/i386/fcntl.c @@ -34,7 +34,7 @@ __fcntl_nocancel (int fd, int cmd, ...) arg = va_arg (ap, void *); va_end (ap); - return INLINE_SYSCALL (fcntl64, 3, fd, cmd, arg); + INLINE_SYSCALL_RETURN (fcntl64, 3, int, fd, cmd, arg); } #endif /* NO_CANCELLATION */ @@ -50,7 +50,7 @@ __libc_fcntl (int fd, int cmd, ...) va_end (ap); if ((cmd != F_SETLKW) && (cmd != F_SETLKW64)) - return INLINE_SYSCALL (fcntl64, 3, fd, cmd, arg); + INLINE_SYSCALL_RETURN (fcntl64, 3, int, fd, cmd, arg); return SYSCALL_CANCEL (fcntl64, fd, cmd, arg); } diff --git a/sysdeps/unix/sysv/linux/i386/fxstat.c b/sysdeps/unix/sysv/linux/i386/fxstat.c index 2f7a8fe..b97a26d 100644 --- a/sysdeps/unix/sysv/linux/i386/fxstat.c +++ b/sysdeps/unix/sysv/linux/i386/fxstat.c @@ -37,15 +37,17 @@ __fxstat (int vers, int fd, struct stat *buf) int result; if (vers == _STAT_VER_KERNEL) - return INLINE_SYSCALL (fstat, 2, fd, (struct kernel_stat *) buf); + INLINE_SYSCALL_RETURN (fstat, 2, int, fd, (struct kernel_stat *) buf); { struct stat64 buf64; - result = INLINE_SYSCALL (fstat64, 2, fd, &buf64); - if (result == 0) - result = __xstat32_conv (vers, &buf64, buf); - return result; + INTERNAL_SYSCALL_DECL (err); + result = INTERNAL_SYSCALL (fstat64, err, 2, fd, &buf64); + if (__glibc_unlikely (INTERNAL_SYSCALL_ERROR_P (result, ))) + INLINE_SYSCALL_ERROR_RETURN (result, int, ); + else + return __xstat32_conv (vers, &buf64, buf); } } diff --git a/sysdeps/unix/sysv/linux/i386/fxstatat.c b/sysdeps/unix/sysv/linux/i386/fxstatat.c index 6f3c251..0f7fb06 100644 --- a/sysdeps/unix/sysv/linux/i386/fxstatat.c +++ b/sysdeps/unix/sysv/linux/i386/fxstatat.c @@ -42,13 +42,10 @@ __fxstatat (int vers, int fd, const char *file, struct stat *st, int flag) struct stat64 st64; result = INTERNAL_SYSCALL (fstatat64, err, 4, fd, file, &st64, flag); - if (!__builtin_expect (INTERNAL_SYSCALL_ERROR_P (result, err), 1)) - return __xstat32_conv (vers, &st64, st); + if (__glibc_unlikely (INTERNAL_SYSCALL_ERROR_P (result, ))) + INLINE_SYSCALL_ERROR_RETURN (result, int, ); else - { - __set_errno (INTERNAL_SYSCALL_ERRNO (result, err)); - return -1; - } + return __xstat32_conv (vers, &st64, st); } libc_hidden_def (__fxstatat) #ifdef XSTAT_IS_XSTAT64 diff --git a/sysdeps/unix/sysv/linux/i386/getmsg.c b/sysdeps/unix/sysv/linux/i386/getmsg.c index 255b867..a19ba36 100644 --- a/sysdeps/unix/sysv/linux/i386/getmsg.c +++ b/sysdeps/unix/sysv/linux/i386/getmsg.c @@ -30,7 +30,8 @@ getmsg (fildes, ctlptr, dataptr, flagsp) struct strbuf *dataptr; int *flagsp; { - return INLINE_SYSCALL (getpmsg, 5, fildes, ctlptr, dataptr, NULL, flagsp); + INLINE_SYSCALL_RETURN (getpmsg, 5, int, fildes, ctlptr, dataptr, + NULL, flagsp); } #else # include diff --git a/sysdeps/unix/sysv/linux/i386/libc-do-syscall.S b/sysdeps/unix/sysv/linux/i386/libc-do-syscall.S index af5c6f0..cdef3d5 100644 --- a/sysdeps/unix/sysv/linux/i386/libc-do-syscall.S +++ b/sysdeps/unix/sysv/linux/i386/libc-do-syscall.S @@ -18,6 +18,8 @@ #include +#if !__GNUC_PREREQ (5,0) + /* %eax, %ecx, %edx and %esi contain the values expected by the kernel. %edi points to a structure with the values of %ebx, %edi and %ebp. */ @@ -48,3 +50,4 @@ ENTRY (__libc_do_syscall) cfi_restore (ebx) ret END (__libc_do_syscall) +#endif diff --git a/sysdeps/unix/sysv/linux/i386/lockf64.c b/sysdeps/unix/sysv/linux/i386/lockf64.c index 61fcf22..d203086 100644 --- a/sysdeps/unix/sysv/linux/i386/lockf64.c +++ b/sysdeps/unix/sysv/linux/i386/lockf64.c @@ -29,6 +29,7 @@ lockf64 (int fd, int cmd, off64_t len64) { struct flock64 fl64; int cmd64; + int result; memset ((char *) &fl64, '\0', sizeof (fl64)); fl64.l_whence = SEEK_CUR; @@ -41,12 +42,13 @@ lockf64 (int fd, int cmd, off64_t len64) /* Test the lock: return 0 if FD is unlocked or locked by this process; return -1, set errno to EACCES, if another process holds the lock. */ fl64.l_type = F_RDLCK; - if (INLINE_SYSCALL (fcntl64, 3, fd, F_GETLK64, &fl64) < 0) - return -1; + INTERNAL_SYSCALL_DECL (err); + result = INTERNAL_SYSCALL (fcntl64, err, 3, fd, F_GETLK64, &fl64); + if (__glibc_unlikely (INTERNAL_SYSCALL_ERROR_P (result, ))) + INLINE_SYSCALL_ERROR_RETURN (result, int, ); if (fl64.l_type == F_UNLCK || fl64.l_pid == __getpid ()) return 0; - __set_errno (EACCES); - return -1; + INLINE_SYSCALL_ERROR_RETURN (-EACCES, int, ); case F_ULOCK: fl64.l_type = F_UNLCK; cmd64 = F_SETLK64; @@ -61,8 +63,7 @@ lockf64 (int fd, int cmd, off64_t len64) break; default: - __set_errno (EINVAL); - return -1; + INLINE_SYSCALL_ERROR_RETURN (-EINVAL, int, ); } - return INLINE_SYSCALL (fcntl64, 3, fd, cmd64, &fl64); + INLINE_SYSCALL_RETURN (fcntl64, 3, int, fd, cmd64, &fl64); } diff --git a/sysdeps/unix/sysv/linux/i386/lxstat.c b/sysdeps/unix/sysv/linux/i386/lxstat.c index 0891127..778c939 100644 --- a/sysdeps/unix/sysv/linux/i386/lxstat.c +++ b/sysdeps/unix/sysv/linux/i386/lxstat.c @@ -38,15 +38,18 @@ __lxstat (int vers, const char *name, struct stat *buf) int result; if (vers == _STAT_VER_KERNEL) - return INLINE_SYSCALL (lstat, 2, name, (struct kernel_stat *) buf); + INLINE_SYSCALL_RETURN (lstat, 2, int, name, + (struct kernel_stat *) buf); { struct stat64 buf64; - result = INLINE_SYSCALL (lstat64, 2, name, &buf64); - if (result == 0) - result = __xstat32_conv (vers, &buf64, buf); - return result; + INTERNAL_SYSCALL_DECL (err); + result = INTERNAL_SYSCALL (lstat64, err, 2, name, &buf64); + if (__glibc_unlikely (INTERNAL_SYSCALL_ERROR_P (result, ))) + INLINE_SYSCALL_ERROR_RETURN (result, int, ); + else + return __xstat32_conv (vers, &buf64, buf); } } diff --git a/sysdeps/unix/sysv/linux/i386/msgctl.c b/sysdeps/unix/sysv/linux/i386/msgctl.c index e42f71d..28beefb 100644 --- a/sysdeps/unix/sysv/linux/i386/msgctl.c +++ b/sysdeps/unix/sysv/linux/i386/msgctl.c @@ -55,7 +55,7 @@ int attribute_compat_text_section __old_msgctl (int msqid, int cmd, struct __old_msqid_ds *buf) { - return INLINE_SYSCALL (ipc, 5, IPCOP_msgctl, msqid, cmd, 0, buf); + INLINE_SYSCALL_RETURN (ipc, 5, int, IPCOP_msgctl, msqid, cmd, 0, buf); } compat_symbol (libc, __old_msgctl, msgctl, GLIBC_2_0); #endif @@ -63,8 +63,8 @@ compat_symbol (libc, __old_msgctl, msgctl, GLIBC_2_0); int __new_msgctl (int msqid, int cmd, struct msqid_ds *buf) { - return INLINE_SYSCALL (ipc, 5, IPCOP_msgctl, - msqid, cmd | __IPC_64, 0, buf); + INLINE_SYSCALL_RETURN (ipc, 5, int, IPCOP_msgctl, msqid, + cmd | __IPC_64, 0, buf); } versioned_symbol (libc, __new_msgctl, msgctl, GLIBC_2_2); diff --git a/sysdeps/unix/sysv/linux/i386/putmsg.c b/sysdeps/unix/sysv/linux/i386/putmsg.c index ab0112d..228011a 100644 --- a/sysdeps/unix/sysv/linux/i386/putmsg.c +++ b/sysdeps/unix/sysv/linux/i386/putmsg.c @@ -30,7 +30,8 @@ putmsg (fildes, ctlptr, dataptr, flags) const struct strbuf *dataptr; int flags; { - return INLINE_SYSCALL (putpmsg, 5, fildes, ctlptr, dataptr, -1, flags); + INLINE_SYSCALL_RETURN (putpmsg, 5, int, fildes, ctlptr, dataptr, -1, + flags); } #else # include diff --git a/sysdeps/unix/sysv/linux/i386/semctl.c b/sysdeps/unix/sysv/linux/i386/semctl.c index 8f13649..5220f71 100644 --- a/sysdeps/unix/sysv/linux/i386/semctl.c +++ b/sysdeps/unix/sysv/linux/i386/semctl.c @@ -71,7 +71,7 @@ __old_semctl (int semid, int semnum, int cmd, ...) va_end (ap); - return INLINE_SYSCALL (ipc, 5, IPCOP_semctl, semid, semnum, cmd, + INLINE_SYSCALL_RETURN (ipc, 5, int, IPCOP_semctl, semid, semnum, cmd, &arg); } compat_symbol (libc, __old_semctl, semctl, GLIBC_2_0); @@ -90,8 +90,8 @@ __new_semctl (int semid, int semnum, int cmd, ...) va_end (ap); - return INLINE_SYSCALL (ipc, 5, IPCOP_semctl, semid, semnum, cmd | __IPC_64, - &arg); + INLINE_SYSCALL_RETURN (ipc, 5, int, IPCOP_semctl, semid, semnum, + cmd | __IPC_64, &arg); } versioned_symbol (libc, __new_semctl, semctl, GLIBC_2_2); diff --git a/sysdeps/unix/sysv/linux/i386/setegid.c b/sysdeps/unix/sysv/linux/i386/setegid.c index 8c39784..f510ee6 100644 --- a/sysdeps/unix/sysv/linux/i386/setegid.c +++ b/sysdeps/unix/sysv/linux/i386/setegid.c @@ -17,23 +17,13 @@ #include #include -#include int -setegid (gid) - gid_t gid; +setegid (gid_t gid) { - int result; - if (gid == (gid_t) ~0) - { - __set_errno (EINVAL); - return -1; - } - - result = INLINE_SETXID_SYSCALL (setresgid32, 3, -1, gid, -1); - - return result; + INLINE_SYSCALL_ERROR_RETURN (-EINVAL, int, ); + INLINE_SYSCALL_RETURN (setresgid32, 3, int, -1, gid, -1); } libc_hidden_def (setegid) diff --git a/sysdeps/unix/sysv/linux/i386/seteuid.c b/sysdeps/unix/sysv/linux/i386/seteuid.c index d6a7a27..dbefac7 100644 --- a/sysdeps/unix/sysv/linux/i386/seteuid.c +++ b/sysdeps/unix/sysv/linux/i386/seteuid.c @@ -17,22 +17,13 @@ #include #include -#include int seteuid (uid_t uid) { - int result; - if (uid == (uid_t) ~0) - { - __set_errno (EINVAL); - return -1; - } - - result = INLINE_SETXID_SYSCALL (setresuid32, 3, -1, uid, -1); - - return result; + INLINE_SYSCALL_ERROR_RETURN (-EINVAL, int, ); + INLINE_SYSCALL_RETURN (setresuid32, 3, int, -1, uid, -1); } libc_hidden_def (seteuid) diff --git a/sysdeps/unix/sysv/linux/i386/setgid.c b/sysdeps/unix/sysv/linux/i386/setgid.c index d60d2bd..bf36034 100644 --- a/sysdeps/unix/sysv/linux/i386/setgid.c +++ b/sysdeps/unix/sysv/linux/i386/setgid.c @@ -17,17 +17,12 @@ #include #include -#include #include int __setgid (gid_t gid) { - int result; - - result = INLINE_SETXID_SYSCALL (setgid32, 1, gid); - - return result; + INLINE_SYSCALL_RETURN (setgid32, 1, int, gid); } #ifndef __setgid weak_alias (__setgid, setgid) diff --git a/sysdeps/unix/sysv/linux/i386/setgroups.c b/sysdeps/unix/sysv/linux/i386/setgroups.c index aabdcf6..e2aae4c 100644 --- a/sysdeps/unix/sysv/linux/i386/setgroups.c +++ b/sysdeps/unix/sysv/linux/i386/setgroups.c @@ -23,7 +23,6 @@ #include #include -#include #include /* Set the group set for the current user to GROUPS (N of them). For @@ -32,6 +31,6 @@ int setgroups (size_t n, const gid_t *groups) { - return INLINE_SETXID_SYSCALL (setgroups32, 2, n, groups); + INLINE_SYSCALL_RETURN (setgroups32, 2, int, n, groups); } libc_hidden_def (setgroups) diff --git a/sysdeps/unix/sysv/linux/i386/setregid.c b/sysdeps/unix/sysv/linux/i386/setregid.c index 865c47b..be63802 100644 --- a/sysdeps/unix/sysv/linux/i386/setregid.c +++ b/sysdeps/unix/sysv/linux/i386/setregid.c @@ -23,11 +23,7 @@ int __setregid (gid_t rgid, gid_t egid) { - int result; - - result = INLINE_SETXID_SYSCALL (setregid32, 2, rgid, egid); - - return result; + INLINE_SYSCALL_RETURN (setregid32, 2, int, rgid, egid); } #ifndef __setregid weak_alias (__setregid, setregid) diff --git a/sysdeps/unix/sysv/linux/i386/setresgid.c b/sysdeps/unix/sysv/linux/i386/setresgid.c index 9639c52..6147b00 100644 --- a/sysdeps/unix/sysv/linux/i386/setresgid.c +++ b/sysdeps/unix/sysv/linux/i386/setresgid.c @@ -24,11 +24,7 @@ int __setresgid (gid_t rgid, gid_t egid, gid_t sgid) { - int result; - - result = INLINE_SETXID_SYSCALL (setresgid32, 3, rgid, egid, sgid); - - return result; + INLINE_SYSCALL_RETURN (setresgid32, 3, int, rgid, egid, sgid); } libc_hidden_def (__setresgid) #ifndef __setresgid diff --git a/sysdeps/unix/sysv/linux/i386/setresuid.c b/sysdeps/unix/sysv/linux/i386/setresuid.c index f7e4107..9b70268 100644 --- a/sysdeps/unix/sysv/linux/i386/setresuid.c +++ b/sysdeps/unix/sysv/linux/i386/setresuid.c @@ -24,11 +24,7 @@ int __setresuid (uid_t ruid, uid_t euid, uid_t suid) { - int result; - - result = INLINE_SETXID_SYSCALL (setresuid32, 3, ruid, euid, suid); - - return result; + INLINE_SYSCALL_RETURN (setresuid32, 3, int, ruid, euid, suid); } libc_hidden_def (__setresuid) #ifndef __setresuid diff --git a/sysdeps/unix/sysv/linux/i386/setreuid.c b/sysdeps/unix/sysv/linux/i386/setreuid.c index af3c5b4..ee86e0a 100644 --- a/sysdeps/unix/sysv/linux/i386/setreuid.c +++ b/sysdeps/unix/sysv/linux/i386/setreuid.c @@ -24,11 +24,7 @@ int __setreuid (uid_t ruid, uid_t euid) { - int result; - - result = INLINE_SETXID_SYSCALL (setreuid32, 2, ruid, euid); - - return result; + INLINE_SYSCALL_RETURN (setreuid32, 2, int, ruid, euid); } #ifndef __setreuid weak_alias (__setreuid, setreuid) diff --git a/sysdeps/unix/sysv/linux/i386/setuid.c b/sysdeps/unix/sysv/linux/i386/setuid.c index ce6016e..eb6d0d5 100644 --- a/sysdeps/unix/sysv/linux/i386/setuid.c +++ b/sysdeps/unix/sysv/linux/i386/setuid.c @@ -24,11 +24,7 @@ int __setuid (uid_t uid) { - int result; - - result = INLINE_SETXID_SYSCALL (setuid32, 1, uid); - - return result; + INLINE_SYSCALL_RETURN (setuid32, 1, int, uid); } #ifndef __setuid weak_alias (__setuid, setuid) diff --git a/sysdeps/unix/sysv/linux/i386/shmctl.c b/sysdeps/unix/sysv/linux/i386/shmctl.c index eeb4453..6794335 100644 --- a/sysdeps/unix/sysv/linux/i386/shmctl.c +++ b/sysdeps/unix/sysv/linux/i386/shmctl.c @@ -62,7 +62,7 @@ int attribute_compat_text_section __old_shmctl (int shmid, int cmd, struct __old_shmid_ds *buf) { - return INLINE_SYSCALL (ipc, 5, IPCOP_shmctl, shmid, cmd, 0, buf); + INLINE_SYSCALL_RETURN (ipc, 5, int, IPCOP_shmctl, shmid, cmd, 0, buf); } compat_symbol (libc, __old_shmctl, shmctl, GLIBC_2_0); #endif @@ -70,8 +70,8 @@ compat_symbol (libc, __old_shmctl, shmctl, GLIBC_2_0); int __new_shmctl (int shmid, int cmd, struct shmid_ds *buf) { - return INLINE_SYSCALL (ipc, 5, IPCOP_shmctl, - shmid, cmd | __IPC_64, 0, buf); + INLINE_SYSCALL_RETURN (ipc, 5, int, IPCOP_shmctl, shmid, + cmd | __IPC_64, 0, buf); } versioned_symbol (libc, __new_shmctl, shmctl, GLIBC_2_2); diff --git a/sysdeps/unix/sysv/linux/i386/sigaction.c b/sysdeps/unix/sysv/linux/i386/sigaction.c index b20a9b9..7e830cd 100644 --- a/sysdeps/unix/sysv/linux/i386/sigaction.c +++ b/sysdeps/unix/sysv/linux/i386/sigaction.c @@ -69,11 +69,13 @@ __libc_sigaction (int sig, const struct sigaction *act, struct sigaction *oact) /* XXX The size argument hopefully will have to be changed to the real size of the user-level sigset_t. */ - result = INLINE_SYSCALL (rt_sigaction, 4, - sig, act ? &kact : NULL, - oact ? &koact : NULL, _NSIG / 8); - - if (oact && result >= 0) + INTERNAL_SYSCALL_DECL (err); + result = INTERNAL_SYSCALL (rt_sigaction, err, 4, + sig, act ? &kact : NULL, + oact ? &koact : NULL, _NSIG / 8); + if (__glibc_unlikely (INTERNAL_SYSCALL_ERROR_P (result, ))) + INLINE_SYSCALL_ERROR_RETURN (result, int, ); + else if (oact && result >= 0) { oact->sa_handler = koact.k_sa_handler; memcpy (&oact->sa_mask, &koact.sa_mask, sizeof (sigset_t)); diff --git a/sysdeps/unix/sysv/linux/i386/sysdep.S b/sysdeps/unix/sysv/linux/i386/sysdep.S deleted file mode 100644 index 4e6c262..0000000 --- a/sysdeps/unix/sysv/linux/i386/sysdep.S +++ /dev/null @@ -1,40 +0,0 @@ -/* Copyright (C) 1995-2015 Free Software Foundation, Inc. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - . */ - -#include - -/* The following code is only used in the shared library when we - compile the reentrant version. Otherwise each system call defines - each own version. */ - -#ifndef PIC - -/* The syscall stubs jump here when they detect an error. - The code for Linux is almost identical to the canonical Unix/i386 - code, except that the error number in %eax is negated. */ - -#undef CALL_MCOUNT -#define CALL_MCOUNT /* Don't insert the profiling call, it clobbers %eax. */ - - .text -ENTRY (__syscall_error) - negl %eax - -#define __syscall_error __syscall_error_1 -#include - -#endif /* !PIC */ diff --git a/sysdeps/unix/sysv/linux/i386/sysdep.c b/sysdeps/unix/sysv/linux/i386/sysdep.c new file mode 100644 index 0000000..2fcc6f8 --- /dev/null +++ b/sysdeps/unix/sysv/linux/i386/sysdep.c @@ -0,0 +1,30 @@ +/* Copyright (C) 2015 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include + +/* This routine is jumped to by all the syscall handlers, to stash + an error number into errno. ERROR is the negative error number + returned from the x86 kernel. */ +int +__attribute__ ((__regparm__ (1))) +__syscall_error (int error) +{ + __set_errno (-error); + return -1; +} diff --git a/sysdeps/unix/sysv/linux/i386/sysdep.h b/sysdeps/unix/sysv/linux/i386/sysdep.h index d76aca5..b43e0f7 100644 --- a/sysdeps/unix/sysv/linux/i386/sysdep.h +++ b/sysdeps/unix/sysv/linux/i386/sysdep.h @@ -55,11 +55,7 @@ /* We don't want the label for the error handle to be global when we define it here. */ -#ifdef PIC -# define SYSCALL_ERROR_LABEL 0f -#else -# define SYSCALL_ERROR_LABEL syscall_error -#endif +#define SYSCALL_ERROR_LABEL __syscall_error #undef PSEUDO #define PSEUDO(name, syscall_name, args) \ @@ -100,55 +96,7 @@ #define ret_ERRVAL ret -#ifndef PIC -# define SYSCALL_ERROR_HANDLER /* Nothing here; code in sysdep.S is used. */ -#else - -# if RTLD_PRIVATE_ERRNO -# define SYSCALL_ERROR_HANDLER \ -0:SETUP_PIC_REG(cx); \ - addl $_GLOBAL_OFFSET_TABLE_, %ecx; \ - negl %eax; \ - movl %eax, rtld_errno@GOTOFF(%ecx); \ - orl $-1, %eax; \ - ret; - -# elif defined _LIBC_REENTRANT - -# if IS_IN (libc) -# define SYSCALL_ERROR_ERRNO __libc_errno -# else -# define SYSCALL_ERROR_ERRNO errno -# endif -# define SYSCALL_ERROR_HANDLER \ -0:SETUP_PIC_REG (cx); \ - addl $_GLOBAL_OFFSET_TABLE_, %ecx; \ - movl SYSCALL_ERROR_ERRNO@GOTNTPOFF(%ecx), %ecx; \ - negl %eax; \ - SYSCALL_ERROR_HANDLER_TLS_STORE (%eax, %ecx); \ - orl $-1, %eax; \ - ret; -# ifndef NO_TLS_DIRECT_SEG_REFS -# define SYSCALL_ERROR_HANDLER_TLS_STORE(src, destoff) \ - movl src, %gs:(destoff) -# else -# define SYSCALL_ERROR_HANDLER_TLS_STORE(src, destoff) \ - addl %gs:0, destoff; \ - movl src, (destoff) -# endif -# else -/* Store (- %eax) into errno through the GOT. */ -# define SYSCALL_ERROR_HANDLER \ -0:SETUP_PIC_REG(cx); \ - addl $_GLOBAL_OFFSET_TABLE_, %ecx; \ - negl %eax; \ - movl errno@GOT(%ecx), %ecx; \ - movl %eax, (%ecx); \ - orl $-1, %eax; \ - ret; -# endif /* _LIBC_REENTRANT */ -#endif /* PIC */ - +#define SYSCALL_ERROR_HANDLER /* Nothing here; code in sysdep.c is used. */ /* The original calling convention for system calls on Linux/i386 is to use int $0x80. */ @@ -275,6 +223,10 @@ #else /* !__ASSEMBLER__ */ +extern int __syscall_error (int) + attribute_hidden __attribute__ ((__regparm__ (1))); + +#if !__GNUC_PREREQ (5,0) /* We need some help from the assembler to generate optimal code. We define some macros here which later will be used. */ asm (".L__X'%ebx = 1\n\t" @@ -314,6 +266,7 @@ struct libc_do_syscall_args { int ebx, edi, ebp; }; +#endif /* Define a macro which expands inline into the wrapper code for a system call. */ @@ -328,6 +281,28 @@ struct libc_do_syscall_args } \ (int) resultvar; }) +/* Define a macro which expands inline into the wrapper code for a system + call and return. */ +#undef INLINE_SYSCALL_RETURN +#define INLINE_SYSCALL_RETURN(name, nr, type, args...) \ + do \ + { \ + unsigned int resultvar = INTERNAL_SYSCALL (name, , nr, args); \ + if (__glibc_unlikely (INTERNAL_SYSCALL_ERROR_P (resultvar, ))) \ + INLINE_SYSCALL_ERROR_RETURN (resultvar, type, ); \ + return (type) resultvar; \ + } \ + while (0) + +/* Define a macro to return for a system call error. */ +#undef INLINE_SYSCALL_ERROR_RETURN +#define INLINE_SYSCALL_ERROR_RETURN(resultvar, type, value) \ + do \ + { \ + return (type) __syscall_error (resultvar); \ + } \ + while (0) + /* List of system calls which are supported as vsyscalls. */ # define HAVE_CLOCK_GETTIME_VSYSCALL 1 # define HAVE_GETTIMEOFDAY_VSYSCALL 1 @@ -354,8 +329,12 @@ struct libc_do_syscall_args INTERNAL_SYSCALL_MAIN_INLINE(name, err, 5, args) /* Each object using 6-argument inline syscalls must include a definition of __libc_do_syscall. */ -#define INTERNAL_SYSCALL_MAIN_6(name, err, arg1, arg2, arg3, \ - arg4, arg5, arg6) \ +#if __GNUC_PREREQ (5,0) +# define INTERNAL_SYSCALL_MAIN_6(name, err, args...) \ + INTERNAL_SYSCALL_MAIN_INLINE(name, err, 6, args) +#else /* GCC 5 */ +# define INTERNAL_SYSCALL_MAIN_6(name, err, arg1, arg2, arg3, \ + arg4, arg5, arg6) \ struct libc_do_syscall_args _xv = \ { \ (int) (arg1), \ @@ -368,14 +347,52 @@ struct libc_do_syscall_args : "=a" (resultvar) \ : "i" (__NR_##name), "c" (arg2), "d" (arg3), "S" (arg4), "D" (&_xv) \ : "memory", "cc") +#endif /* GCC 5 */ #define INTERNAL_SYSCALL(name, err, nr, args...) \ ({ \ register unsigned int resultvar; \ INTERNAL_SYSCALL_MAIN_##nr (name, err, args); \ (int) resultvar; }) #ifdef I386_USE_SYSENTER -# ifdef SHARED -# define INTERNAL_SYSCALL_MAIN_INLINE(name, err, nr, args...) \ +# if __GNUC_PREREQ (5,0) +# ifdef SHARED +# define INTERNAL_SYSCALL_MAIN_INLINE(name, err, nr, args...) \ + LOADREGS_##nr(args) \ + asm volatile ( \ + "call *%%gs:%P2" \ + : "=a" (resultvar) \ + : "a" (__NR_##name), "i" (offsetof (tcbhead_t, sysinfo)) \ + ASMARGS_##nr(args) : "memory", "cc") +# define INTERNAL_SYSCALL_NCS(name, err, nr, args...) \ + ({ \ + register unsigned int resultvar; \ + LOADREGS_##nr(args) \ + asm volatile ( \ + "call *%%gs:%P2" \ + : "=a" (resultvar) \ + : "a" (name), "i" (offsetof (tcbhead_t, sysinfo)) \ + ASMARGS_##nr(args) : "memory", "cc"); \ + (int) resultvar; }) +# else +# define INTERNAL_SYSCALL_MAIN_INLINE(name, err, nr, args...) \ + LOADREGS_##nr(args) \ + asm volatile ( \ + "call *_dl_sysinfo" \ + : "=a" (resultvar) \ + : "a" (__NR_##name) ASMARGS_##nr(args) : "memory", "cc") +# define INTERNAL_SYSCALL_NCS(name, err, nr, args...) \ + ({ \ + register unsigned int resultvar; \ + LOADREGS_##nr(args) \ + asm volatile ( \ + "call *_dl_sysinfo" \ + : "=a" (resultvar) \ + : "a" (name) ASMARGS_##nr(args) : "memory", "cc"); \ + (int) resultvar; }) +# endif +# else /* GCC 5 */ +# ifdef SHARED +# define INTERNAL_SYSCALL_MAIN_INLINE(name, err, nr, args...) \ EXTRAVAR_##nr \ asm volatile ( \ LOADARGS_##nr \ @@ -385,7 +402,7 @@ struct libc_do_syscall_args : "=a" (resultvar) \ : "i" (__NR_##name), "i" (offsetof (tcbhead_t, sysinfo)) \ ASMFMT_##nr(args) : "memory", "cc") -# define INTERNAL_SYSCALL_NCS(name, err, nr, args...) \ +# define INTERNAL_SYSCALL_NCS(name, err, nr, args...) \ ({ \ register unsigned int resultvar; \ EXTRAVAR_##nr \ @@ -397,8 +414,8 @@ struct libc_do_syscall_args : "0" (name), "i" (offsetof (tcbhead_t, sysinfo)) \ ASMFMT_##nr(args) : "memory", "cc"); \ (int) resultvar; }) -# else -# define INTERNAL_SYSCALL_MAIN_INLINE(name, err, nr, args...) \ +# else +# define INTERNAL_SYSCALL_MAIN_INLINE(name, err, nr, args...) \ EXTRAVAR_##nr \ asm volatile ( \ LOADARGS_##nr \ @@ -407,7 +424,7 @@ struct libc_do_syscall_args RESTOREARGS_##nr \ : "=a" (resultvar) \ : "i" (__NR_##name) ASMFMT_##nr(args) : "memory", "cc") -# define INTERNAL_SYSCALL_NCS(name, err, nr, args...) \ +# define INTERNAL_SYSCALL_NCS(name, err, nr, args...) \ ({ \ register unsigned int resultvar; \ EXTRAVAR_##nr \ @@ -418,9 +435,27 @@ struct libc_do_syscall_args : "=a" (resultvar) \ : "0" (name) ASMFMT_##nr(args) : "memory", "cc"); \ (int) resultvar; }) -# endif +# endif +# endif /* GCC 5 */ #else -# define INTERNAL_SYSCALL_MAIN_INLINE(name, err, nr, args...) \ +# if __GNUC_PREREQ (5,0) +# define INTERNAL_SYSCALL_MAIN_INLINE(name, err, nr, args...) \ + LOADREGS_##nr(args) \ + asm volatile ( \ + "int $0x80" \ + : "=a" (resultvar) \ + : "a" (__NR_##name) ASMARGS_##nr(args) : "memory", "cc") +# define INTERNAL_SYSCALL_NCS(name, err, nr, args...) \ + ({ \ + register unsigned int resultvar; \ + LOADREGS_##nr(args) \ + asm volatile ( \ + "int $0x80" \ + : "=a" (resultvar) \ + : "a" (name) ASMARGS_##nr(args) : "memory", "cc"); \ + (int) resultvar; }) +# else /* GCC 5 */ +# define INTERNAL_SYSCALL_MAIN_INLINE(name, err, nr, args...) \ EXTRAVAR_##nr \ asm volatile ( \ LOADARGS_##nr \ @@ -429,7 +464,7 @@ struct libc_do_syscall_args RESTOREARGS_##nr \ : "=a" (resultvar) \ : "i" (__NR_##name) ASMFMT_##nr(args) : "memory", "cc") -# define INTERNAL_SYSCALL_NCS(name, err, nr, args...) \ +# define INTERNAL_SYSCALL_NCS(name, err, nr, args...) \ ({ \ register unsigned int resultvar; \ EXTRAVAR_##nr \ @@ -440,6 +475,7 @@ struct libc_do_syscall_args : "=a" (resultvar) \ : "0" (name) ASMFMT_##nr(args) : "memory", "cc"); \ (int) resultvar; }) +# endif /* GCC 5 */ #endif #undef INTERNAL_SYSCALL_DECL @@ -504,6 +540,36 @@ struct libc_do_syscall_args # define RESTOREARGS_5 #endif +#if __GNUC_PREREQ (5,0) +# define LOADREGS_0() +# define ASMARGS_0() +# define LOADREGS_1(arg1) \ + LOADREGS_0 () +# define ASMARGS_1(arg1) \ + ASMARGS_0 (), "b" ((unsigned int) (arg1)) +# define LOADREGS_2(arg1, arg2) \ + LOADREGS_1 (arg1) +# define ASMARGS_2(arg1, arg2) \ + ASMARGS_1 (arg1), "c" ((unsigned int) (arg2)) +# define LOADREGS_3(arg1, arg2, arg3) \ + LOADREGS_2 (arg1, arg2) +# define ASMARGS_3(arg1, arg2, arg3) \ + ASMARGS_2 (arg1, arg2), "d" ((unsigned int) (arg3)) +# define LOADREGS_4(arg1, arg2, arg3, arg4) \ + LOADREGS_3 (arg1, arg2, arg3) +# define ASMARGS_4(arg1, arg2, arg3, arg4) \ + ASMARGS_3 (arg1, arg2, arg3), "S" ((unsigned int) (arg4)) +# define LOADREGS_5(arg1, arg2, arg3, arg4, arg5) \ + LOADREGS_4 (arg1, arg2, arg3, arg4) +# define ASMARGS_5(arg1, arg2, arg3, arg4, arg5) \ + ASMARGS_4 (arg1, arg2, arg3, arg4), "D" ((unsigned int) (arg5)) +# define LOADREGS_6(arg1, arg2, arg3, arg4, arg5, arg6) \ + register unsigned int _a6 asm ("ebp") = (unsigned int) (arg6); \ + LOADREGS_5 (arg1, arg2, arg3, arg4, arg5) +# define ASMARGS_6(arg1, arg2, arg3, arg4, arg5, arg6) \ + ASMARGS_5 (arg1, arg2, arg3, arg4, arg5), "r" (_a6) +#endif /* GCC 5 */ + #define ASMFMT_0() #ifdef __PIC__ # define ASMFMT_1(arg1) \ diff --git a/sysdeps/unix/sysv/linux/i386/xstat.c b/sysdeps/unix/sysv/linux/i386/xstat.c index 2424434..22a949b 100644 --- a/sysdeps/unix/sysv/linux/i386/xstat.c +++ b/sysdeps/unix/sysv/linux/i386/xstat.c @@ -38,15 +38,18 @@ __xstat (int vers, const char *name, struct stat *buf) int result; if (vers == _STAT_VER_KERNEL) - return INLINE_SYSCALL (stat, 2, name, (struct kernel_stat *) buf); + INLINE_SYSCALL_RETURN (stat, 2, int, name, + (struct kernel_stat *) buf); { struct stat64 buf64; - result = INLINE_SYSCALL (stat64, 2, name, &buf64); - if (result == 0) - result = __xstat32_conv (vers, &buf64, buf); - return result; + INTERNAL_SYSCALL_DECL (err); + result = INTERNAL_SYSCALL (stat64, err, 2, name, &buf64); + if (__glibc_unlikely (INTERNAL_SYSCALL_ERROR_P (result, ))) + INLINE_SYSCALL_ERROR_RETURN (result, int, ); + else + return __xstat32_conv (vers, &buf64, buf); } } hidden_def (__xstat) -- 2.4.3