[4/7] Consolidate Linux getdents{64} implementation

  On 02/03/2018 15:59, Adhemerval Zanella wrote:
> This patch consolidates Linux getdents{64} implementation on just
> the default sysdeps/unix/sysv/linux/getdents{64}{_r}.c ones.
> 
> Although this symbol is used only internally, the non-LFS version
> still need to be build due the non-LFS getdirentries which requires
> its semantic.
> 
> The non-LFS default implementation now uses the wordsize-32 as base
> which uses getdents64 syscall plus adjustment for overflow (it allows
> to use the same code for architectures that does not support non-LFS
> getdents syscall).  It has two main differences to wordsize-32 one:
> 
>   - DIRENT_SET_DP_INO is added to handle alpha requirement to zero
>     the padding.
> 
>   - alloca is removed by allocating a bounded temporary buffer (it
>     increases stack usage by roughly 276 bytes).
> 
> The default implementation handle the Linux requirements:
> 
>   * getdents is only built for _DIRENT_MATCHES_DIRENT64 being 0.
> 
>   * getdents64 is always built and aliased to getdents for ABIs
>     that define _DIRENT_MATCHES_DIRENT64 to 1.
> 
>   * A compat symbol is added for getdents64 for ABI that used to
>     export the old non-LFS version.
> 
> Checked on aarch64-linux-gnu, x86_64-linux-gnu, i686-linux-gnu,
> sparcv9-linux-gnu, sparc64-linux-gnu, powerpc-linux-gnu, and
> powerpc64le-linux-gnu.

Unfortunately MIPS n64 only wire up getdents64 on Linux 3.10, so we
still need to use the non-LFS one and adjust the buffer for the
ABI.  Below it is an updated patch for mips which first tried
getdents64 and if it fails fallbacks to old syscall.  I did a sniff
test on a qemu-system mips64 with dirent tests and saw no regressions.

---

	* sysdeps/unix/sysv/linux/alpha/getdents.c: Add comments with alpha
	requirements.
	 (_DIRENT_MATCHES_DIRENT64): Undef
	* sysdeps/unix/sysv/linux/alpha/getdents64.c: Likewise.
	* sysdeps/unix/sysv/linux/arm/getdents64.c: Remove file.
	* sysdeps/unix/sysv/linux/generic/getdents.c: Likewise.
	* sysdeps/unix/sysv/linux/generic/getdents64.c: Likewise.
	* sysdeps/unix/sysv/linux/generic/wordsize-32/getdents.c: Likewise.
	* sysdeps/unix/sysv/linux/getdents.c: Simplify implementation by
	use getdents64 syscalls as base.
	* sysdeps/unix/sysv/linux/getdents64.c: Likewise and add compatibility
	symbol if required.
	* sysdeps/unix/sysv/linux/hppa/getdents64.c: Likewise.
	* sysdeps/unix/sysv/linux/i386/getdents64.c: Likewise.
	* sysdeps/unix/sysv/linux/m68k/getdents64.c: Likewise.
	* sysdeps/unix/sysv/linux/powerpc/getdents64.c: Likewise.
	* sysdeps/unix/sysv/linux/s390/s390-32/getdents64.c: Likewise.
	* sysdeps/unix/sysv/linux/sparc/sparc32/getdents64.c: Likewise.
	* sysdeps/unix/sysv/linux/wordsize-64/getdents.c: Likewise.
	* sysdeps/unix/sysv/linux/wordsize-64/getdents64.c: Likewise.
	* sysdeps/unix/sysv/linux/sparc/sparc64/get_clockfreq.c
	(__get_clockfreq_via_proc_openprom): Use __getdents64.
	* sysdeps/unix/sysv/linux/mips/mips64/getdents64.c: New file.

---
  

I will commit this shortly if no one opposes it.
 
> Unfortunately MIPS n64 only wire up getdents64 on Linux 3.10, so we
> still need to use the non-LFS one and adjust the buffer for the
> ABI.  Below it is an updated patch for mips which first tried
> getdents64 and if it fails fallbacks to old syscall.  I did a sniff
> test on a qemu-system mips64 with dirent tests and saw no regressions.
> 
> ---
> 
> 	* sysdeps/unix/sysv/linux/alpha/getdents.c: Add comments with alpha
> 	requirements.
> 	 (_DIRENT_MATCHES_DIRENT64): Undef
> 	* sysdeps/unix/sysv/linux/alpha/getdents64.c: Likewise.
> 	* sysdeps/unix/sysv/linux/arm/getdents64.c: Remove file.
> 	* sysdeps/unix/sysv/linux/generic/getdents.c: Likewise.
> 	* sysdeps/unix/sysv/linux/generic/getdents64.c: Likewise.
> 	* sysdeps/unix/sysv/linux/generic/wordsize-32/getdents.c: Likewise.
> 	* sysdeps/unix/sysv/linux/getdents.c: Simplify implementation by
> 	use getdents64 syscalls as base.
> 	* sysdeps/unix/sysv/linux/getdents64.c: Likewise and add compatibility
> 	symbol if required.
> 	* sysdeps/unix/sysv/linux/hppa/getdents64.c: Likewise.
> 	* sysdeps/unix/sysv/linux/i386/getdents64.c: Likewise.
> 	* sysdeps/unix/sysv/linux/m68k/getdents64.c: Likewise.
> 	* sysdeps/unix/sysv/linux/powerpc/getdents64.c: Likewise.
> 	* sysdeps/unix/sysv/linux/s390/s390-32/getdents64.c: Likewise.
> 	* sysdeps/unix/sysv/linux/sparc/sparc32/getdents64.c: Likewise.
> 	* sysdeps/unix/sysv/linux/wordsize-64/getdents.c: Likewise.
> 	* sysdeps/unix/sysv/linux/wordsize-64/getdents64.c: Likewise.
> 	* sysdeps/unix/sysv/linux/sparc/sparc64/get_clockfreq.c
> 	(__get_clockfreq_via_proc_openprom): Use __getdents64.
> 	* sysdeps/unix/sysv/linux/mips/mips64/getdents64.c: New file.
> 
> ---
> 
> diff --git a/sysdeps/unix/sysv/linux/alpha/getdents.c b/sysdeps/unix/sysv/linux/alpha/getdents.c
> index dfecfef..64ccf86 100644
> --- a/sysdeps/unix/sysv/linux/alpha/getdents.c
> +++ b/sysdeps/unix/sysv/linux/alpha/getdents.c
> @@ -1,3 +1,11 @@
> +/* Although Alpha defines _DIRENT_MATCHES_DIRENT64, 'struct dirent' and
> +   'struct dirent64' have slight different internal layout with d_ino
> +   being a __ino_t on non-LFS version with an extra __pad field which should
> +   be zeroed.  */
> +
> +#include <dirent.h>
> +#undef _DIRENT_MATCHES_DIRENT64
> +#define _DIRENT_MATCHES_DIRENT64 0
>  #define DIRENT_SET_DP_INO(dp, value) \
>    do { (dp)->d_ino = (value); (dp)->__pad = 0; } while (0)
>  #include <sysdeps/unix/sysv/linux/getdents.c>
> diff --git a/sysdeps/unix/sysv/linux/alpha/getdents64.c b/sysdeps/unix/sysv/linux/alpha/getdents64.c
> index 50f1368..53cf93c 100644
> --- a/sysdeps/unix/sysv/linux/alpha/getdents64.c
> +++ b/sysdeps/unix/sysv/linux/alpha/getdents64.c
> @@ -1 +1,10 @@
> +/* Although Alpha defines _DIRENT_MATCHES_DIRENT64, 'struct dirent' and
> +   'struct dirent64' have slight different internal layout with d_ino
> +   being a __ino_t on non-LFS version with an extra __pad field which should
> +   be zeroed.  */
> +
> +#include <dirent.h>
> +/* It suppress the __getdents64 to __getdents alias.  */
> +#undef _DIRENT_MATCHES_DIRENT64
> +#define _DIRENT_MATCHES_DIRENT64 0
>  #include <sysdeps/unix/sysv/linux/getdents64.c>
> diff --git a/sysdeps/unix/sysv/linux/arm/getdents64.c b/sysdeps/unix/sysv/linux/arm/getdents64.c
> deleted file mode 100644
> index 0c75fb5..0000000
> --- a/sysdeps/unix/sysv/linux/arm/getdents64.c
> +++ /dev/null
> @@ -1 +0,0 @@
> -#include <sysdeps/unix/sysv/linux/i386/getdents64.c>
> diff --git a/sysdeps/unix/sysv/linux/generic/getdents.c b/sysdeps/unix/sysv/linux/generic/getdents.c
> deleted file mode 100644
> index 14dbbc7..0000000
> --- a/sysdeps/unix/sysv/linux/generic/getdents.c
> +++ /dev/null
> @@ -1 +0,0 @@
> -/* Defined in getdents64.c */
> diff --git a/sysdeps/unix/sysv/linux/generic/getdents64.c b/sysdeps/unix/sysv/linux/generic/getdents64.c
> deleted file mode 100644
> index 0f876b8..0000000
> --- a/sysdeps/unix/sysv/linux/generic/getdents64.c
> +++ /dev/null
> @@ -1,37 +0,0 @@
> -/* Copyright (C) 2011-2018 Free Software Foundation, Inc.
> -   This file is part of the GNU C Library.
> -   Contributed by Chris Metcalf <cmetcalf@tilera.com>, 2011.
> -
> -   The GNU C Library is free software; you can redistribute it and/or
> -   modify it under the terms of the GNU Lesser General Public
> -   License as published by the Free Software Foundation; either
> -   version 2.1 of the License, or (at your option) any later version.
> -
> -   The GNU C Library is distributed in the hope that it will be useful,
> -   but WITHOUT ANY WARRANTY; without even the implied warranty of
> -   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> -   Lesser General Public License for more details.
> -
> -   You should have received a copy of the GNU Lesser General Public
> -   License along with the GNU C Library.  If not, see
> -   <http://www.gnu.org/licenses/>.  */
> -
> -#include <stddef.h>
> -#include <stdint.h>
> -#include <unistd.h>
> -#include <sys/types.h>
> -#include <bits/wordsize.h>
> -
> -#include <sysdep.h>
> -#include <sys/syscall.h>
> -
> -/* The kernel struct linux_dirent64 matches the 'struct getdents64' type.  */
> -ssize_t
> -__getdents64 (int fd, char *buf, size_t nbytes)
> -{
> -  return INLINE_SYSCALL (getdents64, 3, fd, buf, nbytes);
> -}
> -
> -#if __WORDSIZE == 64
> -strong_alias (__getdents64, __getdents)
> -#endif
> diff --git a/sysdeps/unix/sysv/linux/generic/wordsize-32/getdents.c b/sysdeps/unix/sysv/linux/generic/wordsize-32/getdents.c
> deleted file mode 100644
> index 7158fd1..0000000
> --- a/sysdeps/unix/sysv/linux/generic/wordsize-32/getdents.c
> +++ /dev/null
> @@ -1,115 +0,0 @@
> -/* Copyright (C) 1993-2018 Free Software Foundation, Inc.
> -   This file is part of the GNU C Library.
> -   Simplified from sysdeps/unix/sysv/linux/getdents.c.
> -
> -   The GNU C Library is free software; you can redistribute it and/or
> -   modify it under the terms of the GNU Lesser General Public
> -   License as published by the Free Software Foundation; either
> -   version 2.1 of the License, or (at your option) any later version.
> -
> -   The GNU C Library is distributed in the hope that it will be useful,
> -   but WITHOUT ANY WARRANTY; without even the implied warranty of
> -   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> -   Lesser General Public License for more details.
> -
> -   You should have received a copy of the GNU Lesser General Public
> -   License along with the GNU C Library.  If not, see
> -   <http://www.gnu.org/licenses/>.  */
> -
> -#include <alloca.h>
> -#include <assert.h>
> -#include <errno.h>
> -#include <dirent.h>
> -#include <stddef.h>
> -#include <stdint.h>
> -#include <string.h>
> -#include <unistd.h>
> -#include <sys/param.h>
> -#include <sys/types.h>
> -
> -#include <sysdep.h>
> -#include <sys/syscall.h>
> -
> -/* Pack the dirent64 struct down into 32-bit offset/inode fields, and
> -   ensure that no overflow occurs.  */
> -ssize_t
> -__getdents (int fd, char *buf, size_t nbytes)
> -{
> -  union
> -  {
> -    struct dirent64 k;  /* Kernel structure.  */
> -    struct dirent u;
> -    char b[1];
> -  } *kbuf = (void *) buf, *outp, *inp;
> -  size_t kbytes = nbytes;
> -  off64_t last_offset = -1;
> -  ssize_t retval;
> -
> -  const size_t size_diff = (offsetof (struct dirent64, d_name)
> -                            - offsetof (struct dirent, d_name));
> -  if (nbytes <= sizeof (struct dirent))
> -    {
> -      kbytes = nbytes + offsetof (struct dirent64, d_name)
> -        - offsetof (struct dirent, d_name);
> -      kbuf = __alloca(kbytes);
> -    }
> -
> -  retval = INLINE_SYSCALL (getdents64, 3, fd, kbuf, kbytes);
> -  if (retval == -1)
> -    return -1;
> -
> -  /* These two pointers might alias the same memory buffer.
> -     Standard C requires that we always use the same type for them,
> -     so we must use the union type.  */
> -  inp = kbuf;
> -  outp = (void *) buf;
> -
> -  while (&inp->b < &kbuf->b + retval)
> -    {
> -      const size_t alignment = __alignof__ (struct dirent);
> -      /* Since inp->k.d_reclen is already aligned for the kernel
> -         structure this may compute a value that is bigger
> -         than necessary.  */
> -      size_t old_reclen = inp->k.d_reclen;
> -      size_t new_reclen = ((old_reclen - size_diff + alignment - 1)
> -                           & ~(alignment - 1));
> -
> -      /* Copy the data out of the old structure into temporary space.
> -         Then copy the name, which may overlap if BUF == KBUF.  */
> -      const uint64_t d_ino = inp->k.d_ino;
> -      const int64_t d_off = inp->k.d_off;
> -      const uint8_t d_type = inp->k.d_type;
> -
> -      memmove (outp->u.d_name, inp->k.d_name,
> -               old_reclen - offsetof (struct dirent64, d_name));
> -
> -      /* Now we have copied the data from INP and access only OUTP.  */
> -
> -      outp->u.d_ino = d_ino;
> -      outp->u.d_off = d_off;
> -      if ((sizeof (outp->u.d_ino) != sizeof (inp->k.d_ino)
> -           && outp->u.d_ino != d_ino)
> -          || (sizeof (outp->u.d_off) != sizeof (inp->k.d_off)
> -              && outp->u.d_off != d_off))
> -        {
> -          /* Overflow.  If there was at least one entry before this one,
> -             return them without error, otherwise signal overflow.  */
> -          if (last_offset != -1)
> -            {
> -              __lseek64 (fd, last_offset, SEEK_SET);
> -              return outp->b - buf;
> -            }
> -          __set_errno (EOVERFLOW);
> -          return -1;
> -        }
> -
> -      last_offset = d_off;
> -      outp->u.d_reclen = new_reclen;
> -      outp->u.d_type = d_type;
> -
> -      inp = (void *) inp + old_reclen;
> -      outp = (void *) outp + new_reclen;
> -    }
> -
> -  return outp->b - buf;
> -}
> diff --git a/sysdeps/unix/sysv/linux/getdents.c b/sysdeps/unix/sysv/linux/getdents.c
> index 591ce67..4e52729 100644
> --- a/sysdeps/unix/sysv/linux/getdents.c
> +++ b/sysdeps/unix/sysv/linux/getdents.c
> @@ -1,4 +1,5 @@
> -/* Copyright (C) 1993-2018 Free Software Foundation, Inc.
> +/* Get directory entries.  Linux no-LFS version.
> +   Copyright (C) 1993-2018 Free Software Foundation, Inc.
>     This file is part of the GNU C Library.
>  
>     The GNU C Library is free software; you can redistribute it and/or
> @@ -12,260 +13,102 @@
>     Lesser General Public License for more details.
>  
>     You should have received a copy of the GNU Lesser General Public
> -   License along with the GNU C Library; if not, see
> +   License along with the GNU C Library.  If not, see
>     <http://www.gnu.org/licenses/>.  */
>  
> -#include <alloca.h>
> -#include <assert.h>
> -#include <errno.h>
>  #include <dirent.h>
> -#include <stddef.h>
> -#include <stdint.h>
> -#include <string.h>
> -#include <unistd.h>
> -#include <sys/param.h>
> -#include <sys/types.h>
>  
> -#include <sysdep.h>
> -#include <sys/syscall.h>
> +#if !_DIRENT_MATCHES_DIRENT64
>  
> -#include <linux/posix_types.h>
> +# include <unistd.h>
> +# include <string.h>
> +# include <errno.h>
>  
> -#include <kernel-features.h>
> +# ifndef DIRENT_SET_DP_INO
> +#  define DIRENT_SET_DP_INO(dp, value) (dp)->d_ino = (value)
> +# endif
>  
> -/* For Linux we need a special version of this file since the
> -   definition of `struct dirent' is not the same for the kernel and
> -   the libc.  There is one additional field which might be introduced
> -   in the kernel structure in the future.
> -
> -   Here is the kernel definition of `struct dirent' as of 2.1.20:  */
> -
> -struct kernel_dirent
> -  {
> -    long int d_ino;
> -    __kernel_off_t d_off;
> -    unsigned short int d_reclen;
> -    char d_name[256];
> -  };
> -
> -struct kernel_dirent64
> -  {
> -    uint64_t		d_ino;
> -    int64_t		d_off;
> -    unsigned short int	d_reclen;
> -    unsigned char	d_type;
> -    char		d_name[256];
> -  };
> -
> -#ifndef __GETDENTS
> -# define __GETDENTS __getdents
> -#endif
> -#ifndef DIRENT_TYPE
> -# define DIRENT_TYPE struct dirent
> -#endif
> -#ifndef DIRENT_SET_DP_INO
> -# define DIRENT_SET_DP_INO(dp, value) (dp)->d_ino = (value)
> -#endif
> -
> -/* The problem here is that we cannot simply read the next NBYTES
> -   bytes.  We need to take the additional field into account.  We use
> -   some heuristic.  Assuming the directory contains names with 14
> -   characters on average we can compute an estimated number of entries
> -   which fit in the buffer.  Taking this number allows us to specify a
> -   reasonable number of bytes to read.  If we should be wrong, we can
> -   reset the file descriptor.  In practice the kernel is limiting the
> -   amount of data returned much more then the reduced buffer size.  */
> +/* Pack the dirent64 struct down into 32-bit offset/inode fields, and
> +   ensure that no overflow occurs.  */
>  ssize_t
> -__GETDENTS (int fd, char *buf, size_t nbytes)
> +__getdents (int fd, char *buf, size_t nbytes)
>  {
> +  union
> +  {
> +    struct dirent64 k;  /* Kernel structure.  */
> +    struct dirent u;
> +    char b[1];
> +  } *kbuf = (void *) buf, *outp, *inp;
> +  size_t kbytes = nbytes;
> +  off64_t last_offset = -1;
>    ssize_t retval;
>  
> -  /* The d_ino and d_off fields in kernel_dirent and dirent must have
> -     the same sizes and alignments.  */
> -  if (sizeof (DIRENT_TYPE) == sizeof (struct dirent)
> -      && (sizeof (((struct kernel_dirent *) 0)->d_ino)
> -	  == sizeof (((struct dirent *) 0)->d_ino))
> -      && (sizeof (((struct kernel_dirent *) 0)->d_off)
> -	  == sizeof (((struct dirent *) 0)->d_off))
> -      && (offsetof (struct kernel_dirent, d_off)
> -	  == offsetof (struct dirent, d_off))
> -      && (offsetof (struct kernel_dirent, d_reclen)
> -	  == offsetof (struct dirent, d_reclen)))
> -    {
> -      retval = INLINE_SYSCALL (getdents, 3, fd, buf, nbytes);
> +# define size_diff (offsetof (struct dirent64, d_name) \
> +		    - offsetof (struct dirent, d_name))
> +  char kbuftmp[sizeof (struct dirent) + size_diff];
> +  if (nbytes <= sizeof (struct dirent))
> +    kbuf = (void*) kbuftmp;
>  
> -      /* The kernel added the d_type value after the name.  Change
> -	 this now.  */
> -      if (retval != -1)
> -	{
> -	  union
> -	  {
> -	    struct kernel_dirent k;
> -	    struct dirent u;
> -	  } *kbuf = (void *) buf;
> +  retval = INLINE_SYSCALL_CALL (getdents64, fd, kbuf, kbytes);
> +  if (retval == -1)
> +    return -1;
>  
> -	  while ((char *) kbuf < buf + retval)
> -	    {
> -	      char d_type = *((char *) kbuf + kbuf->k.d_reclen - 1);
> -	      memmove (kbuf->u.d_name, kbuf->k.d_name,
> -		       strlen (kbuf->k.d_name) + 1);
> -	      kbuf->u.d_type = d_type;
> +  /* These two pointers might alias the same memory buffer.
> +     Standard C requires that we always use the same type for them,
> +     so we must use the union type.  */
> +  inp = kbuf;
> +  outp = (void *) buf;
>  
> -	      kbuf = (void *) ((char *) kbuf + kbuf->k.d_reclen);
> -	    }
> -	}
> -
> -      return retval;
> -    }
> -
> -  off64_t last_offset = -1;
> -
> -#ifdef __NR_getdents64
> -  {
> -    union
> +  while (&inp->b < &kbuf->b + retval)
>      {
> -      struct kernel_dirent64 k;
> -      DIRENT_TYPE u;
> -      char b[1];
> -    } *kbuf = (void *) buf, *outp, *inp;
> -    size_t kbytes = nbytes;
> -    if (offsetof (DIRENT_TYPE, d_name)
> -	< offsetof (struct kernel_dirent64, d_name)
> -	&& nbytes <= sizeof (DIRENT_TYPE))
> -      {
> -	kbytes = (nbytes + offsetof (struct kernel_dirent64, d_name)
> -		  - offsetof (DIRENT_TYPE, d_name));
> -	kbuf = __alloca(kbytes);
> -      }
> -    retval = INLINE_SYSCALL (getdents64, 3, fd, kbuf, kbytes);
> -    const size_t size_diff = (offsetof (struct kernel_dirent64, d_name)
> -			      - offsetof (DIRENT_TYPE, d_name));
> -
> -    /* Return the error if encountered.  */
> -    if (retval == -1)
> -      return -1;
> -
> -    /* If the structure returned by the kernel is identical to what we
> -       need, don't do any conversions.  */
> -    if (offsetof (DIRENT_TYPE, d_name)
> -	== offsetof (struct kernel_dirent64, d_name)
> -	&& sizeof (outp->u.d_ino) == sizeof (inp->k.d_ino)
> -	&& sizeof (outp->u.d_off) == sizeof (inp->k.d_off))
> -      return retval;
> -
> -    /* These two pointers might alias the same memory buffer.
> -       Standard C requires that we always use the same type for them,
> -       so we must use the union type.  */
> -    inp = kbuf;
> -    outp = (void *) buf;
> -
> -    while (&inp->b < &kbuf->b + retval)
> -      {
> -	const size_t alignment = __alignof__ (DIRENT_TYPE);
> -	/* Since inp->k.d_reclen is already aligned for the kernel
> -	   structure this may compute a value that is bigger
> -	   than necessary.  */
> -	size_t old_reclen = inp->k.d_reclen;
> -	size_t new_reclen = ((old_reclen - size_diff + alignment - 1)
> -			     & ~(alignment - 1));
> -
> -	/* Copy the data out of the old structure into temporary space.
> -	   Then copy the name, which may overlap if BUF == KBUF.  */
> -	const uint64_t d_ino = inp->k.d_ino;
> -	const int64_t d_off = inp->k.d_off;
> -	const uint8_t d_type = inp->k.d_type;
> -
> -	memmove (outp->u.d_name, inp->k.d_name,
> -		 old_reclen - offsetof (struct kernel_dirent64, d_name));
> -
> -	/* Now we have copied the data from INP and access only OUTP.  */
> -
> -	DIRENT_SET_DP_INO (&outp->u, d_ino);
> -	outp->u.d_off = d_off;
> -	if ((sizeof (outp->u.d_ino) != sizeof (inp->k.d_ino)
> -	     && outp->u.d_ino != d_ino)
> -	    || (sizeof (outp->u.d_off) != sizeof (inp->k.d_off)
> -		&& outp->u.d_off != d_off))
> -	  {
> -	    /* Overflow.  If there was at least one entry
> -	       before this one, return them without error,
> -	       otherwise signal overflow.  */
> -	    if (last_offset != -1)
> -	      {
> -		__lseek64 (fd, last_offset, SEEK_SET);
> -		return outp->b - buf;
> -	      }
> -	    __set_errno (EOVERFLOW);
> -	    return -1;
> -	  }
> -
> -	last_offset = d_off;
> -	outp->u.d_reclen = new_reclen;
> -	outp->u.d_type = d_type;
> -
> -	inp = (void *) inp + old_reclen;
> -	outp = (void *) outp + new_reclen;
> -      }
> -
> -    return outp->b - buf;
> -  }
> -#endif
> -  {
> -    size_t red_nbytes;
> -    struct kernel_dirent *skdp, *kdp;
> -    const size_t size_diff = (offsetof (DIRENT_TYPE, d_name)
> -			      - offsetof (struct kernel_dirent, d_name));
> -
> -    red_nbytes = MIN (nbytes
> -		      - ((nbytes / (offsetof (DIRENT_TYPE, d_name) + 14))
> -			 * size_diff),
> -		      nbytes - size_diff);
> -
> -    skdp = kdp = __alloca (red_nbytes);
> -
> -    retval = INLINE_SYSCALL (getdents, 3, fd, (char *) kdp, red_nbytes);
> -
> -    if (retval == -1)
> -      return -1;
> -
> -    DIRENT_TYPE *dp = (DIRENT_TYPE *) buf;
> -    while ((char *) kdp < (char *) skdp + retval)
> -      {
> -	const size_t alignment = __alignof__ (DIRENT_TYPE);
> -	/* Since kdp->d_reclen is already aligned for the kernel structure
> -	   this may compute a value that is bigger than necessary.  */
> -	size_t new_reclen = ((kdp->d_reclen + size_diff + alignment - 1)
> -			     & ~(alignment - 1));
> -	if ((char *) dp + new_reclen > buf + nbytes)
> -	  {
> -	    /* Our heuristic failed.  We read too many entries.  Reset
> -	       the stream.  */
> -	    assert (last_offset != -1);
> -	    __lseek64 (fd, last_offset, SEEK_SET);
> -
> -	    if ((char *) dp == buf)
> -	      {
> -		/* The buffer the user passed in is too small to hold even
> -		   one entry.  */
> -		__set_errno (EINVAL);
> -		return -1;
> -	      }
> -
> -	    break;
> -	  }
> +      const size_t alignment = __alignof__ (struct dirent);
> +      /* Since inp->k.d_reclen is already aligned for the kernel
> +         structure this may compute a value that is bigger
> +         than necessary.  */
> +      size_t old_reclen = inp->k.d_reclen;
> +      size_t new_reclen = ((old_reclen - size_diff + alignment - 1)
> +                           & ~(alignment - 1));
> +
> +      /* Copy the data out of the old structure into temporary space.
> +         Then copy the name, which may overlap if BUF == KBUF.  */
> +      const uint64_t d_ino = inp->k.d_ino;
> +      const int64_t d_off = inp->k.d_off;
> +      const uint8_t d_type = inp->k.d_type;
> +
> +      memmove (outp->u.d_name, inp->k.d_name,
> +               old_reclen - offsetof (struct dirent64, d_name));
> +
> +      /* Now we have copied the data from INP and access only OUTP.  */
> +
> +      DIRENT_SET_DP_INO (&outp->u, d_ino);
> +      outp->u.d_off = d_off;
> +      if ((sizeof (outp->u.d_ino) != sizeof (inp->k.d_ino)
> +           && outp->u.d_ino != d_ino)
> +          || (sizeof (outp->u.d_off) != sizeof (inp->k.d_off)
> +              && outp->u.d_off != d_off))
> +        {
> +          /* Overflow.  If there was at least one entry before this one,
> +             return them without error, otherwise signal overflow.  */
> +          if (last_offset != -1)
> +            {
> +              __lseek64 (fd, last_offset, SEEK_SET);
> +              return outp->b - buf;
> +            }
> +          __set_errno (EOVERFLOW);
> +          return -1;
> +        }
> +
> +      last_offset = d_off;
> +      outp->u.d_reclen = new_reclen;
> +      outp->u.d_type = d_type;
> +
> +      inp = (void *) inp + old_reclen;
> +      outp = (void *) outp + new_reclen;
> +    }
>  
> -	last_offset = kdp->d_off;
> -	DIRENT_SET_DP_INO(dp, kdp->d_ino);
> -	dp->d_off = kdp->d_off;
> -	dp->d_reclen = new_reclen;
> -	dp->d_type = *((char *) kdp + kdp->d_reclen - 1);
> -	memcpy (dp->d_name, kdp->d_name,
> -		kdp->d_reclen - offsetof (struct kernel_dirent, d_name));
> +  return outp->b - buf;
> +}
>  
> -	dp = (DIRENT_TYPE *) ((char *) dp + new_reclen);
> -	kdp = (struct kernel_dirent *) (((char *) kdp) + kdp->d_reclen);
> -      }
> +# undef DIRENT_SET_DP_INO
>  
> -    return (char *) dp - buf;
> -  }
> -}
> +#endif /* _DIRENT_MATCHES_DIRENT64  */
> diff --git a/sysdeps/unix/sysv/linux/getdents64.c b/sysdeps/unix/sysv/linux/getdents64.c
> index 805917e..f37fe15 100644
> --- a/sysdeps/unix/sysv/linux/getdents64.c
> +++ b/sysdeps/unix/sysv/linux/getdents64.c
> @@ -1,3 +1,77 @@
> -#define __GETDENTS __getdents64
> -#define DIRENT_TYPE struct dirent64
> -#include <sysdeps/unix/sysv/linux/getdents.c>
> +/* Get directory entries.  Linux LFS version.
> +   Copyright (C) 1997-2018 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library.  If not, see
> +   <http://www.gnu.org/licenses/>.  */
> +
> +#include <string.h>
> +#include <dirent.h>
> +#include <errno.h>
> +
> +/* The kernel struct linux_dirent64 matches the 'struct getdents64' type.  */
> +ssize_t
> +__getdents64 (int fd, char *buf, size_t nbytes)
> +{
> +  return INLINE_SYSCALL_CALL (getdents64, fd, buf, nbytes);
> +}
> +
> +#if _DIRENT_MATCHES_DIRENT64
> +strong_alias (__getdents64, __getdents)
> +#else
> +# include <shlib-compat.h>
> +
> +# if SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2)
> +
> +# include <olddirent.h>
> +
> +/* kernel definition of as of 3.2.  */
> +struct compat_linux_dirent
> +{
> +  /* Both d_ino and d_off are compat_ulong_t which are defined in all
> +     architectures as 'u32'.  */
> +  uint32_t        d_ino;
> +  uint32_t        d_off;
> +  unsigned short  d_reclen;
> +  char            d_name[1];
> +};
> +
> +ssize_t
> +__old_getdents64 (int fd, char *buf, size_t nbytes)
> +{
> +  ssize_t retval = INLINE_SYSCALL_CALL (getdents, fd, buf, nbytes);
> +
> +  /* The kernel added the d_type value after the name.  Change this now.  */
> +  if (retval != -1)
> +    {
> +      union
> +      {
> +	struct compat_linux_dirent k;
> +	struct dirent u;
> +      } *kbuf = (void *) buf;
> +
> +      while ((char *) kbuf < buf + retval)
> +	{
> +	  char d_type = *((char *) kbuf + kbuf->k.d_reclen - 1);
> +	  memmove (kbuf->u.d_name, kbuf->k.d_name,
> +		   strlen (kbuf->k.d_name) + 1);
> +	  kbuf->u.d_type = d_type;
> +
> +	  kbuf = (void *) ((char *) kbuf + kbuf->k.d_reclen);
> +	}
> +     }
> +  return retval;
> +}
> +# endif /* SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2)  */
> +#endif /* _DIRENT_MATCHES_DIRENT64  */
> diff --git a/sysdeps/unix/sysv/linux/hppa/getdents64.c b/sysdeps/unix/sysv/linux/hppa/getdents64.c
> deleted file mode 100644
> index 0c75fb5..0000000
> --- a/sysdeps/unix/sysv/linux/hppa/getdents64.c
> +++ /dev/null
> @@ -1 +0,0 @@
> -#include <sysdeps/unix/sysv/linux/i386/getdents64.c>
> diff --git a/sysdeps/unix/sysv/linux/i386/getdents64.c b/sysdeps/unix/sysv/linux/i386/getdents64.c
> deleted file mode 100644
> index 0a2c194..0000000
> --- a/sysdeps/unix/sysv/linux/i386/getdents64.c
> +++ /dev/null
> @@ -1,39 +0,0 @@
> -/* Copyright (C) 2000-2018 Free Software Foundation, Inc.
> -   This file is part of the GNU C Library.
> -
> -   The GNU C Library is free software; you can redistribute it and/or
> -   modify it under the terms of the GNU Lesser General Public
> -   License as published by the Free Software Foundation; either
> -   version 2.1 of the License, or (at your option) any later version.
> -
> -   The GNU C Library is distributed in the hope that it will be useful,
> -   but WITHOUT ANY WARRANTY; without even the implied warranty of
> -   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> -   Lesser General Public License for more details.
> -
> -   You should have received a copy of the GNU Lesser General Public
> -   License along with the GNU C Library; if not, see
> -   <http://www.gnu.org/licenses/>.  */
> -
> -#define __GETDENTS __getdents64
> -#define DIRENT_TYPE struct dirent64
> -
> -#include <sysdeps/unix/sysv/linux/getdents.c>
> -
> -#include <shlib-compat.h>
> -
> -#undef __READDIR
> -#undef __GETDENTS
> -#undef DIRENT_TYPE
> -
> -#if SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2)
> -
> -#include <olddirent.h>
> -
> -#define __GETDENTS __old_getdents64
> -#define DIRENT_TYPE struct __old_dirent64
> -#define kernel_dirent old_kernel_dirent
> -#define kernel_dirent64 old_kernel_dirent64
> -
> -#include <sysdeps/unix/sysv/linux/getdents.c>
> -#endif
> diff --git a/sysdeps/unix/sysv/linux/m68k/getdents64.c b/sysdeps/unix/sysv/linux/m68k/getdents64.c
> deleted file mode 100644
> index 0c75fb5..0000000
> --- a/sysdeps/unix/sysv/linux/m68k/getdents64.c
> +++ /dev/null
> @@ -1 +0,0 @@
> -#include <sysdeps/unix/sysv/linux/i386/getdents64.c>
> diff --git a/sysdeps/unix/sysv/linux/mips/mips64/getdents64.c b/sysdeps/unix/sysv/linux/mips/mips64/getdents64.c
> new file mode 100644
> index 0000000..5b62791
> --- /dev/null
> +++ b/sysdeps/unix/sysv/linux/mips/mips64/getdents64.c
> @@ -0,0 +1,113 @@
> +/* Get directory entries.  Linux/MIPSn64 LFS version.
> +   Copyright (C) 2018 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library.  If not, see
> +   <http://www.gnu.org/licenses/>.  */
> +
> +#include <string.h>
> +#include <dirent.h>
> +#include <errno.h>
> +#include <assert.h>
> +#include <sys/param.h>
> +#include <unistd.h>
> +#include <scratch_buffer.h>
> +
> +ssize_t
> +__getdents64 (int fd, char *buf, size_t nbytes)
> +{
> +  ssize_t ret;
> +#ifdef __NR_getdents64
> +  ret = INLINE_SYSCALL_CALL (getdents64, fd, buf, nbytes);
> +  if (ret != -1)
> +    return ret;
> +#endif
> +
> +  /* Unfortunately getdents64 was only wire-up for MIPS n64 on Linux 3.10.
> +     If syscall is not available it need to fallback to old one.  */
> +
> +  struct kernel_dirent
> +    {
> +      unsigned long d_ino;
> +      unsigned long d_off;
> +      unsigned short int d_reclen;
> +      char d_name[256];
> +    };
> +
> +  const size_t size_diff = (offsetof (struct dirent64, d_name)
> +			   - offsetof (struct kernel_dirent, d_name));
> +
> +  size_t red_nbytes = MIN (nbytes
> +			   - ((nbytes / (offsetof (struct dirent64, d_name)
> +					 + 14)) * size_diff),
> +			   nbytes - size_diff);
> +
> +  struct scratch_buffer tmpbuf;
> +  scratch_buffer_init (&tmpbuf);
> +  if (!scratch_buffer_set_array_size (&tmpbuf, red_nbytes, sizeof (uint8_t)))
> +    INLINE_SYSCALL_ERROR_RETURN_VALUE (ENOMEM);
> +
> +  struct kernel_dirent *skdp, *kdp;
> +  skdp = kdp = tmpbuf.data;
> +
> +  ssize_t retval = INLINE_SYSCALL_CALL (getdents, fd, kdp, red_nbytes);
> +  if (retval == -1)
> +    {
> +      scratch_buffer_free (&tmpbuf);
> +      return -1;
> +    }
> +
> +  off64_t last_offset = -1;
> +  struct dirent64 *dp = (struct dirent64 *) buf;
> +  while ((char *) kdp < (char *) skdp + retval)
> +    {
> +      const size_t alignment = __alignof__ (struct dirent64);
> +      /* Since kdp->d_reclen is already aligned for the kernel structure
> +	 this may compute a value that is bigger than necessary.  */
> +      size_t new_reclen = ((kdp->d_reclen + size_diff + alignment - 1)
> +			   & ~(alignment - 1));
> +      if ((char *) dp + new_reclen > buf + nbytes)
> +        {
> +	  /* Our heuristic failed.  We read too many entries.  Reset
> +	     the stream.  */
> +	  assert (last_offset != -1);
> +	  __lseek64 (fd, last_offset, SEEK_SET);
> +
> +	  if ((char *) dp == buf)
> +	    {
> +	      scratch_buffer_free (&tmpbuf);
> +	      INLINE_SYSCALL_ERROR_RETURN_VALUE (EINVAL);
> +	    }
> +
> +	  break;
> +	}
> +
> +      last_offset = kdp->d_off;
> +      dp->d_ino = kdp->d_ino;
> +      dp->d_off = kdp->d_off;
> +      dp->d_reclen = new_reclen;
> +      dp->d_type = *((char *) kdp + kdp->d_reclen - 1);
> +      memcpy (dp->d_name, kdp->d_name,
> +	      kdp->d_reclen - offsetof (struct kernel_dirent, d_name));
> +
> +      dp = (struct dirent64 *) ((char *) dp + new_reclen);
> +      kdp = (struct kernel_dirent *) (((char *) kdp) + kdp->d_reclen);
> +    }
> +
> +  scratch_buffer_free (&tmpbuf);
> +  return (char *) dp - buf;
> +}
> +#if _DIRENT_MATCHES_DIRENT64
> +strong_alias (__getdents64, __getdents)
> +#endif
> diff --git a/sysdeps/unix/sysv/linux/powerpc/getdents64.c b/sysdeps/unix/sysv/linux/powerpc/getdents64.c
> deleted file mode 100644
> index 0c75fb5..0000000
> --- a/sysdeps/unix/sysv/linux/powerpc/getdents64.c
> +++ /dev/null
> @@ -1 +0,0 @@
> -#include <sysdeps/unix/sysv/linux/i386/getdents64.c>
> diff --git a/sysdeps/unix/sysv/linux/s390/s390-32/getdents64.c b/sysdeps/unix/sysv/linux/s390/s390-32/getdents64.c
> deleted file mode 100644
> index 0c75fb5..0000000
> --- a/sysdeps/unix/sysv/linux/s390/s390-32/getdents64.c
> +++ /dev/null
> @@ -1 +0,0 @@
> -#include <sysdeps/unix/sysv/linux/i386/getdents64.c>
> diff --git a/sysdeps/unix/sysv/linux/sparc/sparc32/getdents64.c b/sysdeps/unix/sysv/linux/sparc/sparc32/getdents64.c
> deleted file mode 100644
> index 0c75fb5..0000000
> --- a/sysdeps/unix/sysv/linux/sparc/sparc32/getdents64.c
> +++ /dev/null
> @@ -1 +0,0 @@
> -#include <sysdeps/unix/sysv/linux/i386/getdents64.c>
> diff --git a/sysdeps/unix/sysv/linux/sparc/sparc64/get_clockfreq.c b/sysdeps/unix/sysv/linux/sparc/sparc64/get_clockfreq.c
> index c54d301..6838a77 100644
> --- a/sysdeps/unix/sysv/linux/sparc/sparc64/get_clockfreq.c
> +++ b/sysdeps/unix/sysv/linux/sparc/sparc64/get_clockfreq.c
> @@ -90,12 +90,12 @@ __get_clockfreq_via_proc_openprom (void)
>    if (obp_fd != -1)
>      {
>        unsigned long int buf[4096 / sizeof (unsigned long int)];
> -      struct dirent *dirp = (struct dirent *) buf;
> +      struct dirent64 *dirp = (struct dirent64 *) buf;
>        ssize_t len;
>  
> -      while ((len = __getdents (obp_fd, (char *) dirp, sizeof (buf))) > 0)
> +      while ((len = __getdents64 (obp_fd, (char *) dirp, sizeof (buf))) > 0)
>  	{
> -	  struct dirent *this_dirp = dirp;
> +	  struct dirent64 *this_dirp = dirp;
>  
>  	  while (len > 0)
>  	    {
> @@ -140,7 +140,7 @@ __get_clockfreq_via_proc_openprom (void)
>  		break;
>  
>  	      len -= this_dirp->d_reclen;
> -	      this_dirp = (struct dirent *)
> +	      this_dirp = (struct dirent64 *)
>  		((char *) this_dirp + this_dirp->d_reclen);
>  	    }
>  	  if (result != 0)
> diff --git a/sysdeps/unix/sysv/linux/wordsize-64/getdents.c b/sysdeps/unix/sysv/linux/wordsize-64/getdents.c
> deleted file mode 100644
> index 5ea4c57..0000000
> --- a/sysdeps/unix/sysv/linux/wordsize-64/getdents.c
> +++ /dev/null
> @@ -1,4 +0,0 @@
> -#define __getdents64 __no___getdents64_decl
> -#include <sysdeps/unix/sysv/linux/getdents.c>
> -#undef __getdents64
> -weak_alias (__getdents, __getdents64);
> diff --git a/sysdeps/unix/sysv/linux/wordsize-64/getdents64.c b/sysdeps/unix/sysv/linux/wordsize-64/getdents64.c
> deleted file mode 100644
> index 0df2c8f..0000000
> --- a/sysdeps/unix/sysv/linux/wordsize-64/getdents64.c
> +++ /dev/null
> @@ -1 +0,0 @@
> -/* getdents64 is in getdents.c */
>
  

I'm seeing:

/scratch/jmyers/glibc-bot/build/glibcs/mips64-linux-gnu-n32/glibc/libc_pic.a(getdents64.os): In function `__getdents64':
/scratch/jmyers/glibc-bot/build/glibcs/mips64-linux-gnu-n32/glibc-src/dirent/../sysdeps/unix/sysv/linux/mips/mips64/getdents64.c:29: multiple definition of `__getdents'
/scratch/jmyers/glibc-bot/build/glibcs/mips64-linux-gnu-n32/glibc/libc_pic.a(getdents.os):/scratch/jmyers/glibc-bot/build/glibcs/mips64-linux-gnu-n32/glibc-src/dirent/../sysdeps/unix/sysv/linux/getdents.c:35: first defined here

for which this change seems the likely cause.

https://sourceware.org/ml/libc-testresults/2018-q2/msg00093.html
  

On 19/04/2018 14:29, Joseph Myers wrote:
> I'm seeing:
> 
> /scratch/jmyers/glibc-bot/build/glibcs/mips64-linux-gnu-n32/glibc/libc_pic.a(getdents64.os): In function `__getdents64':
> /scratch/jmyers/glibc-bot/build/glibcs/mips64-linux-gnu-n32/glibc-src/dirent/../sysdeps/unix/sysv/linux/mips/mips64/getdents64.c:29: multiple definition of `__getdents'
> /scratch/jmyers/glibc-bot/build/glibcs/mips64-linux-gnu-n32/glibc/libc_pic.a(getdents.os):/scratch/jmyers/glibc-bot/build/glibcs/mips64-linux-gnu-n32/glibc-src/dirent/../sysdeps/unix/sysv/linux/getdents.c:35: first defined here
> 
> for which this change seems the likely cause.
> 
> https://sourceware.org/ml/libc-testresults/2018-q2/msg00093.html
> 

It was a wrong last minute change, where I forgot that mips64n32
do not define _DIRENT_MATCHES_DIRENT64. I fixed it on master, thanks
for bring this up and sorry for the trouble.