Patchwork [v3] zic, various tests: use 64-bit stat/ftw/readdir functions

login
register
mail settings
Submitter Nix
Date June 26, 2017, 10:16 p.m.
Message ID <87lgoez96k.fsf@esperi.org.uk>
Download mbox | patch
Permalink /patch/21270/
State New
Headers show

Comments

Nix - June 26, 2017, 10:16 p.m.
From: Nick Alcock <nick.alcock@oracle.com>

This fixes this compilation failure when testing a 32-bit glibc on a
system using a 64-bit-inode XFS filesystem, and a variety of related
test failures of dirent/list, io/bug-ftw2, and posix/tst-regex:

env GCONV_PATH=/usr/src/glibc/i686-loom/iconvdata LOCPATH=/usr/src/glibc/i686-loom/localedata LC_ALL=C  /usr/src/glibc/i686-loom/elf/ld-linux.so.2 --library-path /usr/src/glibc/i686-loom:/usr/src/glibc/i686-loom/math:/usr/src/glibc/i686-loom/elf:/usr/src/glibc/i686-loom/dlfcn:/usr/src/glibc/i686-loom/nss:/usr/src/glibc/i686-loom/nis:/usr/src/glibc/i686-loom/rt:/usr/src/glibc/i686-loom/resolv:/usr/src/glibc/i686-loom/crypt:/usr/src/glibc/i686-loom/mathvec:/usr/src/glibc/i686-loom/support:/usr/src/glibc/i686-loom/nptl /usr/src/glibc/i686-loom/timezone/zic -d /usr/src/glibc/i686-loom/timezone/testdata -y ./yearistype australasia; ../scripts/evaluate-test.sh timezone/testdata/Australia/Melbourne $? false false > /usr/src/glibc/i686-loom/timezone/testdata/Australia/Melbourne.test-result
warning: "etcetera", line 13: /usr/src/glibc/i686-loom/timezone/zic: Can't create directory /usr/src: File exists
/bin/sh: /usr/src/glibc/i686-loom/timezone/testdata/Etc/UTC.test-result: No such file or directory
make[2]: *** [Makefile:98: /usr/src/glibc/i686-loom/timezone/testdata/Etc/UTC] Error 1

This happens because itsdir() in timezone/zic.c does a stat() of each
element of the path in turn, and this returns -EOVERFLOW because on this
system /usr has an inode number of 7516193792 and we did not compile zic
with -D_FILE_OFFSET_BITS=64 or use stat64().  Do the former in
tz-cflags, to avoid unnecessarily skewing zic from its upstream.

We can fix the other test failures via use of the *64() functions
and by adding 64-bit versions of the tests directly, and adding
appropriate checks for -EOVERFLOW: when building on 32-bit-inode
filesystems, both ftw() and ftw64() get tested; on 64-bit ones,
only ftw64() gets tested and ftw() gets UNSUPPORTED, which is the
best we can do.  (The 64-bit ftwtest must be serialized with the
non-64-bit version since the test driver script uses the same
filesystem hierarchy in both cases.)

This at least allows a 32-bit libc with the source tree on a
64-bit-inode fs to pass make check for me. It may well not be the
minimum required for successful operations, though nothing else in
glibc's tools is obviously troublesome that I can see: among other
things, my *objdir* is still 32-bit... (There is one test that calls ftw
("/") which is unlikely to have an inode number > 2^32 on any current
filesystem, and one that calls it on a directory under /tmp which I
cannot easily test a fix for because my /tmp is a tmpfs with 32-bit
inodes. I can fix these two as well if you like, though, and make this
more of a maximal fix rather than a minimal one.)

There are probably still bits of #15333 left to fix, but this definitely
fixes a lot of them.  I haven't yet tested with objdir on a 64-bit-inode
fs, only srcdir: I'll try that next.

(Verified that everything is still fine with a 64-bit glibc on the same
64-bit-inode fs, too.)

v2: Use tz-cflags rather than modifying zic.c.
v3: Add bug-ftw*64.c, ftwtest-64.c; check for EOVERFLOW.
    The ChangeLog is horrifically repetitive now :( is doing it in this
    much detail really useful to anyone

	[BZ #15333]
	* dirent/list.c (list): Use dirent64/readdir64.
	* io/Makefile (test-srcs): Add ftwtest-64.
	[ifeq ($(run-built-tests),yes)] ($(objpfx)ftwtest-64.out):
	And here too, serialized with ftwtest.
	(tests): Add bug-ftw1-64, bug-ftw2-64, bug-ftw3-64,
	and bug-ftw5-64.
	* io/bug-ftw1.c [FTW]: New macro.
	[STAT]: Likewise.
	(callback): Use STAT.
	(main): Use FTW.  Check for EOVERFLOW.
	* io/bug-ftw1-64.c: New test, using above macros.
	* io/bug-ftw2.c [FTW]: New macro.
	[STAT]: Likewise.
	(callback): Use STAT.
	(main): Use FTW.  Check for EOVERFLOW.
	* io/bug-ftw2-64.c: New test, using above macros.
	* io/bug-ftw3.c [FTW]: New macro.
	[STAT]: Likewise.
	(cb): Use STAT.
	(main): Use FTW.  Check for EOVERFLOW.
	* io/bug-ftw3-64.c: New test, using above macros.
	* io/bug-ftw5.c [NFTW]: New macro.
	[STAT]: Likewise.
	(fn): Use STAT.
	(do_test): Use NFTW.  Check for EOVERFLOW.
	* io/bug-ftw5-64.c: New test, using above macros.
	* io/ftwtest.c [NFTW]: New macro.
	[STAT]: Likewise.
	[STR]: Likewise.
	(cb): Use STAT.
	(main): Use NFTW.  Adapt error message.
	* io/ftwtest-64.c: New test, using above macros.
	* posix/tst-regex.c (do_test): Use fstat64.
	* timezone/Makefile (tz-cflags): Set FILE_OFFSET_BITS.

Patch

diff --git a/dirent/list.c b/dirent/list.c
index e99b253808..f792084243 100644
--- a/dirent/list.c
+++ b/dirent/list.c
@@ -26,7 +26,7 @@  static int
 test (const char *name)
 {
   DIR *dirp;
-  struct dirent *entp;
+  struct dirent64 *entp;
   int retval = 0;
 
   puts (name);
@@ -39,7 +39,7 @@  test (const char *name)
     }
 
   errno = 0;
-  while ((entp = readdir (dirp)) != NULL)
+  while ((entp = readdir64 (dirp)) != NULL)
     printf ("%s\tfile number %lu\n",
 	    entp->d_name, (unsigned long int) entp->d_fileno);
 
diff --git a/io/Makefile b/io/Makefile
index 2f26bf56db..595707914a 100644
--- a/io/Makefile
+++ b/io/Makefile
@@ -62,18 +62,19 @@  static-only-routines = stat fstat lstat stat64 fstat64 lstat64	\
 		       fstatat fstatat64 mknod mknodat
 
 others		:= pwd
-test-srcs	:= ftwtest
+test-srcs	:= ftwtest ftwtest-64
 tests		:= test-utime test-stat test-stat2 test-lfs tst-getcwd \
-		   tst-fcntl bug-ftw1 bug-ftw2 bug-ftw3 bug-ftw4 tst-statvfs \
-		   tst-openat tst-unlinkat tst-fstatat tst-futimesat \
-		   tst-renameat tst-fchownat tst-fchmodat tst-faccessat \
-		   tst-symlinkat tst-linkat tst-readlinkat tst-mkdirat \
-		   tst-mknodat tst-mkfifoat tst-ttyname_r bug-ftw5 \
+		   tst-fcntl bug-ftw1 bug-ftw1-64 bug-ftw2 bug-ftw2-64 \
+		   bug-ftw3 bug-ftw3-64 bug-ftw4 bug-ftw5 bug-ftw5-64 \
+		   tst-statvfs tst-openat tst-unlinkat tst-fstatat \
+		   tst-futimesat tst-renameat tst-fchownat tst-fchmodat \
+		   tst-faccessat tst-symlinkat tst-linkat tst-readlinkat \
+		   tst-mkdirat tst-mknodat tst-mkfifoat tst-ttyname_r \
 		   tst-posix_fallocate tst-posix_fallocate64 \
 		   tst-fts tst-fts-lfs tst-open-tmpfile
 
 ifeq ($(run-built-tests),yes)
-tests-special += $(objpfx)ftwtest.out
+tests-special += $(objpfx)ftwtest.out $(objpfx)ftwtest-64.out
 endif
 
 include ../Rules
@@ -113,4 +114,9 @@  ifeq ($(run-built-tests),yes)
 $(objpfx)ftwtest.out: ftwtest-sh $(objpfx)ftwtest
 	$(SHELL) $< $(common-objpfx) '$(test-program-cmd)' > $@; \
 	$(evaluate-test)
+
+# Prevent ftwtest and ftwtest-64 runs from overlapping
+$(objpfx)ftwtest-64.out: ftwtest-sh $(objpfx)ftwtest-64 $(objpfx)ftwtest.out
+	$(SHELL) $< $(common-objpfx) '$(test-program-cmd)' > $@; \
+	$(evaluate-test)
 endif
diff --git a/io/bug-ftw1-64.c b/io/bug-ftw1-64.c
new file mode 100644
index 0000000000..3ec10190e3
--- /dev/null
+++ b/io/bug-ftw1-64.c
@@ -0,0 +1,3 @@ 
+#define FTW ftw64
+#define STAT stat64
+#include "bug-ftw1.c"
diff --git a/io/bug-ftw1.c b/io/bug-ftw1.c
index 29c3f7d531..f2466472f1 100644
--- a/io/bug-ftw1.c
+++ b/io/bug-ftw1.c
@@ -17,11 +17,19 @@ 
    License along with the GNU C Library; if not, see
    <http://www.gnu.org/licenses/>.  */
 
+#include <errno.h>
 #include <ftw.h>
 #include <mcheck.h>
 #include <stdio.h>
 #include <string.h>
 
+#ifndef FTW
+#define FTW ftw
+#endif
+
+#ifndef STAT
+#define STAT stat
+#endif
 
 int result;
 int cnt;
@@ -29,7 +37,7 @@  int sawroot;
 
 
 static int
-callback (const char *fname, const struct stat *st, int flag)
+callback (const char *fname, const struct STAT *st, int flag)
 {
   if (++cnt >= 10)
     return 1;
@@ -65,7 +73,10 @@  main (void)
 {
   mtrace ();
 
-  ftw ("/", callback, 10);
+  FTW ("/", callback, 10);
+
+  if (errno == EOVERFLOW)
+    return 77;
 
   if (! sawroot)
     {
diff --git a/io/bug-ftw2-64.c b/io/bug-ftw2-64.c
new file mode 100644
index 0000000000..9395f8f793
--- /dev/null
+++ b/io/bug-ftw2-64.c
@@ -0,0 +1,3 @@ 
+#define FTW ftw64
+#define STAT stat64
+#include "bug-ftw2.c"
diff --git a/io/bug-ftw2.c b/io/bug-ftw2.c
index ce8823d28e..6b0eabd7d3 100644
--- a/io/bug-ftw2.c
+++ b/io/bug-ftw2.c
@@ -17,11 +17,19 @@ 
    License along with the GNU C Library; if not, see
    <http://www.gnu.org/licenses/>.  */
 
+#include <errno.h>
 #include <ftw.h>
 #include <mcheck.h>
 #include <stdio.h>
 #include <string.h>
 
+#ifndef FTW
+#define FTW ftw
+#endif
+
+#ifndef STAT
+#define STAT stat
+#endif
 
 int cnt;
 int result;
@@ -30,7 +38,7 @@  int sawcur;
 
 
 static int
-callback (const char *fname, const struct stat *st, int flag)
+callback (const char *fname, const struct STAT *st, int flag)
 {
   printf ("%d: \"%s\" -> ", ++cnt, fname);
   if (strcmp (fname, ".") == 0 && sawcur)
@@ -69,7 +77,9 @@  main (void)
 {
   mtrace ();
 
-  ftw (".", callback, 10);
+  FTW (".", callback, 10);
+  if (errno == EOVERFLOW)
+    return 77;
 
   if (! sawcur)
     {
diff --git a/io/bug-ftw3-64.c b/io/bug-ftw3-64.c
new file mode 100644
index 0000000000..822684338e
--- /dev/null
+++ b/io/bug-ftw3-64.c
@@ -0,0 +1,3 @@ 
+#define FTW ftw64
+#define STAT stat64
+#include "bug-ftw3.c"
diff --git a/io/bug-ftw3.c b/io/bug-ftw3.c
index 19740f49f3..a64c242f19 100644
--- a/io/bug-ftw3.c
+++ b/io/bug-ftw3.c
@@ -5,10 +5,18 @@ 
 #include <string.h>
 #include <unistd.h>
 
+#ifndef FTW
+#define FTW ftw
+#endif
+
+#ifndef STAT
+#define STAT stat
+#endif
+
 static int cb_called;
 
 static int
-cb (const char *fname, const struct stat *st, int flag)
+cb (const char *fname, const struct STAT *st, int flag)
 {
   printf ("%s %d\n", fname, flag);
   cb_called = 1;
@@ -54,8 +62,10 @@  main (void)
       exit (1);
     }
 
-  r = ftw (dname2, cb, 10);
+  r = FTW (dname2, cb, 10);
   e = errno;
+  if (e == EOVERFLOW)
+    return 77;
   printf ("r = %d", r);
   if (r != 0)
     printf (", errno = %d", errno);
diff --git a/io/bug-ftw5-64.c b/io/bug-ftw5-64.c
new file mode 100644
index 0000000000..dd3a4501cb
--- /dev/null
+++ b/io/bug-ftw5-64.c
@@ -0,0 +1,3 @@ 
+#define NFTW nftw64
+#define STAT stat64
+#include "bug-ftw5.c"
diff --git a/io/bug-ftw5.c b/io/bug-ftw5.c
index c1cd81d30e..e4327f5e34 100644
--- a/io/bug-ftw5.c
+++ b/io/bug-ftw5.c
@@ -2,8 +2,16 @@ 
 #include <ftw.h>
 #include <stdio.h>
 
+#ifndef NFTW
+#define NFTW nftw
+#endif
+
+#ifndef STAT
+#define STAT stat
+#endif
+
 static int
-fn (const char *file, const struct stat *sb, int flag, struct FTW *s)
+fn (const char *file, const struct STAT *sb, int flag, struct FTW *s)
 {
   puts (file);
   return FTW_STOP;
@@ -12,8 +20,10 @@  fn (const char *file, const struct stat *sb, int flag, struct FTW *s)
 static int
 do_test (void)
 {
-  if (nftw ("/", fn, 0, FTW_CHDIR | FTW_ACTIONRETVAL) < 0)
+  if (NFTW ("/", fn, 0, FTW_CHDIR | FTW_ACTIONRETVAL) < 0)
     {
+      if (errno == EOVERFLOW)
+        return 77;
       printf ("nftw / FTW_CHDIR: %m\n");
       return 1;
     }
diff --git a/io/ftwtest-64.c b/io/ftwtest-64.c
new file mode 100644
index 0000000000..f5334b5256
--- /dev/null
+++ b/io/ftwtest-64.c
@@ -0,0 +1,3 @@ 
+#define NFTW nftw64
+#define STAT stat64
+#include "ftwtest.c"
diff --git a/io/ftwtest.c b/io/ftwtest.c
index 77debd2805..f0137a8f1a 100644
--- a/io/ftwtest.c
+++ b/io/ftwtest.c
@@ -7,6 +7,15 @@ 
 #include <unistd.h>
 #include <sys/stat.h>
 
+#ifndef NFTW
+#define NFTW nftw
+#endif
+
+#ifndef STAT
+#define STAT stat
+#endif
+
+#define STR(x) #x
 
 int do_depth;
 int do_chdir;
@@ -39,7 +48,7 @@  const char *flag2name[] =
 
 
 static int
-cb (const char *name, const struct stat *st, int flag, struct FTW *f)
+cb (const char *name, const struct STAT *st, int flag, struct FTW *f)
 {
   if (do_exit && strcmp (name + f->base, "file@2"))
     return FTW_CONTINUE;
@@ -97,9 +106,9 @@  main (int argc, char *argv[])
 
   char *cw1 = getcwd (NULL, 0);
 
-  r = nftw (optind < argc ? argv[optind] : ".", cb, do_exit ? 1 : 3, flag);
+  r = NFTW (optind < argc ? argv[optind] : ".", cb, do_exit ? 1 : 3, flag);
   if (r < 0)
-    perror ("nftw");
+    perror (STR(NFTW));
 
   char *cw2 = getcwd (NULL, 0);
 
diff --git a/posix/tst-regex.c b/posix/tst-regex.c
index df2c108be5..ab9a854d98 100644
--- a/posix/tst-regex.c
+++ b/posix/tst-regex.c
@@ -57,7 +57,7 @@  do_test (void)
 {
   const char *file;
   int fd;
-  struct stat st;
+  struct stat64 st;
   int result;
   char *inmem;
   char *outmem;
@@ -72,7 +72,7 @@  do_test (void)
   if (fd == -1)
     error (EXIT_FAILURE, errno, "cannot open %s", basename (file));
 
-  if (fstat (fd, &st) != 0)
+  if (fstat64 (fd, &st) != 0)
     error (EXIT_FAILURE, errno, "cannot stat %s", basename (file));
   memlen = st.st_size;
 
diff --git a/timezone/Makefile b/timezone/Makefile
index d6cc7ba357..c0b4ea8020 100644
--- a/timezone/Makefile
+++ b/timezone/Makefile
@@ -60,7 +60,8 @@  tz-cflags = -DTZDIR='"$(zonedir)"' \
 	    -DTZDEFRULES='"$(posixrules-file)"' \
 	    -DTM_GMTOFF=tm_gmtoff -DTM_ZONE=tm_zone \
 	    -DHAVE_GETTEXT -DUSE_LTZ=0 -D_ISOMAC -DTZ_DOMAIN='"libc"' \
-	    -include $(common-objpfx)config.h -Wno-maybe-uninitialized
+	    -include $(common-objpfx)config.h -Wno-maybe-uninitialized \
+	    -D_FILE_OFFSET_BITS=64
 
 # The -Wno-unused-variable flag is used to prevent GCC 6
 # from warning about time_t_min and time_t_max which are