From patchwork Mon Jun 26 22:16:35 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nix X-Patchwork-Id: 21270 Received: (qmail 106951 invoked by alias); 26 Jun 2017 22:16:45 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 106941 invoked by uid 89); 26 Jun 2017 22:16:44 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-25.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, KAM_LAZY_DOMAIN_SECURITY, UNPARSEABLE_RELAY autolearn=ham version=3.3.2 spammy= X-HELO: aserp1040.oracle.com From: Nick Alcock To: libc-alpha@sourceware.org Subject: [PATCH v3] zic, various tests: use 64-bit stat/ftw/readdir functions Date: Mon, 26 Jun 2017 23:16:35 +0100 Message-ID: <87lgoez96k.fsf@esperi.org.uk> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.91 (gnu/linux) MIME-Version: 1.0 From: Nick Alcock This fixes this compilation failure when testing a 32-bit glibc on a system using a 64-bit-inode XFS filesystem, and a variety of related test failures of dirent/list, io/bug-ftw2, and posix/tst-regex: env GCONV_PATH=/usr/src/glibc/i686-loom/iconvdata LOCPATH=/usr/src/glibc/i686-loom/localedata LC_ALL=C /usr/src/glibc/i686-loom/elf/ld-linux.so.2 --library-path /usr/src/glibc/i686-loom:/usr/src/glibc/i686-loom/math:/usr/src/glibc/i686-loom/elf:/usr/src/glibc/i686-loom/dlfcn:/usr/src/glibc/i686-loom/nss:/usr/src/glibc/i686-loom/nis:/usr/src/glibc/i686-loom/rt:/usr/src/glibc/i686-loom/resolv:/usr/src/glibc/i686-loom/crypt:/usr/src/glibc/i686-loom/mathvec:/usr/src/glibc/i686-loom/support:/usr/src/glibc/i686-loom/nptl /usr/src/glibc/i686-loom/timezone/zic -d /usr/src/glibc/i686-loom/timezone/testdata -y ./yearistype australasia; ../scripts/evaluate-test.sh timezone/testdata/Australia/Melbourne $? false false > /usr/src/glibc/i686-loom/timezone/testdata/Australia/Melbourne.test-result warning: "etcetera", line 13: /usr/src/glibc/i686-loom/timezone/zic: Can't create directory /usr/src: File exists /bin/sh: /usr/src/glibc/i686-loom/timezone/testdata/Etc/UTC.test-result: No such file or directory make[2]: *** [Makefile:98: /usr/src/glibc/i686-loom/timezone/testdata/Etc/UTC] Error 1 This happens because itsdir() in timezone/zic.c does a stat() of each element of the path in turn, and this returns -EOVERFLOW because on this system /usr has an inode number of 7516193792 and we did not compile zic with -D_FILE_OFFSET_BITS=64 or use stat64(). Do the former in tz-cflags, to avoid unnecessarily skewing zic from its upstream. We can fix the other test failures via use of the *64() functions and by adding 64-bit versions of the tests directly, and adding appropriate checks for -EOVERFLOW: when building on 32-bit-inode filesystems, both ftw() and ftw64() get tested; on 64-bit ones, only ftw64() gets tested and ftw() gets UNSUPPORTED, which is the best we can do. (The 64-bit ftwtest must be serialized with the non-64-bit version since the test driver script uses the same filesystem hierarchy in both cases.) This at least allows a 32-bit libc with the source tree on a 64-bit-inode fs to pass make check for me. It may well not be the minimum required for successful operations, though nothing else in glibc's tools is obviously troublesome that I can see: among other things, my *objdir* is still 32-bit... (There is one test that calls ftw ("/") which is unlikely to have an inode number > 2^32 on any current filesystem, and one that calls it on a directory under /tmp which I cannot easily test a fix for because my /tmp is a tmpfs with 32-bit inodes. I can fix these two as well if you like, though, and make this more of a maximal fix rather than a minimal one.) There are probably still bits of #15333 left to fix, but this definitely fixes a lot of them. I haven't yet tested with objdir on a 64-bit-inode fs, only srcdir: I'll try that next. (Verified that everything is still fine with a 64-bit glibc on the same 64-bit-inode fs, too.) v2: Use tz-cflags rather than modifying zic.c. v3: Add bug-ftw*64.c, ftwtest-64.c; check for EOVERFLOW. The ChangeLog is horrifically repetitive now :( is doing it in this much detail really useful to anyone [BZ #15333] * dirent/list.c (list): Use dirent64/readdir64. * io/Makefile (test-srcs): Add ftwtest-64. [ifeq ($(run-built-tests),yes)] ($(objpfx)ftwtest-64.out): And here too, serialized with ftwtest. (tests): Add bug-ftw1-64, bug-ftw2-64, bug-ftw3-64, and bug-ftw5-64. * io/bug-ftw1.c [FTW]: New macro. [STAT]: Likewise. (callback): Use STAT. (main): Use FTW. Check for EOVERFLOW. * io/bug-ftw1-64.c: New test, using above macros. * io/bug-ftw2.c [FTW]: New macro. [STAT]: Likewise. (callback): Use STAT. (main): Use FTW. Check for EOVERFLOW. * io/bug-ftw2-64.c: New test, using above macros. * io/bug-ftw3.c [FTW]: New macro. [STAT]: Likewise. (cb): Use STAT. (main): Use FTW. Check for EOVERFLOW. * io/bug-ftw3-64.c: New test, using above macros. * io/bug-ftw5.c [NFTW]: New macro. [STAT]: Likewise. (fn): Use STAT. (do_test): Use NFTW. Check for EOVERFLOW. * io/bug-ftw5-64.c: New test, using above macros. * io/ftwtest.c [NFTW]: New macro. [STAT]: Likewise. [STR]: Likewise. (cb): Use STAT. (main): Use NFTW. Adapt error message. * io/ftwtest-64.c: New test, using above macros. * posix/tst-regex.c (do_test): Use fstat64. * timezone/Makefile (tz-cflags): Set FILE_OFFSET_BITS. diff --git a/dirent/list.c b/dirent/list.c index e99b253808..f792084243 100644 --- a/dirent/list.c +++ b/dirent/list.c @@ -26,7 +26,7 @@ static int test (const char *name) { DIR *dirp; - struct dirent *entp; + struct dirent64 *entp; int retval = 0; puts (name); @@ -39,7 +39,7 @@ test (const char *name) } errno = 0; - while ((entp = readdir (dirp)) != NULL) + while ((entp = readdir64 (dirp)) != NULL) printf ("%s\tfile number %lu\n", entp->d_name, (unsigned long int) entp->d_fileno); diff --git a/io/Makefile b/io/Makefile index 2f26bf56db..595707914a 100644 --- a/io/Makefile +++ b/io/Makefile @@ -62,18 +62,19 @@ static-only-routines = stat fstat lstat stat64 fstat64 lstat64 \ fstatat fstatat64 mknod mknodat others := pwd -test-srcs := ftwtest +test-srcs := ftwtest ftwtest-64 tests := test-utime test-stat test-stat2 test-lfs tst-getcwd \ - tst-fcntl bug-ftw1 bug-ftw2 bug-ftw3 bug-ftw4 tst-statvfs \ - tst-openat tst-unlinkat tst-fstatat tst-futimesat \ - tst-renameat tst-fchownat tst-fchmodat tst-faccessat \ - tst-symlinkat tst-linkat tst-readlinkat tst-mkdirat \ - tst-mknodat tst-mkfifoat tst-ttyname_r bug-ftw5 \ + tst-fcntl bug-ftw1 bug-ftw1-64 bug-ftw2 bug-ftw2-64 \ + bug-ftw3 bug-ftw3-64 bug-ftw4 bug-ftw5 bug-ftw5-64 \ + tst-statvfs tst-openat tst-unlinkat tst-fstatat \ + tst-futimesat tst-renameat tst-fchownat tst-fchmodat \ + tst-faccessat tst-symlinkat tst-linkat tst-readlinkat \ + tst-mkdirat tst-mknodat tst-mkfifoat tst-ttyname_r \ tst-posix_fallocate tst-posix_fallocate64 \ tst-fts tst-fts-lfs tst-open-tmpfile ifeq ($(run-built-tests),yes) -tests-special += $(objpfx)ftwtest.out +tests-special += $(objpfx)ftwtest.out $(objpfx)ftwtest-64.out endif include ../Rules @@ -113,4 +114,9 @@ ifeq ($(run-built-tests),yes) $(objpfx)ftwtest.out: ftwtest-sh $(objpfx)ftwtest $(SHELL) $< $(common-objpfx) '$(test-program-cmd)' > $@; \ $(evaluate-test) + +# Prevent ftwtest and ftwtest-64 runs from overlapping +$(objpfx)ftwtest-64.out: ftwtest-sh $(objpfx)ftwtest-64 $(objpfx)ftwtest.out + $(SHELL) $< $(common-objpfx) '$(test-program-cmd)' > $@; \ + $(evaluate-test) endif diff --git a/io/bug-ftw1-64.c b/io/bug-ftw1-64.c new file mode 100644 index 0000000000..3ec10190e3 --- /dev/null +++ b/io/bug-ftw1-64.c @@ -0,0 +1,3 @@ +#define FTW ftw64 +#define STAT stat64 +#include "bug-ftw1.c" diff --git a/io/bug-ftw1.c b/io/bug-ftw1.c index 29c3f7d531..f2466472f1 100644 --- a/io/bug-ftw1.c +++ b/io/bug-ftw1.c @@ -17,11 +17,19 @@ License along with the GNU C Library; if not, see . */ +#include #include #include #include #include +#ifndef FTW +#define FTW ftw +#endif + +#ifndef STAT +#define STAT stat +#endif int result; int cnt; @@ -29,7 +37,7 @@ int sawroot; static int -callback (const char *fname, const struct stat *st, int flag) +callback (const char *fname, const struct STAT *st, int flag) { if (++cnt >= 10) return 1; @@ -65,7 +73,10 @@ main (void) { mtrace (); - ftw ("/", callback, 10); + FTW ("/", callback, 10); + + if (errno == EOVERFLOW) + return 77; if (! sawroot) { diff --git a/io/bug-ftw2-64.c b/io/bug-ftw2-64.c new file mode 100644 index 0000000000..9395f8f793 --- /dev/null +++ b/io/bug-ftw2-64.c @@ -0,0 +1,3 @@ +#define FTW ftw64 +#define STAT stat64 +#include "bug-ftw2.c" diff --git a/io/bug-ftw2.c b/io/bug-ftw2.c index ce8823d28e..6b0eabd7d3 100644 --- a/io/bug-ftw2.c +++ b/io/bug-ftw2.c @@ -17,11 +17,19 @@ License along with the GNU C Library; if not, see . */ +#include #include #include #include #include +#ifndef FTW +#define FTW ftw +#endif + +#ifndef STAT +#define STAT stat +#endif int cnt; int result; @@ -30,7 +38,7 @@ int sawcur; static int -callback (const char *fname, const struct stat *st, int flag) +callback (const char *fname, const struct STAT *st, int flag) { printf ("%d: \"%s\" -> ", ++cnt, fname); if (strcmp (fname, ".") == 0 && sawcur) @@ -69,7 +77,9 @@ main (void) { mtrace (); - ftw (".", callback, 10); + FTW (".", callback, 10); + if (errno == EOVERFLOW) + return 77; if (! sawcur) { diff --git a/io/bug-ftw3-64.c b/io/bug-ftw3-64.c new file mode 100644 index 0000000000..822684338e --- /dev/null +++ b/io/bug-ftw3-64.c @@ -0,0 +1,3 @@ +#define FTW ftw64 +#define STAT stat64 +#include "bug-ftw3.c" diff --git a/io/bug-ftw3.c b/io/bug-ftw3.c index 19740f49f3..a64c242f19 100644 --- a/io/bug-ftw3.c +++ b/io/bug-ftw3.c @@ -5,10 +5,18 @@ #include #include +#ifndef FTW +#define FTW ftw +#endif + +#ifndef STAT +#define STAT stat +#endif + static int cb_called; static int -cb (const char *fname, const struct stat *st, int flag) +cb (const char *fname, const struct STAT *st, int flag) { printf ("%s %d\n", fname, flag); cb_called = 1; @@ -54,8 +62,10 @@ main (void) exit (1); } - r = ftw (dname2, cb, 10); + r = FTW (dname2, cb, 10); e = errno; + if (e == EOVERFLOW) + return 77; printf ("r = %d", r); if (r != 0) printf (", errno = %d", errno); diff --git a/io/bug-ftw5-64.c b/io/bug-ftw5-64.c new file mode 100644 index 0000000000..dd3a4501cb --- /dev/null +++ b/io/bug-ftw5-64.c @@ -0,0 +1,3 @@ +#define NFTW nftw64 +#define STAT stat64 +#include "bug-ftw5.c" diff --git a/io/bug-ftw5.c b/io/bug-ftw5.c index c1cd81d30e..e4327f5e34 100644 --- a/io/bug-ftw5.c +++ b/io/bug-ftw5.c @@ -2,8 +2,16 @@ #include #include +#ifndef NFTW +#define NFTW nftw +#endif + +#ifndef STAT +#define STAT stat +#endif + static int -fn (const char *file, const struct stat *sb, int flag, struct FTW *s) +fn (const char *file, const struct STAT *sb, int flag, struct FTW *s) { puts (file); return FTW_STOP; @@ -12,8 +20,10 @@ fn (const char *file, const struct stat *sb, int flag, struct FTW *s) static int do_test (void) { - if (nftw ("/", fn, 0, FTW_CHDIR | FTW_ACTIONRETVAL) < 0) + if (NFTW ("/", fn, 0, FTW_CHDIR | FTW_ACTIONRETVAL) < 0) { + if (errno == EOVERFLOW) + return 77; printf ("nftw / FTW_CHDIR: %m\n"); return 1; } diff --git a/io/ftwtest-64.c b/io/ftwtest-64.c new file mode 100644 index 0000000000..f5334b5256 --- /dev/null +++ b/io/ftwtest-64.c @@ -0,0 +1,3 @@ +#define NFTW nftw64 +#define STAT stat64 +#include "ftwtest.c" diff --git a/io/ftwtest.c b/io/ftwtest.c index 77debd2805..f0137a8f1a 100644 --- a/io/ftwtest.c +++ b/io/ftwtest.c @@ -7,6 +7,15 @@ #include #include +#ifndef NFTW +#define NFTW nftw +#endif + +#ifndef STAT +#define STAT stat +#endif + +#define STR(x) #x int do_depth; int do_chdir; @@ -39,7 +48,7 @@ const char *flag2name[] = static int -cb (const char *name, const struct stat *st, int flag, struct FTW *f) +cb (const char *name, const struct STAT *st, int flag, struct FTW *f) { if (do_exit && strcmp (name + f->base, "file@2")) return FTW_CONTINUE; @@ -97,9 +106,9 @@ main (int argc, char *argv[]) char *cw1 = getcwd (NULL, 0); - r = nftw (optind < argc ? argv[optind] : ".", cb, do_exit ? 1 : 3, flag); + r = NFTW (optind < argc ? argv[optind] : ".", cb, do_exit ? 1 : 3, flag); if (r < 0) - perror ("nftw"); + perror (STR(NFTW)); char *cw2 = getcwd (NULL, 0); diff --git a/posix/tst-regex.c b/posix/tst-regex.c index df2c108be5..ab9a854d98 100644 --- a/posix/tst-regex.c +++ b/posix/tst-regex.c @@ -57,7 +57,7 @@ do_test (void) { const char *file; int fd; - struct stat st; + struct stat64 st; int result; char *inmem; char *outmem; @@ -72,7 +72,7 @@ do_test (void) if (fd == -1) error (EXIT_FAILURE, errno, "cannot open %s", basename (file)); - if (fstat (fd, &st) != 0) + if (fstat64 (fd, &st) != 0) error (EXIT_FAILURE, errno, "cannot stat %s", basename (file)); memlen = st.st_size; diff --git a/timezone/Makefile b/timezone/Makefile index d6cc7ba357..c0b4ea8020 100644 --- a/timezone/Makefile +++ b/timezone/Makefile @@ -60,7 +60,8 @@ tz-cflags = -DTZDIR='"$(zonedir)"' \ -DTZDEFRULES='"$(posixrules-file)"' \ -DTM_GMTOFF=tm_gmtoff -DTM_ZONE=tm_zone \ -DHAVE_GETTEXT -DUSE_LTZ=0 -D_ISOMAC -DTZ_DOMAIN='"libc"' \ - -include $(common-objpfx)config.h -Wno-maybe-uninitialized + -include $(common-objpfx)config.h -Wno-maybe-uninitialized \ + -D_FILE_OFFSET_BITS=64 # The -Wno-unused-variable flag is used to prevent GCC 6 # from warning about time_t_min and time_t_max which are