From patchwork Fri Oct 2 17:06:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 40646 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 39253398E45E; Fri, 2 Oct 2020 17:06:34 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 39253398E45E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1601658394; bh=AoIm5KChl2fKwGfrTC2cnC1T5C0uQxAqBVjmJz64DJw=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=Id519w3qRWxrxBOWQnJVyhlKCGbR8YkmQgZTmRkjVFi7ZceBt3YUDN4r6Q1WztByy issDgrQQSz50HZ69ilPd8vl5ES6INNcNurd3HdntBvWUABxrKZTpGyqN+PWczgshb6 QJMotblBLkdFgqqOmDVckYf0VZlb/zExZhSHuiEo= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) by sourceware.org (Postfix) with ESMTPS id A038B38708C5 for ; Fri, 2 Oct 2020 17:06:29 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org A038B38708C5 Received: by mail-qt1-x841.google.com with SMTP id o21so1839825qtp.2 for ; Fri, 02 Oct 2020 10:06:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AoIm5KChl2fKwGfrTC2cnC1T5C0uQxAqBVjmJz64DJw=; b=UZQQ0SiM37ptXXBnf40YMvrnaaWb8+dvvZFcwYBRpVh4pmHJTtH+7dapGEvud5+TtR SPyB0U0IfULrkrt4mWxbxicPi4tEG2+wGj6gEh9cX23MXNlF9xFazQvyH4IshNl2kzWR Dw0H6XTh35grV3XQUpdOseYzM4VJseNS1wfR3uDgYmVYIzDm3494WiY2PIPYBtAluHOV WRWIRDS3GvjEWOg6KDADKFcMk4KTV2pyRbXLDpcI4ZPMTiQzbDEqmga0kkqanpT0bDCC 7Ucl8RdyNmYLChRuTds4JVRxjON+qogS4TOSTjYAwTZoIrES+zs5De2CeyYxSD/X0JDD 6xlg== X-Gm-Message-State: AOAM530OKzU7SBMzKNYLK+jDvpoqnIQ3GiP+YlKYk3TtEvTphdDs3kYp 5XWXsVvHwEOcQSYYp+SGQrH/Q3tkL/Mm3A== X-Google-Smtp-Source: ABdhPJzzo1WHMoPslBUptETVyDEhl85pxs6nWSonbVXzaEcoKGx89TMgKS5crVXfpu+wRyORYO4Gwg== X-Received: by 2002:ac8:4295:: with SMTP id o21mr3455846qtl.241.1601658388454; Fri, 02 Oct 2020 10:06:28 -0700 (PDT) Received: from localhost.localdomain ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id d10sm1436891qkk.1.2020.10.02.10.06.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Oct 2020 10:06:27 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v2 1/9] linux: Move posix dir implementations to Linux Date: Fri, 2 Oct 2020 14:06:12 -0300 Message-Id: <20201002170620.1611673-2-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201002170620.1611673-1-adhemerval.zanella@linaro.org> References: <20201002170620.1611673-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-13.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Cc: James Clarke , John Paul Adrian Glaubitz Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" This generic implementation already expects a getdents API which is Linux specific. It also allows simplify it by assuming _DIRENT_HAVE_D_RECLEN and _DIRENT_HAVE_D_OFF support. The readdir are also expanded on each required implementation, futher fixes and improvements will make parametrize the implementation more complex. Checked on x86_64-linux-gnu, i686-linux-gnu, and with a build for all affected ABIs. --- sysdeps/posix/readdir.c | 127 ------------ sysdeps/posix/readdir_r.c | 159 -------------- sysdeps/{posix => unix/sysv/linux}/closedir.c | 0 sysdeps/{posix => unix/sysv/linux}/dirfd.c | 0 .../{posix => unix/sysv/linux}/dirstream.h | 0 .../{posix => unix/sysv/linux}/fdopendir.c | 0 sysdeps/{posix => unix/sysv/linux}/opendir.c | 0 sysdeps/unix/sysv/linux/readdir.c | 65 +++++- sysdeps/unix/sysv/linux/readdir64.c | 131 +++++++++++- sysdeps/unix/sysv/linux/readdir64_r.c | 194 +++++++++++++++++- sysdeps/unix/sysv/linux/readdir_r.c | 95 ++++++++- .../{posix => unix/sysv/linux}/rewinddir.c | 0 sysdeps/{posix => unix/sysv/linux}/seekdir.c | 0 sysdeps/{posix => unix/sysv/linux}/telldir.c | 0 14 files changed, 462 insertions(+), 309 deletions(-) delete mode 100644 sysdeps/posix/readdir.c delete mode 100644 sysdeps/posix/readdir_r.c rename sysdeps/{posix => unix/sysv/linux}/closedir.c (100%) rename sysdeps/{posix => unix/sysv/linux}/dirfd.c (100%) rename sysdeps/{posix => unix/sysv/linux}/dirstream.h (100%) rename sysdeps/{posix => unix/sysv/linux}/fdopendir.c (100%) rename sysdeps/{posix => unix/sysv/linux}/opendir.c (100%) rename sysdeps/{posix => unix/sysv/linux}/rewinddir.c (100%) rename sysdeps/{posix => unix/sysv/linux}/seekdir.c (100%) rename sysdeps/{posix => unix/sysv/linux}/telldir.c (100%) diff --git a/sysdeps/posix/readdir.c b/sysdeps/posix/readdir.c deleted file mode 100644 index b36278b5f4..0000000000 --- a/sysdeps/posix/readdir.c +++ /dev/null @@ -1,127 +0,0 @@ -/* Copyright (C) 1991-2020 Free Software Foundation, Inc. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - . */ - -#include -#include -#include -#include -#include -#include -#include -#include - -#include - -#ifndef __READDIR -# define __READDIR __readdir -# define __GETDENTS __getdents -# define DIRENT_TYPE struct dirent -# define __READDIR_ALIAS -#endif - -/* Read a directory entry from DIRP. */ -DIRENT_TYPE * -__READDIR (DIR *dirp) -{ - DIRENT_TYPE *dp; - int saved_errno = errno; - -#if IS_IN (libc) - __libc_lock_lock (dirp->lock); -#endif - - do - { - size_t reclen; - - if (dirp->offset >= dirp->size) - { - /* We've emptied out our buffer. Refill it. */ - - size_t maxread; - ssize_t bytes; - -#ifndef _DIRENT_HAVE_D_RECLEN - /* Fixed-size struct; must read one at a time (see below). */ - maxread = sizeof *dp; -#else - maxread = dirp->allocation; -#endif - - bytes = __GETDENTS (dirp->fd, dirp->data, maxread); - if (bytes <= 0) - { - /* On some systems getdents fails with ENOENT when the - open directory has been rmdir'd already. POSIX.1 - requires that we treat this condition like normal EOF. */ - if (bytes < 0 && errno == ENOENT) - bytes = 0; - - /* Don't modifiy errno when reaching EOF. */ - if (bytes == 0) - __set_errno (saved_errno); - dp = NULL; - break; - } - dirp->size = (size_t) bytes; - - /* Reset the offset into the buffer. */ - dirp->offset = 0; - } - - dp = (DIRENT_TYPE *) &dirp->data[dirp->offset]; - -#ifdef _DIRENT_HAVE_D_RECLEN - reclen = dp->d_reclen; -#else - /* The only version of `struct dirent*' that lacks `d_reclen' - is fixed-size. */ - assert (sizeof dp->d_name > 1); - reclen = sizeof *dp; - /* The name is not terminated if it is the largest possible size. - Clobber the following byte to ensure proper null termination. We - read jst one entry at a time above so we know that byte will not - be used later. */ - dp->d_name[sizeof dp->d_name] = '\0'; -#endif - - dirp->offset += reclen; - -#ifdef _DIRENT_HAVE_D_OFF - dirp->filepos = dp->d_off; -#else - dirp->filepos += reclen; -#endif - - /* Skip deleted files. */ - } while (dp->d_ino == 0); - -#if IS_IN (libc) - __libc_lock_unlock (dirp->lock); -#endif - - return dp; -} - -#ifdef __READDIR_ALIAS -weak_alias (__readdir, readdir) -#endif - -#undef __READDIR -#undef __GETDENTS -#undef DIRENT_TYPE -#undef __READDIR_ALIAS diff --git a/sysdeps/posix/readdir_r.c b/sysdeps/posix/readdir_r.c deleted file mode 100644 index 9079abc2ff..0000000000 --- a/sysdeps/posix/readdir_r.c +++ /dev/null @@ -1,159 +0,0 @@ -/* Copyright (C) 1991-2020 Free Software Foundation, Inc. - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - . */ - -#include -#include -#include -#include -#include -#include -#include -#include - -#include - -#ifndef __READDIR_R -# define __READDIR_R __readdir_r -# define __GETDENTS __getdents -# define DIRENT_TYPE struct dirent -# define __READDIR_R_ALIAS -#endif - -/* Read a directory entry from DIRP. */ -int -__READDIR_R (DIR *dirp, DIRENT_TYPE *entry, DIRENT_TYPE **result) -{ - DIRENT_TYPE *dp; - size_t reclen; - const int saved_errno = errno; - int ret; - - __libc_lock_lock (dirp->lock); - - do - { - if (dirp->offset >= dirp->size) - { - /* We've emptied out our buffer. Refill it. */ - - size_t maxread; - ssize_t bytes; - -#ifndef _DIRENT_HAVE_D_RECLEN - /* Fixed-size struct; must read one at a time (see below). */ - maxread = sizeof *dp; -#else - maxread = dirp->allocation; -#endif - - bytes = __GETDENTS (dirp->fd, dirp->data, maxread); - if (bytes <= 0) - { - /* On some systems getdents fails with ENOENT when the - open directory has been rmdir'd already. POSIX.1 - requires that we treat this condition like normal EOF. */ - if (bytes < 0 && errno == ENOENT) - { - bytes = 0; - __set_errno (saved_errno); - } - if (bytes < 0) - dirp->errcode = errno; - - dp = NULL; - break; - } - dirp->size = (size_t) bytes; - - /* Reset the offset into the buffer. */ - dirp->offset = 0; - } - - dp = (DIRENT_TYPE *) &dirp->data[dirp->offset]; - -#ifdef _DIRENT_HAVE_D_RECLEN - reclen = dp->d_reclen; -#else - /* The only version of `struct dirent*' that lacks `d_reclen' - is fixed-size. */ - assert (sizeof dp->d_name > 1); - reclen = sizeof *dp; - /* The name is not terminated if it is the largest possible size. - Clobber the following byte to ensure proper null termination. We - read just one entry at a time above so we know that byte will not - be used later. */ - dp->d_name[sizeof dp->d_name] = '\0'; -#endif - - dirp->offset += reclen; - -#ifdef _DIRENT_HAVE_D_OFF - dirp->filepos = dp->d_off; -#else - dirp->filepos += reclen; -#endif - -#ifdef NAME_MAX - if (reclen > offsetof (DIRENT_TYPE, d_name) + NAME_MAX + 1) - { - /* The record is very long. It could still fit into the - caller-supplied buffer if we can skip padding at the - end. */ - size_t namelen = _D_EXACT_NAMLEN (dp); - if (namelen <= NAME_MAX) - reclen = offsetof (DIRENT_TYPE, d_name) + namelen + 1; - else - { - /* The name is too long. Ignore this file. */ - dirp->errcode = ENAMETOOLONG; - dp->d_ino = 0; - continue; - } - } -#endif - - /* Skip deleted and ignored files. */ - } - while (dp->d_ino == 0); - - if (dp != NULL) - { - *result = memcpy (entry, dp, reclen); -#ifdef _DIRENT_HAVE_D_RECLEN - entry->d_reclen = reclen; -#endif - ret = 0; - } - else - { - *result = NULL; - ret = dirp->errcode; - } - - __libc_lock_unlock (dirp->lock); - - return ret; -} - -#ifdef __READDIR_R_ALIAS -weak_alias (__readdir_r, readdir_r) -#endif - -#undef __READDIR_R -#undef __GETDENTS -#undef DIRENT_TYPE -#undef __READDIR_R_ALIAS diff --git a/sysdeps/posix/closedir.c b/sysdeps/unix/sysv/linux/closedir.c similarity index 100% rename from sysdeps/posix/closedir.c rename to sysdeps/unix/sysv/linux/closedir.c diff --git a/sysdeps/posix/dirfd.c b/sysdeps/unix/sysv/linux/dirfd.c similarity index 100% rename from sysdeps/posix/dirfd.c rename to sysdeps/unix/sysv/linux/dirfd.c diff --git a/sysdeps/posix/dirstream.h b/sysdeps/unix/sysv/linux/dirstream.h similarity index 100% rename from sysdeps/posix/dirstream.h rename to sysdeps/unix/sysv/linux/dirstream.h diff --git a/sysdeps/posix/fdopendir.c b/sysdeps/unix/sysv/linux/fdopendir.c similarity index 100% rename from sysdeps/posix/fdopendir.c rename to sysdeps/unix/sysv/linux/fdopendir.c diff --git a/sysdeps/posix/opendir.c b/sysdeps/unix/sysv/linux/opendir.c similarity index 100% rename from sysdeps/posix/opendir.c rename to sysdeps/unix/sysv/linux/opendir.c diff --git a/sysdeps/unix/sysv/linux/readdir.c b/sysdeps/unix/sysv/linux/readdir.c index df7a92aa78..2e03e66e69 100644 --- a/sysdeps/unix/sysv/linux/readdir.c +++ b/sysdeps/unix/sysv/linux/readdir.c @@ -19,5 +19,68 @@ #include #if !_DIRENT_MATCHES_DIRENT64 -# include +#include + +/* Read a directory entry from DIRP. */ +struct dirent * +__readdir (DIR *dirp) +{ + struct dirent *dp; + int saved_errno = errno; + +#if IS_IN (libc) + __libc_lock_lock (dirp->lock); +#endif + + do + { + size_t reclen; + + if (dirp->offset >= dirp->size) + { + /* We've emptied out our buffer. Refill it. */ + + size_t maxread = dirp->allocation; + ssize_t bytes; + + bytes = __getdents (dirp->fd, dirp->data, maxread); + if (bytes <= 0) + { + /* On some systems getdents fails with ENOENT when the + open directory has been rmdir'd already. POSIX.1 + requires that we treat this condition like normal EOF. */ + if (bytes < 0 && errno == ENOENT) + bytes = 0; + + /* Don't modifiy errno when reaching EOF. */ + if (bytes == 0) + __set_errno (saved_errno); + dp = NULL; + break; + } + dirp->size = (size_t) bytes; + + /* Reset the offset into the buffer. */ + dirp->offset = 0; + } + + dp = (struct dirent *) &dirp->data[dirp->offset]; + + reclen = dp->d_reclen; + + dirp->offset += reclen; + + dirp->filepos = dp->d_off; + + /* Skip deleted files. */ + } while (dp->d_ino == 0); + +#if IS_IN (libc) + __libc_lock_unlock (dirp->lock); +#endif + + return dp; +} +weak_alias (__readdir, readdir) + #endif diff --git a/sysdeps/unix/sysv/linux/readdir64.c b/sysdeps/unix/sysv/linux/readdir64.c index 170a889c51..1aa6e2664f 100644 --- a/sysdeps/unix/sysv/linux/readdir64.c +++ b/sysdeps/unix/sysv/linux/readdir64.c @@ -23,17 +23,71 @@ #define readdir __no_readdir_decl #define __readdir __no___readdir_decl #include +#undef __readdir +#undef readdir -#define __READDIR __readdir64 -#define __GETDENTS __getdents64 -#define DIRENT_TYPE struct dirent64 +/* Read a directory entry from DIRP. */ +struct dirent64 * +__readdir64 (DIR *dirp) +{ + struct dirent64 *dp; + int saved_errno = errno; -#include +#if IS_IN (libc) + __libc_lock_lock (dirp->lock); +#endif -#undef __readdir -#undef readdir + do + { + size_t reclen; + + if (dirp->offset >= dirp->size) + { + /* We've emptied out our buffer. Refill it. */ + + size_t maxread = dirp->allocation; + ssize_t bytes; + + bytes = __getdents64 (dirp->fd, dirp->data, maxread); + if (bytes <= 0) + { + /* On some systems getdents fails with ENOENT when the + open directory has been rmdir'd already. POSIX.1 + requires that we treat this condition like normal EOF. */ + if (bytes < 0 && errno == ENOENT) + bytes = 0; + + /* Don't modifiy errno when reaching EOF. */ + if (bytes == 0) + __set_errno (saved_errno); + dp = NULL; + break; + } + dirp->size = (size_t) bytes; + /* Reset the offset into the buffer. */ + dirp->offset = 0; + } + + dp = (struct dirent64 *) &dirp->data[dirp->offset]; + + reclen = dp->d_reclen; + + dirp->offset += reclen; + + dirp->filepos = dp->d_off; + + /* Skip deleted files. */ + } while (dp->d_ino == 0); + +#if IS_IN (libc) + __libc_lock_unlock (dirp->lock); +#endif + + return dp; +} libc_hidden_def (__readdir64) + #if _DIRENT_MATCHES_DIRENT64 strong_alias (__readdir64, __readdir) weak_alias (__readdir64, readdir64) @@ -49,10 +103,67 @@ versioned_symbol (libc, __readdir64, readdir64, GLIBC_2_2); # endif # if SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2) # include -# define __READDIR attribute_compat_text_section __old_readdir64 -# define __GETDENTS __old_getdents64 -# define DIRENT_TYPE struct __old_dirent64 -# include + +attribute_compat_text_section +struct __old_dirent64 * +__old_readdir64 (DIR *dirp) +{ + struct __old_dirent64 *dp; + int saved_errno = errno; + +#if IS_IN (libc) + __libc_lock_lock (dirp->lock); +#endif + + do + { + size_t reclen; + + if (dirp->offset >= dirp->size) + { + /* We've emptied out our buffer. Refill it. */ + + size_t maxread = dirp->allocation; + ssize_t bytes; + + bytes = __old_getdents64 (dirp->fd, dirp->data, maxread); + if (bytes <= 0) + { + /* On some systems getdents fails with ENOENT when the + open directory has been rmdir'd already. POSIX.1 + requires that we treat this condition like normal EOF. */ + if (bytes < 0 && errno == ENOENT) + bytes = 0; + + /* Don't modifiy errno when reaching EOF. */ + if (bytes == 0) + __set_errno (saved_errno); + dp = NULL; + break; + } + dirp->size = (size_t) bytes; + + /* Reset the offset into the buffer. */ + dirp->offset = 0; + } + + dp = (struct __old_dirent64 *) &dirp->data[dirp->offset]; + + reclen = dp->d_reclen; + + dirp->offset += reclen; + + dirp->filepos = dp->d_off; + + /* Skip deleted files. */ + } while (dp->d_ino == 0); + +#if IS_IN (libc) + __libc_lock_unlock (dirp->lock); +#endif + + return dp; +} libc_hidden_def (__old_readdir64) compat_symbol (libc, __old_readdir64, readdir64, GLIBC_2_1); # endif /* SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2) */ diff --git a/sysdeps/unix/sysv/linux/readdir64_r.c b/sysdeps/unix/sysv/linux/readdir64_r.c index 6d589f36f5..c587787417 100644 --- a/sysdeps/unix/sysv/linux/readdir64_r.c +++ b/sysdeps/unix/sysv/linux/readdir64_r.c @@ -23,15 +23,100 @@ #define readdir_r __no_readdir_r_decl #define __readdir_r __no___readdir_r_decl #include +#undef __readdir_r +#undef readdir_r -#define __READDIR_R __readdir64_r -#define __GETDENTS __getdents64 -#define DIRENT_TYPE struct dirent64 +/* Read a directory entry from DIRP. */ +int +__readdir64_r (DIR *dirp, struct dirent64 *entry, struct dirent64 **result) +{ + struct dirent64 *dp; + size_t reclen; + const int saved_errno = errno; + int ret; -#include + __libc_lock_lock (dirp->lock); + + do + { + if (dirp->offset >= dirp->size) + { + /* We've emptied out our buffer. Refill it. */ + + size_t maxread = dirp->allocation; + ssize_t bytes; + + maxread = dirp->allocation; + + bytes = __getdents64 (dirp->fd, dirp->data, maxread); + if (bytes <= 0) + { + /* On some systems getdents fails with ENOENT when the + open directory has been rmdir'd already. POSIX.1 + requires that we treat this condition like normal EOF. */ + if (bytes < 0 && errno == ENOENT) + { + bytes = 0; + __set_errno (saved_errno); + } + if (bytes < 0) + dirp->errcode = errno; + + dp = NULL; + break; + } + dirp->size = (size_t) bytes; + + /* Reset the offset into the buffer. */ + dirp->offset = 0; + } + + dp = (struct dirent64 *) &dirp->data[dirp->offset]; + + reclen = dp->d_reclen; + + dirp->offset += reclen; + + dirp->filepos = dp->d_off; + + if (reclen > offsetof (struct dirent64, d_name) + NAME_MAX + 1) + { + /* The record is very long. It could still fit into the + caller-supplied buffer if we can skip padding at the + end. */ + size_t namelen = _D_EXACT_NAMLEN (dp); + if (namelen <= NAME_MAX) + reclen = offsetof (struct dirent64, d_name) + namelen + 1; + else + { + /* The name is too long. Ignore this file. */ + dirp->errcode = ENAMETOOLONG; + dp->d_ino = 0; + continue; + } + } + + /* Skip deleted and ignored files. */ + } + while (dp->d_ino == 0); + + if (dp != NULL) + { + *result = memcpy (entry, dp, reclen); + entry->d_reclen = reclen; + ret = 0; + } + else + { + *result = NULL; + ret = dirp->errcode; + } + + __libc_lock_unlock (dirp->lock); + + return ret; +} -#undef __readdir_r -#undef readdir_r #if _DIRENT_MATCHES_DIRENT64 strong_alias (__readdir64_r, __readdir_r) @@ -44,10 +129,99 @@ weak_alias (__readdir64_r, readdir64_r) versioned_symbol (libc, __readdir64_r, readdir64_r, GLIBC_2_2); # if SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2) # include -# define __READDIR_R attribute_compat_text_section __old_readdir64_r -# define __GETDENTS __old_getdents64 -# define DIRENT_TYPE struct __old_dirent64 -# include + +int +attribute_compat_text_section +__old_readdir64_r (DIR *dirp, struct __old_dirent64 *entry, + struct __old_dirent64 **result) +{ + struct __old_dirent64 *dp; + size_t reclen; + const int saved_errno = errno; + int ret; + + __libc_lock_lock (dirp->lock); + + do + { + if (dirp->offset >= dirp->size) + { + /* We've emptied out our buffer. Refill it. */ + + size_t maxread = dirp->allocation; + ssize_t bytes; + + maxread = dirp->allocation; + + bytes = __old_getdents64 (dirp->fd, dirp->data, maxread); + if (bytes <= 0) + { + /* On some systems getdents fails with ENOENT when the + open directory has been rmdir'd already. POSIX.1 + requires that we treat this condition like normal EOF. */ + if (bytes < 0 && errno == ENOENT) + { + bytes = 0; + __set_errno (saved_errno); + } + if (bytes < 0) + dirp->errcode = errno; + + dp = NULL; + break; + } + dirp->size = (size_t) bytes; + + /* Reset the offset into the buffer. */ + dirp->offset = 0; + } + + dp = (struct __old_dirent64 *) &dirp->data[dirp->offset]; + + reclen = dp->d_reclen; + + dirp->offset += reclen; + + dirp->filepos = dp->d_off; + + if (reclen > offsetof (struct __old_dirent64, d_name) + NAME_MAX + 1) + { + /* The record is very long. It could still fit into the + caller-supplied buffer if we can skip padding at the + end. */ + size_t namelen = _D_EXACT_NAMLEN (dp); + if (namelen <= NAME_MAX) + reclen = offsetof (struct __old_dirent64, d_name) + namelen + 1; + else + { + /* The name is too long. Ignore this file. */ + dirp->errcode = ENAMETOOLONG; + dp->d_ino = 0; + continue; + } + } + + /* Skip deleted and ignored files. */ + } + while (dp->d_ino == 0); + + if (dp != NULL) + { + *result = memcpy (entry, dp, reclen); + entry->d_reclen = reclen; + ret = 0; + } + else + { + *result = NULL; + ret = dirp->errcode; + } + + __libc_lock_unlock (dirp->lock); + + return ret; +} + compat_symbol (libc, __old_readdir64_r, readdir64_r, GLIBC_2_1); # endif /* SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2) */ #endif /* _DIRENT_MATCHES_DIRENT64 */ diff --git a/sysdeps/unix/sysv/linux/readdir_r.c b/sysdeps/unix/sysv/linux/readdir_r.c index 30f237dbcc..0069041394 100644 --- a/sysdeps/unix/sysv/linux/readdir_r.c +++ b/sysdeps/unix/sysv/linux/readdir_r.c @@ -19,5 +19,96 @@ #include #if !_DIRENT_MATCHES_DIRENT64 -# include -#endif +/* Read a directory entry from DIRP. */ +int +__readdir_r (DIR *dirp, struct dirent *entry, struct dirent **result) +{ + struct dirent *dp; + size_t reclen; + const int saved_errno = errno; + int ret; + + __libc_lock_lock (dirp->lock); + + do + { + if (dirp->offset >= dirp->size) + { + /* We've emptied out our buffer. Refill it. */ + + size_t maxread = dirp->allocation; + ssize_t bytes; + + maxread = dirp->allocation; + + bytes = __getdents (dirp->fd, dirp->data, maxread); + if (bytes <= 0) + { + /* On some systems getdents fails with ENOENT when the + open directory has been rmdir'd already. POSIX.1 + requires that we treat this condition like normal EOF. */ + if (bytes < 0 && errno == ENOENT) + { + bytes = 0; + __set_errno (saved_errno); + } + if (bytes < 0) + dirp->errcode = errno; + + dp = NULL; + break; + } + dirp->size = (size_t) bytes; + + /* Reset the offset into the buffer. */ + dirp->offset = 0; + } + + dp = (struct dirent *) &dirp->data[dirp->offset]; + + reclen = dp->d_reclen; + + dirp->offset += reclen; + + dirp->filepos = dp->d_off; + + if (reclen > offsetof (struct dirent, d_name) + NAME_MAX + 1) + { + /* The record is very long. It could still fit into the + caller-supplied buffer if we can skip padding at the + end. */ + size_t namelen = _D_EXACT_NAMLEN (dp); + if (namelen <= NAME_MAX) + reclen = offsetof (struct dirent, d_name) + namelen + 1; + else + { + /* The name is too long. Ignore this file. */ + dirp->errcode = ENAMETOOLONG; + dp->d_ino = 0; + continue; + } + } + + /* Skip deleted and ignored files. */ + } + while (dp->d_ino == 0); + + if (dp != NULL) + { + *result = memcpy (entry, dp, reclen); + entry->d_reclen = reclen; + ret = 0; + } + else + { + *result = NULL; + ret = dirp->errcode; + } + + __libc_lock_unlock (dirp->lock); + + return ret; +} + +weak_alias (__readdir_r, readdir_r) +#endif /* _DIRENT_MATCHES_DIRENT64 */ diff --git a/sysdeps/posix/rewinddir.c b/sysdeps/unix/sysv/linux/rewinddir.c similarity index 100% rename from sysdeps/posix/rewinddir.c rename to sysdeps/unix/sysv/linux/rewinddir.c diff --git a/sysdeps/posix/seekdir.c b/sysdeps/unix/sysv/linux/seekdir.c similarity index 100% rename from sysdeps/posix/seekdir.c rename to sysdeps/unix/sysv/linux/seekdir.c diff --git a/sysdeps/posix/telldir.c b/sysdeps/unix/sysv/linux/telldir.c similarity index 100% rename from sysdeps/posix/telldir.c rename to sysdeps/unix/sysv/linux/telldir.c From patchwork Fri Oct 2 17:06:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 40645 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 8C8AC38708C5; Fri, 2 Oct 2020 17:06:33 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 8C8AC38708C5 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1601658393; bh=Hrn6bHXjH3zOReF1k+X5PgyJxMKj1lxBtrh8OdZKdoM=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ycL2rTy7A4RFY3OQPUlkPB7Ihx5+DPao0u+Xozl3dN/IuEFpW03Bua+pvx+85nxcL 9SBCZq9oakbSFQxb2ymQ2R6J+5kpMEFUzVxn7Dd0M1LZ1B7SeL1I3rLNG2B0ZFRGyT Ox2d1qT8ZqJ+C3CT1/+NVMG61OFW/zWHPXsp0TlA= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qk1-x741.google.com (mail-qk1-x741.google.com [IPv6:2607:f8b0:4864:20::741]) by sourceware.org (Postfix) with ESMTPS id 36298398546A for ; Fri, 2 Oct 2020 17:06:31 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 36298398546A Received: by mail-qk1-x741.google.com with SMTP id f142so2066424qke.13 for ; Fri, 02 Oct 2020 10:06:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Hrn6bHXjH3zOReF1k+X5PgyJxMKj1lxBtrh8OdZKdoM=; b=MtArRvgVyZjhOqTTyxe1r249KGqOuSh1dGsYEvMIXXnu2U7w0Z1dZvnlCqtjaDQv40 nxpz+DuUm+vxHjq4EQv30Bx0dT9GVlyCUnfgE88BlKq4/YFS63gnnWlU8YeGg5/h2keW eOOywL9PsLd7kRhuHY+chjnxlLmIWLnmxGD6Wd4b8WKS9AqXgEHZ0TRJ4gHXi3iwyBnB XpZ0cX8LzAdl4zTO7Vcastog1+6EYcq8hsao4ftD4v0w2UMmNn8IpDXi23kOcPC3Qpq+ LSCyyNJ1GXrJHsxLEm6QARXarE+iZZdbjr4VhCqMMQyR/OdresKDOyu7DX0dci8tAMMZ jDSQ== X-Gm-Message-State: AOAM533rMbtVWcnhXW+9r1kjESj6z/SMPiemwN2ubJ+vY0dCgG8eS/GR 7Fnum7wk7tUFaPojrgjDN5B4Pe3V5IlSIg== X-Google-Smtp-Source: ABdhPJz3w3jdF/oWLMxnkXJt77F4D4f1UazCMT4KSIh0XvyU/0jZJ+9iLWhqF2tAY15HMCRtKEMEPA== X-Received: by 2002:a37:a187:: with SMTP id k129mr3025588qke.147.1601658390568; Fri, 02 Oct 2020 10:06:30 -0700 (PDT) Received: from localhost.localdomain ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id d10sm1436891qkk.1.2020.10.02.10.06.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Oct 2020 10:06:30 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v2 2/9] linux: Simplify opendir buffer allocation Date: Fri, 2 Oct 2020 14:06:13 -0300 Message-Id: <20201002170620.1611673-3-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201002170620.1611673-1-adhemerval.zanella@linaro.org> References: <20201002170620.1611673-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-13.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Cc: James Clarke , John Paul Adrian Glaubitz Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" The fallback allocation is removed, so the possible size constraint should be analyzed just once; __alloc_dir assumes that 'statp' argument is non-null, and the max_buffer_size move to close its used. Checked on x86_64-linux-gnu and i686-linux-gnu. --- include/dirent.h | 3 +- sysdeps/unix/sysv/linux/opendir.c | 51 +++++++++++-------------------- 2 files changed, 20 insertions(+), 34 deletions(-) diff --git a/include/dirent.h b/include/dirent.h index 2b1cdcf8bd..fdf4c4a2f1 100644 --- a/include/dirent.h +++ b/include/dirent.h @@ -48,7 +48,8 @@ extern int __versionsort64 (const struct dirent64 **a, const struct dirent64 **b) __attribute_pure__; extern DIR *__alloc_dir (int fd, bool close_fd, int flags, - const struct stat64 *statp) attribute_hidden; + const struct stat64 *statp) + __nonnull (4) attribute_hidden; extern __typeof (rewinddir) __rewinddir; extern __typeof (seekdir) __seekdir; extern __typeof (dirfd) __dirfd; diff --git a/sysdeps/unix/sysv/linux/opendir.c b/sysdeps/unix/sysv/linux/opendir.c index e89e09bfc7..2198224588 100644 --- a/sysdeps/unix/sysv/linux/opendir.c +++ b/sysdeps/unix/sysv/linux/opendir.c @@ -23,12 +23,6 @@ #include -/* The st_blksize value of the directory is used as a hint for the - size of the buffer which receives struct dirent values from the - kernel. st_blksize is limited to MAX_DIR_BUFFER_SIZE, in case the - file system provides a bogus value. */ -#define MAX_DIR_BUFFER_SIZE 1048576U - enum { opendir_oflags = O_RDONLY|O_NDELAY|O_DIRECTORY|O_LARGEFILE|O_CLOEXEC }; @@ -100,38 +94,29 @@ __alloc_dir (int fd, bool close_fd, int flags, const struct stat64 *statp) file descriptor. */ if (!close_fd && __glibc_unlikely (__fcntl64_nocancel (fd, F_SETFD, FD_CLOEXEC) < 0)) - goto lose; - - const size_t default_allocation = (4 * BUFSIZ < sizeof (struct dirent64) - ? sizeof (struct dirent64) : 4 * BUFSIZ); - const size_t small_allocation = (BUFSIZ < sizeof (struct dirent64) - ? sizeof (struct dirent64) : BUFSIZ); - size_t allocation = default_allocation; -#ifdef _STATBUF_ST_BLKSIZE + return NULL; + + /* The st_blksize value of the directory is used as a hint for the + size of the buffer which receives struct dirent values from the + kernel. st_blksize is limited to max_buffer_size, in case the + file system provides a bogus value. */ + enum { max_buffer_size = 1048576 }; + + const size_t allocation_size = 32768; + _Static_assert (allocation_size >= sizeof (struct dirent64), + "allocation_size < sizeof (struct dirent64)"); + /* Increase allocation if requested, but not if the value appears to - be bogus. */ - if (statp != NULL) - allocation = MIN (MAX ((size_t) statp->st_blksize, default_allocation), - MAX_DIR_BUFFER_SIZE); -#endif + be bogus. It will be between 32Kb and 1Mb. */ + size_t allocation = MIN (MAX ((size_t) statp->st_blksize, allocation_size), + max_buffer_size); DIR *dirp = (DIR *) malloc (sizeof (DIR) + allocation); if (dirp == NULL) { - allocation = small_allocation; - dirp = (DIR *) malloc (sizeof (DIR) + allocation); - - if (dirp == NULL) - lose: - { - if (close_fd) - { - int save_errno = errno; - __close_nocancel_nostatus (fd); - __set_errno (save_errno); - } - return NULL; - } + if (close_fd) + __close_nocancel_nostatus (fd); + return NULL; } dirp->fd = fd; From patchwork Fri Oct 2 17:06:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 40647 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 4D17C398E44A; Fri, 2 Oct 2020 17:06:36 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 4D17C398E44A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1601658396; bh=i5w1Ncxkbq1PvhPT78w61pRht1KfXCXubEwqp4OtlvY=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=wMe2YeUxZ6UU5D1jtiTwWai6rbxMTwOLQn+/eIDy1kfdFZa7NLwIrFT8K52Q7tMa4 pGAYPDecoLOyASwVYFcZhQy2avpKUleQgAv6HCwsQdd6SOrsyFGhPLwV5eA/tueCmU Of/Zr2rpP/jR7t+vCY+yf/pLIa44+WuGbkDJ/598= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by sourceware.org (Postfix) with ESMTPS id B758C398E451 for ; Fri, 2 Oct 2020 17:06:33 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org B758C398E451 Received: by mail-qt1-x842.google.com with SMTP id m23so1843205qtq.4 for ; Fri, 02 Oct 2020 10:06:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=i5w1Ncxkbq1PvhPT78w61pRht1KfXCXubEwqp4OtlvY=; b=At4Eff1FSgb6ttkBBBtNl1ucswZmulFP7/LQki7cujP1n3xf2kT2uTfTgVOHC1K7nE J7hjs8WTwRaWaoRB6RI1ikbpTn8567ZE0PtXUose0uIy076RgHx3ZrJ4X8RM0FJ0y2gn ImkwUw9ddi8LtFZxNiYB5fBctfsPaBGUx/ux1nlzwGeSH6/csC7EbMd8yhEaqo3MB710 noBVbZHKM2bHMZcjDpYa8J0fIARRT6lc8ndF1AiQUL+n7rrMsMwy4kOmzLwxVGA7Tr0S Jgy4j/strUjvFwB6V7T6SRUxejWdFdABkhM7hIfmz3Cc4TynhX7/ME6Lh++LpxAw+Ooy v4jA== X-Gm-Message-State: AOAM533V0Z/80fCqVEenYh05HgxffZsXTbh7Mwh0NG6kfq+K5stJ6nK2 vGLm3V8psnP83IjYB/1kZ+yjCDhlfWPHmQ== X-Google-Smtp-Source: ABdhPJwBe/vwM07TpTYJHlzG1W7D1hbwdFxzDFNvsbe0qNNHp3ZKcC2anzbILAT70nbnaYhOy/kyiw== X-Received: by 2002:ac8:4e0b:: with SMTP id c11mr3308256qtw.37.1601658392775; Fri, 02 Oct 2020 10:06:32 -0700 (PDT) Received: from localhost.localdomain ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id d10sm1436891qkk.1.2020.10.02.10.06.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Oct 2020 10:06:32 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v2 3/9] linux: Add __readdir_unlocked Date: Fri, 2 Oct 2020 14:06:14 -0300 Message-Id: <20201002170620.1611673-4-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201002170620.1611673-1-adhemerval.zanella@linaro.org> References: <20201002170620.1611673-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-13.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Cc: James Clarke , John Paul Adrian Glaubitz Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" And use it on readdir_r implementation. Checked on i686-linux-gnu. --- include/dirent.h | 1 + sysdeps/unix/sysv/linux/readdir.c | 18 +++++-- sysdeps/unix/sysv/linux/readdir_r.c | 79 +++++++---------------------- 3 files changed, 31 insertions(+), 67 deletions(-) diff --git a/include/dirent.h b/include/dirent.h index fdf4c4a2f1..8325a19e5f 100644 --- a/include/dirent.h +++ b/include/dirent.h @@ -20,6 +20,7 @@ extern DIR *__opendirat (int dfd, const char *__name) attribute_hidden; extern DIR *__fdopendir (int __fd) attribute_hidden; extern int __closedir (DIR *__dirp) attribute_hidden; extern struct dirent *__readdir (DIR *__dirp) attribute_hidden; +extern struct dirent *__readdir_unlocked (DIR *__dirp) attribute_hidden; extern struct dirent64 *__readdir64 (DIR *__dirp); libc_hidden_proto (__readdir64) extern int __readdir_r (DIR *__dirp, struct dirent *__entry, diff --git a/sysdeps/unix/sysv/linux/readdir.c b/sysdeps/unix/sysv/linux/readdir.c index 2e03e66e69..ca2a8964e9 100644 --- a/sysdeps/unix/sysv/linux/readdir.c +++ b/sysdeps/unix/sysv/linux/readdir.c @@ -23,15 +23,11 @@ /* Read a directory entry from DIRP. */ struct dirent * -__readdir (DIR *dirp) +__readdir_unlocked (DIR *dirp) { struct dirent *dp; int saved_errno = errno; -#if IS_IN (libc) - __libc_lock_lock (dirp->lock); -#endif - do { size_t reclen; @@ -75,6 +71,18 @@ __readdir (DIR *dirp) /* Skip deleted files. */ } while (dp->d_ino == 0); + return dp; +} + +struct dirent * +__readdir (DIR *dirp) +{ + struct dirent *dp; + +#if IS_IN (libc) + __libc_lock_lock (dirp->lock); +#endif + dp = __readdir_unlocked (dirp); #if IS_IN (libc) __libc_lock_unlock (dirp->lock); #endif diff --git a/sysdeps/unix/sysv/linux/readdir_r.c b/sysdeps/unix/sysv/linux/readdir_r.c index 0069041394..a01d2845a6 100644 --- a/sysdeps/unix/sysv/linux/readdir_r.c +++ b/sysdeps/unix/sysv/linux/readdir_r.c @@ -25,89 +25,44 @@ __readdir_r (DIR *dirp, struct dirent *entry, struct dirent **result) { struct dirent *dp; size_t reclen; - const int saved_errno = errno; - int ret; __libc_lock_lock (dirp->lock); - do + while (1) { - if (dirp->offset >= dirp->size) - { - /* We've emptied out our buffer. Refill it. */ - - size_t maxread = dirp->allocation; - ssize_t bytes; - - maxread = dirp->allocation; - - bytes = __getdents (dirp->fd, dirp->data, maxread); - if (bytes <= 0) - { - /* On some systems getdents fails with ENOENT when the - open directory has been rmdir'd already. POSIX.1 - requires that we treat this condition like normal EOF. */ - if (bytes < 0 && errno == ENOENT) - { - bytes = 0; - __set_errno (saved_errno); - } - if (bytes < 0) - dirp->errcode = errno; - - dp = NULL; - break; - } - dirp->size = (size_t) bytes; - - /* Reset the offset into the buffer. */ - dirp->offset = 0; - } - - dp = (struct dirent *) &dirp->data[dirp->offset]; + dp = __readdir_unlocked (dirp); + if (dp == NULL) + break; reclen = dp->d_reclen; + if (reclen <= offsetof (struct dirent, d_name) + NAME_MAX + 1) + break; - dirp->offset += reclen; - - dirp->filepos = dp->d_off; - - if (reclen > offsetof (struct dirent, d_name) + NAME_MAX + 1) + /* The record is very long. It could still fit into the caller-supplied + buffer if we can skip padding at the end. */ + size_t namelen = _D_EXACT_NAMLEN (dp); + if (namelen <= NAME_MAX) { - /* The record is very long. It could still fit into the - caller-supplied buffer if we can skip padding at the - end. */ - size_t namelen = _D_EXACT_NAMLEN (dp); - if (namelen <= NAME_MAX) - reclen = offsetof (struct dirent, d_name) + namelen + 1; - else - { - /* The name is too long. Ignore this file. */ - dirp->errcode = ENAMETOOLONG; - dp->d_ino = 0; - continue; - } + reclen = offsetof (struct dirent, d_name) + namelen + 1; + break; } - /* Skip deleted and ignored files. */ + /* The name is too long. Ignore this file. */ + dirp->errcode = ENAMETOOLONG; + dp->d_ino = 0; } - while (dp->d_ino == 0); if (dp != NULL) { *result = memcpy (entry, dp, reclen); entry->d_reclen = reclen; - ret = 0; } else - { - *result = NULL; - ret = dirp->errcode; - } + *result = NULL; __libc_lock_unlock (dirp->lock); - return ret; + return dp != NULL ? 0 : dirp->errcode; } weak_alias (__readdir_r, readdir_r) From patchwork Fri Oct 2 17:06:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 40648 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 532E8398E467; Fri, 2 Oct 2020 17:06:39 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 532E8398E467 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1601658399; bh=+afOXFMgBIVbJ7bAdwvbWBkbhttPtUAm5ToxE5uIXXM=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=hFcEVM45jb3u9vr2tno9om/sLwWLRhp0pIlEQRQcbhcNyp93tXqBvn1lAgeLrVcnJ 2jlfoW0zBtvgovW/XrrA/8Js+LDr8toxbzIuSxaSGwWCM3Cy4Wc8Qyh5Ot92deJwvG 1sNafS7i7gMc1j1kHhOyw/aiIuZKTMsfeT5HGmEY= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) by sourceware.org (Postfix) with ESMTPS id A8834398E460 for ; Fri, 2 Oct 2020 17:06:35 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org A8834398E460 Received: by mail-qt1-x841.google.com with SMTP id d1so1827974qtr.6 for ; Fri, 02 Oct 2020 10:06:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+afOXFMgBIVbJ7bAdwvbWBkbhttPtUAm5ToxE5uIXXM=; b=m5sUeVJE8wp2r/yCUay1eVJfgLFfdbn4tRTv6ygQ+Y5j/x2MqEtDSUJMswdIbHofN6 P4kab+OZGZDckuAGtC/jkJOwYtsHRPl7bNNHkoFPdQm7vNYIua0bt89i+ONaz4Tfm5h3 rkuUhBHOEP52ScMWUEKv2ZXarXPG2uUuPwaj4jzUizQ6BjOThnylZNTQIS+WRyATSfpO vhHB7RkJwOkc3iQi0Xk/qgLkodqPOJblKghPF/SxhXromo+F87Dxw5R1CgjGBzjV1B4S gVkbmBZsvciZ+zIMFahdK2wRqLf5XHkQy36XS2363RZkT87JbQOSjct3SaqUnTlvOC+r 4VPw== X-Gm-Message-State: AOAM532c/Z5CKKVgeUd8gyxmA0D9GADvge/oj7lt/sovAhIzhLKLq1v2 xQBu8ozoxEqQNRMQb30bM0kODCv+eiN5XA== X-Google-Smtp-Source: ABdhPJyOSeT8pmm12auPzqcTvLq2k7xh6Nas7x3my1nPOU8zmf7rVg+s2qh6UZ3bnXvFIhh/eehyMg== X-Received: by 2002:ac8:44aa:: with SMTP id a10mr3440826qto.116.1601658394862; Fri, 02 Oct 2020 10:06:34 -0700 (PDT) Received: from localhost.localdomain ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id d10sm1436891qkk.1.2020.10.02.10.06.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Oct 2020 10:06:34 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v2 4/9] linux: Use getdents64 on non-LFS readdir Date: Fri, 2 Oct 2020 14:06:15 -0300 Message-Id: <20201002170620.1611673-5-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201002170620.1611673-1-adhemerval.zanella@linaro.org> References: <20201002170620.1611673-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-13.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Cc: James Clarke , John Paul Adrian Glaubitz Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" It reserves some space on the allocated internal buffer to be used as a the returned dirent struct. The kernel obtained dirent64 struct are copied to the temporary buffer on each readdir call. The overflow test is moved once the dirent64 entry is copied to the temporary buffer, and a subsequent readdir will obtain the next entry. The idea is an overflow fails to return the entry on readdir, but a next readdir might still obtain the next entry. (for filesystem that does not have the concept of sequential d_off, such as ext4). Checked on x86_64-linux-gnu and i686-linux-gnu. --- sysdeps/unix/sysv/linux/opendir.c | 6 +- sysdeps/unix/sysv/linux/readdir.c | 25 +++---- sysdeps/unix/sysv/linux/readdir.h | 117 ++++++++++++++++++++++++++++++ 3 files changed, 131 insertions(+), 17 deletions(-) create mode 100644 sysdeps/unix/sysv/linux/readdir.h diff --git a/sysdeps/unix/sysv/linux/opendir.c b/sysdeps/unix/sysv/linux/opendir.c index 2198224588..24bd63d2ba 100644 --- a/sysdeps/unix/sysv/linux/opendir.c +++ b/sysdeps/unix/sysv/linux/opendir.c @@ -22,6 +22,7 @@ #include /* For MIN and MAX. */ #include +#include /* For return_buffer_size. */ enum { opendir_oflags = O_RDONLY|O_NDELAY|O_DIRECTORY|O_LARGEFILE|O_CLOEXEC @@ -103,8 +104,9 @@ __alloc_dir (int fd, bool close_fd, int flags, const struct stat64 *statp) enum { max_buffer_size = 1048576 }; const size_t allocation_size = 32768; - _Static_assert (allocation_size >= sizeof (struct dirent64), - "allocation_size < sizeof (struct dirent64)"); + _Static_assert (allocation_size >= sizeof (struct dirent64) + + return_buffer_size, + "opendir buffer size smaller than required"); /* Increase allocation if requested, but not if the value appears to be bogus. It will be between 32Kb and 1Mb. */ diff --git a/sysdeps/unix/sysv/linux/readdir.c b/sysdeps/unix/sysv/linux/readdir.c index ca2a8964e9..8eab0f4c9b 100644 --- a/sysdeps/unix/sysv/linux/readdir.c +++ b/sysdeps/unix/sysv/linux/readdir.c @@ -19,7 +19,7 @@ #include #if !_DIRENT_MATCHES_DIRENT64 -#include +#include /* Read a directory entry from DIRP. */ struct dirent * @@ -30,16 +30,12 @@ __readdir_unlocked (DIR *dirp) do { - size_t reclen; - if (dirp->offset >= dirp->size) { /* We've emptied out our buffer. Refill it. */ - size_t maxread = dirp->allocation; - ssize_t bytes; - - bytes = __getdents (dirp->fd, dirp->data, maxread); + ssize_t bytes = __getdents64 (dirp->fd, dirstream_data (dirp), + dirstream_alloc_size (dirp)); if (bytes <= 0) { /* On some systems getdents fails with ENOENT when the @@ -54,19 +50,18 @@ __readdir_unlocked (DIR *dirp) dp = NULL; break; } - dirp->size = (size_t) bytes; + dirp->size = bytes; /* Reset the offset into the buffer. */ dirp->offset = 0; } - dp = (struct dirent *) &dirp->data[dirp->offset]; - - reclen = dp->d_reclen; - - dirp->offset += reclen; - - dirp->filepos = dp->d_off; + dp = dirstream_ret_entry (dirp); + if (dp == NULL) + { + __set_errno (EOVERFLOW); + break; + } /* Skip deleted files. */ } while (dp->d_ino == 0); diff --git a/sysdeps/unix/sysv/linux/readdir.h b/sysdeps/unix/sysv/linux/readdir.h new file mode 100644 index 0000000000..4dc219e220 --- /dev/null +++ b/sysdeps/unix/sysv/linux/readdir.h @@ -0,0 +1,117 @@ +/* Linux readdir internal implementation details. + Copyright (C) 2020 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef _DIRSTREAM_NOLFS_H +#define _DIRSTREAM_NOLFS_H + +#if !_DIRENT_MATCHES_DIRENT64 +# include + +/* getdents64 is used internally for both LFS and non-LFS implementations. + The non-LFS interface reserves part of the allocated buffer to return the + non-LFS 'struct dirent' entry. */ + +/* This defines the reserved space size on DIR internal buffer to use as the + returned 'struct dirent' from a 'readdir' call. + + The largest possible practical length of the d_name member are 255 + Unicode characters in UTF-8 encoding, so d_name is 766 bytes long, plus + 10 bytes from header, for a total of 776 bytes total. + + Also it should take in cosideration the alignment requirement for + getdents64 call. */ +enum { return_buffer_size = 1024 + + sizeof (off64_t) + - _Alignof (((struct __dirstream) {0}).data) }; + +_Static_assert ((_Alignof (((struct __dirstream) {0}).data) + + return_buffer_size) % sizeof (off64_t) == 0, + "return_buffer_size does not align the buffer properly"); + +/* Return the avaliable buffer size to use with getdents64 calls. */ +static inline size_t +dirstream_alloc_size (struct __dirstream *ds) +{ + return ds->allocation - return_buffer_size; +} + +/* Return the start of the allocated buffer minus the reserved part to use on + non-LFS readdir call. */ +static inline void * +dirstream_data (struct __dirstream *ds) +{ + return (char *) ds->data + return_buffer_size; +} + +/* Return the allocated buffer used on non-LFS readdir call. */ +static inline struct dirent * +dirstream_ret (struct __dirstream *ds) +{ + return (struct dirent *) ds->data; +} + +/* Return the current dirent64 entry from the reserved buffer used on + getdent64. */ +static inline struct dirent64 * +dirstream_entry (struct __dirstream *ds) +{ + size_t offset = return_buffer_size + ds->offset; + return (struct dirent64 *) ((char *) ds->data + offset); +} + +/* Copy one obtained entry from 'getdents64' call to the reserved space + on DS allocated buffer and updated its internal state. */ +static inline struct dirent * +dirstream_ret_entry (struct __dirstream *ds) +{ + struct dirent64 *dp64 = dirstream_entry (ds); + struct dirent *dp = dirstream_ret (ds); + + dp->d_ino = dp64->d_ino; + + dp->d_off = dp64->d_off; + if (dp->d_off != dp64->d_off) + /* Overflow. */ + return NULL; + + const size_t size_diff = (offsetof (struct dirent64, d_name) + - offsetof (struct dirent, d_name)); + const size_t alignment = _Alignof (struct dirent); + size_t new_reclen = (dp64->d_reclen - size_diff + alignment - 1) + & ~(alignment - 1); + if (new_reclen > return_buffer_size) + /* Overflow. */ + return NULL; + dp->d_reclen = new_reclen; + + dp->d_type = dp64->d_type; + + memcpy (dp->d_name, dp64->d_name, + dp64->d_reclen - offsetof (struct dirent64, d_name)); + + ds->offset += dp64->d_reclen; + ds->filepos = dp64->d_off; + + return dp; +} +#else +/* No need to reserve an buffer space if dirent has already LFS support. */ +enum { return_buffer_size = 0 }; +#endif /* _DIRENT_MATCHES_DIRENT64 */ + +#endif From patchwork Fri Oct 2 17:06:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 40649 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id DC030398E46C; Fri, 2 Oct 2020 17:06:41 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org DC030398E46C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1601658401; bh=KJbogMR45qZ0sCQsDhW49AI3nQ8McQCLmDL6FhJ3trs=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=KkW/KXL9zAlNgBEUfl1mjfq9pZbfLn/KSqtqt+U+6QgKwsOW/gCr3QyqGYjfmAQGA aKNL0iVKXJyi/ciO0+23QmELvZ9KVrvKILK7igZvsa3NFbcGvnl8NAzlHKh2iKt/1Y ZGV+2c7vpp45HqIeWXJBxS3TqAlfQdNKi10LKHwg= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qt1-x844.google.com (mail-qt1-x844.google.com [IPv6:2607:f8b0:4864:20::844]) by sourceware.org (Postfix) with ESMTPS id F0DFB398E460 for ; Fri, 2 Oct 2020 17:06:37 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org F0DFB398E460 Received: by mail-qt1-x844.google.com with SMTP id m23so1843386qtq.4 for ; Fri, 02 Oct 2020 10:06:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KJbogMR45qZ0sCQsDhW49AI3nQ8McQCLmDL6FhJ3trs=; b=jQlsZcrrYaldthn2aGUeBcxupMI1LClODEe/l1xRFLh/Th34IbxwedJUxFLSgsDHlj 7uawHBUpSCCmasVZ5O022Tf6oinCZWg04L1G1Bo1v/uHbW1v7KoqFJbzeSNZpTXruIP2 ZtKev4M3lk+jaIao2CLpznoE36lPUOqEm2u531NrX+MXAKJslXg4BzE7/e+jq3pdB1yl wdkUOyQ7YmHfh8+HLhOZkKGo5+81W6E6xE48UGDAykjYieB29iaSEPdTqF0xYmWclbG9 dt8Tda4hRHANs73IvZAuaVYB5OqrebcFpE9tj0FCxXYFpXiKQ+TUnR0xY0RbPv21lcrV 9aUg== X-Gm-Message-State: AOAM5308frY5RGBEWT/KwxmVrTb6XqaXh4SV1i+BODyqTgG8OKVzgofS umE4eBIBXJwy/IA5I1/vBWYqXsBos4DBlg== X-Google-Smtp-Source: ABdhPJyYzElkM/lSabsE7N7VfrD9x3GViPrhiIxRejPUOIB4cYqjq7wQlBuW91EDrTT521SuE+1e3g== X-Received: by 2002:ac8:6f49:: with SMTP id n9mr3367101qtv.143.1601658396920; Fri, 02 Oct 2020 10:06:36 -0700 (PDT) Received: from localhost.localdomain ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id d10sm1436891qkk.1.2020.10.02.10.06.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Oct 2020 10:06:36 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v2 5/9] linux: Set internal DIR filepos as off64_t [BZ #23960, BZ #24050] Date: Fri, 2 Oct 2020 14:06:16 -0300 Message-Id: <20201002170620.1611673-6-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201002170620.1611673-1-adhemerval.zanella@linaro.org> References: <20201002170620.1611673-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-13.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Cc: James Clarke , John Paul Adrian Glaubitz Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" It allows to obtain the expected entry offset on telldir and set it correctly on seekdir on platforms where long int is smaller than off64_t. On such cases telldir will mantain an internal list that maps the DIR object off64_t offsets to the returned long int (the function return value). The seekdir will then set the correct offset from the internal list using the telldir as the list key. It also removes the overflow check on readdir and the returned value will be truncated by the non-LFS off_t size. As Joseph has noted in BZ #23960 comment #22, d_off is an opaque value and since telldir/seekdir works regardless of the returned dirent d_off value. Finally it removed the requirement to check for overflow values on telldir (BZ #24050). Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc-linux-gnu, and arm-linux-gnueabihf. --- dirent/Makefile | 2 +- dirent/tst-seekdir2.c | 156 ++++++++++++++++++++++++++++ support/temp_file.c | 12 ++- support/temp_file.h | 7 ++ sysdeps/unix/sysv/linux/closedir.c | 4 + sysdeps/unix/sysv/linux/dirstream.h | 7 +- sysdeps/unix/sysv/linux/opendir.c | 3 + sysdeps/unix/sysv/linux/readdir.c | 1 + sysdeps/unix/sysv/linux/readdir.h | 7 +- sysdeps/unix/sysv/linux/rewinddir.c | 5 + sysdeps/unix/sysv/linux/seekdir.c | 36 ++++++- sysdeps/unix/sysv/linux/telldir.c | 47 ++++++++- sysdeps/unix/sysv/linux/telldir.h | 64 ++++++++++++ 13 files changed, 333 insertions(+), 18 deletions(-) create mode 100644 dirent/tst-seekdir2.c create mode 100644 sysdeps/unix/sysv/linux/telldir.h diff --git a/dirent/Makefile b/dirent/Makefile index e917d5ceab..ba9755da64 100644 --- a/dirent/Makefile +++ b/dirent/Makefile @@ -31,7 +31,7 @@ routines := opendir closedir readdir readdir_r rewinddir \ scandir-cancel scandir-tail scandir64-tail tests := list tst-seekdir opendir-tst1 bug-readdir1 tst-fdopendir \ - tst-fdopendir2 tst-scandir tst-scandir64 + tst-fdopendir2 tst-scandir tst-scandir64 tst-seekdir2 CFLAGS-scandir.c += $(uses-callbacks) CFLAGS-scandir64.c += $(uses-callbacks) diff --git a/dirent/tst-seekdir2.c b/dirent/tst-seekdir2.c new file mode 100644 index 0000000000..4bd2509f72 --- /dev/null +++ b/dirent/tst-seekdir2.c @@ -0,0 +1,156 @@ +/* Check multiple telldir and seekdir. + Copyright (C) 2020 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include +#include +#include + +#include +#include +#include + +/* Some filesystems returns a arbitrary value for d_off direnty entry (ext4 + for instance, where the value is an internal hash key). The idea of + create a large number of file is to try trigger a overflow d_off value + in a entry to check if telldir/seekdir does work corretly in such + case. */ +static const char *dirname; +static const size_t nfiles = 10240; + +static void +do_prepare (int argc, char *argv[]) +{ + dirname = support_create_temp_directory ("tst-seekdir2-"); + + for (size_t i = 0; i < nfiles; i++) + { + int fd = create_temp_file_in_dir ("tempfile.", dirname, NULL); + TEST_VERIFY_EXIT (fd > 0); + close (fd); + } +} +#define PREPARE do_prepare + +/* Check for old non Large File Support (LFS). */ +static int +do_test_not_lfs (void) +{ + DIR *dirp; + struct dirent *dp; + size_t dirp_count; + + dirp = opendir (dirname); + TEST_VERIFY_EXIT (dirp != NULL); + + dirp_count = 0; + for (dp = readdir (dirp); + dp != NULL; + dp = readdir (dirp)) + dirp_count++; + + /* The 2 extra files are '.' and '..'. */ + TEST_COMPARE (dirp_count, nfiles + 2); + + rewinddir (dirp); + + long *tdirp = xmalloc (dirp_count * sizeof (long)); + struct dirent *ddirp = xmalloc (dirp_count * sizeof (struct dirent)); + + size_t i = 0; + do + { + tdirp[i] = telldir (dirp); + dp = readdir (dirp); + TEST_VERIFY_EXIT (dp != NULL); + memcpy (&ddirp[i], dp, sizeof (struct dirent)); + } while (++i < dirp_count); + + for (i = 0; i < dirp_count - 1; i++) + { + seekdir (dirp, tdirp[i]); + dp = readdir (dirp); + TEST_COMPARE (strcmp (dp->d_name, ddirp[i].d_name), 0); + TEST_COMPARE (dp->d_ino, ddirp[i].d_ino); + TEST_COMPARE (dp->d_off, ddirp[i].d_off); + } + + closedir (dirp); + + return 0; +} + +/* Same as before but with LFS support. */ +static int +do_test_lfs (void) +{ + DIR *dirp; + struct dirent64 *dp; + size_t dirp_count; + + dirp = opendir (dirname); + TEST_VERIFY_EXIT (dirp != NULL); + + dirp_count = 0; + for (dp = readdir64 (dirp); + dp != NULL; + dp = readdir64 (dirp)) + dirp_count++; + + /* The 2 extra files are '.' and '..'. */ + TEST_COMPARE (dirp_count, nfiles + 2); + + rewinddir (dirp); + + long *tdirp = xmalloc (dirp_count * sizeof (long)); + struct dirent64 *ddirp = xmalloc (dirp_count * sizeof (struct dirent64)); + + size_t i = 0; + do + { + tdirp[i] = telldir (dirp); + dp = readdir64 (dirp); + TEST_VERIFY_EXIT (dp != NULL); + memcpy (&ddirp[i], dp, sizeof (struct dirent64)); + } while (++i < dirp_count); + + for (i = 0; i < dirp_count - 1; i++) + { + seekdir (dirp, tdirp[i]); + dp = readdir64 (dirp); + TEST_COMPARE (strcmp (dp->d_name, ddirp[i].d_name), 0); + TEST_COMPARE (dp->d_ino, ddirp[i].d_ino); + TEST_COMPARE (dp->d_off, ddirp[i].d_off); + } + + closedir (dirp); + + return 0; +} + +static int +do_test (void) +{ + do_test_not_lfs (); + do_test_lfs (); + + return 0; +} + +#include diff --git a/support/temp_file.c b/support/temp_file.c index 277c5e0cf1..98bd235526 100644 --- a/support/temp_file.c +++ b/support/temp_file.c @@ -60,14 +60,12 @@ add_temp_file (const char *name) } int -create_temp_file (const char *base, char **filename) +create_temp_file_in_dir (const char *base, const char *dir, char **filename) { char *fname; int fd; - fname = (char *) xmalloc (strlen (test_dir) + 1 + strlen (base) - + sizeof ("XXXXXX")); - strcpy (stpcpy (stpcpy (stpcpy (fname, test_dir), "/"), base), "XXXXXX"); + fname = xasprintf ("%s/%sXXXXXX", dir, base); fd = mkstemp (fname); if (fd == -1) @@ -86,6 +84,12 @@ create_temp_file (const char *base, char **filename) return fd; } +int +create_temp_file (const char *base, char **filename) +{ + return create_temp_file_in_dir (base, test_dir, filename); +} + char * support_create_temp_directory (const char *base) { diff --git a/support/temp_file.h b/support/temp_file.h index 8b6303a6e4..ac61105428 100644 --- a/support/temp_file.h +++ b/support/temp_file.h @@ -32,6 +32,13 @@ void add_temp_file (const char *name); *FILENAME. */ int create_temp_file (const char *base, char **filename); +/* Create a temporary file in directory DIR. Return the opened file + descriptor on success, or -1 on failure. Write the file name to + *FILENAME if FILENAME is not NULL. In this case, the caller is + expected to free *FILENAME. */ +int create_temp_file_in_dir (const char *base, const char *dir, + char **filename); + /* Create a temporary directory and schedule it for deletion. BASE is used as a prefix for the unique directory name, which the function returns. The caller should free this string. */ diff --git a/sysdeps/unix/sysv/linux/closedir.c b/sysdeps/unix/sysv/linux/closedir.c index ccc19eefcd..c39f58aba5 100644 --- a/sysdeps/unix/sysv/linux/closedir.c +++ b/sysdeps/unix/sysv/linux/closedir.c @@ -43,6 +43,10 @@ __closedir (DIR *dirp) fd = dirp->fd; +#ifndef __LP64__ + dirstream_loc_clear (&dirp->locs); +#endif + #if IS_IN (libc) __libc_lock_fini (dirp->lock); #endif diff --git a/sysdeps/unix/sysv/linux/dirstream.h b/sysdeps/unix/sysv/linux/dirstream.h index a3ea2b7197..664d73cd40 100644 --- a/sysdeps/unix/sysv/linux/dirstream.h +++ b/sysdeps/unix/sysv/linux/dirstream.h @@ -21,6 +21,7 @@ #include #include +#include /* Directory stream type. @@ -37,10 +38,14 @@ struct __dirstream size_t size; /* Total valid data in the block. */ size_t offset; /* Current offset into the block. */ - off_t filepos; /* Position of next entry to read. */ + off64_t filepos; /* Position of next entry to read. */ int errcode; /* Delayed error code. */ +#ifndef __LP64__ + struct dirstream_loc_t locs; +#endif + /* Directory block. We must make sure that this block starts at an address that is aligned adequately enough to store dirent entries. Using the alignment of "void *" is not diff --git a/sysdeps/unix/sysv/linux/opendir.c b/sysdeps/unix/sysv/linux/opendir.c index 24bd63d2ba..fb32c95ad0 100644 --- a/sysdeps/unix/sysv/linux/opendir.c +++ b/sysdeps/unix/sysv/linux/opendir.c @@ -130,6 +130,9 @@ __alloc_dir (int fd, bool close_fd, int flags, const struct stat64 *statp) dirp->offset = 0; dirp->filepos = 0; dirp->errcode = 0; +#ifndef __LP64__ + dirstream_loc_init (&dirp->locs); +#endif return dirp; } diff --git a/sysdeps/unix/sysv/linux/readdir.c b/sysdeps/unix/sysv/linux/readdir.c index 8eab0f4c9b..010ccf0a00 100644 --- a/sysdeps/unix/sysv/linux/readdir.c +++ b/sysdeps/unix/sysv/linux/readdir.c @@ -17,6 +17,7 @@ . */ #include +#include #if !_DIRENT_MATCHES_DIRENT64 #include diff --git a/sysdeps/unix/sysv/linux/readdir.h b/sysdeps/unix/sysv/linux/readdir.h index 4dc219e220..7f6cf3783d 100644 --- a/sysdeps/unix/sysv/linux/readdir.h +++ b/sysdeps/unix/sysv/linux/readdir.h @@ -85,15 +85,12 @@ dirstream_ret_entry (struct __dirstream *ds) dp->d_ino = dp64->d_ino; dp->d_off = dp64->d_off; - if (dp->d_off != dp64->d_off) - /* Overflow. */ - return NULL; const size_t size_diff = (offsetof (struct dirent64, d_name) - offsetof (struct dirent, d_name)); const size_t alignment = _Alignof (struct dirent); - size_t new_reclen = (dp64->d_reclen - size_diff + alignment - 1) - & ~(alignment - 1); + size_t new_reclen = (dp64->d_reclen - size_diff + alignment - 1) + & ~(alignment - 1); if (new_reclen > return_buffer_size) /* Overflow. */ return NULL; diff --git a/sysdeps/unix/sysv/linux/rewinddir.c b/sysdeps/unix/sysv/linux/rewinddir.c index 860bfda004..8db0d0be4a 100644 --- a/sysdeps/unix/sysv/linux/rewinddir.c +++ b/sysdeps/unix/sysv/linux/rewinddir.c @@ -33,6 +33,11 @@ __rewinddir (DIR *dirp) dirp->offset = 0; dirp->size = 0; dirp->errcode = 0; + +#ifndef __LP64__ + dirstream_loc_clear (&dirp->locs); +#endif + #if IS_IN (libc) __libc_lock_unlock (dirp->lock); #endif diff --git a/sysdeps/unix/sysv/linux/seekdir.c b/sysdeps/unix/sysv/linux/seekdir.c index 3c30520928..5f39ef9eef 100644 --- a/sysdeps/unix/sysv/linux/seekdir.c +++ b/sysdeps/unix/sysv/linux/seekdir.c @@ -22,14 +22,40 @@ #include /* Seek to position POS in DIRP. */ -/* XXX should be __seekdir ? */ void seekdir (DIR *dirp, long int pos) { + off64_t filepos; + __libc_lock_lock (dirp->lock); - (void) __lseek (dirp->fd, pos, SEEK_SET); - dirp->size = 0; - dirp->offset = 0; - dirp->filepos = pos; + +#ifndef __LP64__ + union dirstream_packed dsp; + + dsp.l = pos; + + if (dsp.p.is_packed == 1) + filepos = dsp.p.info; + else + { + size_t index = dsp.p.info; + + if (index >= dirstream_loc_size (&dirp->locs)) + return; + struct dirstream_loc *loc = dirstream_loc_at (&dirp->locs, index); + filepos = loc->filepos; + } +#else + filepos = pos; +#endif + + if (dirp->filepos != filepos) + { + __lseek64 (dirp->fd, filepos, SEEK_SET); + dirp->filepos = filepos; + dirp->offset = 0; + dirp->size = 0; + } + __libc_lock_unlock (dirp->lock); } diff --git a/sysdeps/unix/sysv/linux/telldir.c b/sysdeps/unix/sysv/linux/telldir.c index 57d435ed21..bb33626fa4 100644 --- a/sysdeps/unix/sysv/linux/telldir.c +++ b/sysdeps/unix/sysv/linux/telldir.c @@ -18,16 +18,59 @@ #include #include +#include /* Return the current position of DIRP. */ long int telldir (DIR *dirp) { - long int ret; +#ifndef __LP64__ + /* If the directory position fits in the packet structure returns it. + Otherwise, check if the position is already been recorded in the + dynamic array. If not, add the new record. */ + + union dirstream_packed dsp; + size_t i; __libc_lock_lock (dirp->lock); - ret = dirp->filepos; + + if (dirp->filepos < (1U << 31)) + { + dsp.p.is_packed = 1; + dsp.p.info = dirp->filepos; + goto out; + } + + dsp.l = -1; + + for (i = 0; i < dirstream_loc_size (&dirp->locs); i++) + { + struct dirstream_loc *loc = dirstream_loc_at (&dirp->locs, i); + if (loc->filepos == dirp->filepos) + break; + } + if (i == dirstream_loc_size (&dirp->locs)) + { + dirstream_loc_add (&dirp->locs, + (struct dirstream_loc) { dirp->filepos }); + if (dirstream_loc_has_failed (&dirp->locs)) + goto out; + } + + dsp.p.is_packed = 0; + /* This assignment might overflow, however most likely ENOMEM would happen + long before. */ + dsp.p.info = i; + +out: __libc_lock_unlock (dirp->lock); + return dsp.l; +#else + long int ret; + __libc_lock_lock (dirp->lock); + ret = dirp->filepos; + __libc_lock_unlock (dirp->lock); return ret; +#endif } diff --git a/sysdeps/unix/sysv/linux/telldir.h b/sysdeps/unix/sysv/linux/telldir.h new file mode 100644 index 0000000000..7c45886341 --- /dev/null +++ b/sysdeps/unix/sysv/linux/telldir.h @@ -0,0 +1,64 @@ +/* Linux internal telldir definitions. + Copyright (C) 2020 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef _TELLDIR_H +#define _TELLDIR_H 1 + +#ifndef __LP64__ + +/* On platforms where long int is smaller than off64_t this is how the + returned value is encoded and returned by 'telldir'. If the directory + offset can be enconded in 31 bits it is returned in the 'info' member + with 'is_packed' set to 1. + + Otherwise, the 'info' member describes an index in a dynamic array at + 'DIR' structure. */ + +union dirstream_packed +{ + long int l; + struct + { + unsigned long is_packed:1; + unsigned long info:31; + } p; +}; + +_Static_assert (sizeof (long int) == sizeof (union dirstream_packed), + "sizeof (long int) != sizeof (union dirstream_packed)"); + +/* telldir will mantain a list of offsets that describe the obtained diretory + position if it can fit this information in the returned 'dirstream_packed' + struct. */ + +struct dirstream_loc +{ + off64_t filepos; +}; + +# define DYNARRAY_STRUCT dirstream_loc_t +# define DYNARRAY_ELEMENT struct dirstream_loc +# define DYNARRAY_PREFIX dirstream_loc_ +# include +#else + +_Static_assert (sizeof (long int) == sizeof (off64_t), + "sizeof (long int) != sizeof (off64_t)"); +#endif /* __LP64__ */ + +#endif /* _TELLDIR_H */ From patchwork Fri Oct 2 17:06:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 40650 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id F08C0398E46F; Fri, 2 Oct 2020 17:06:42 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org F08C0398E46F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1601658403; bh=PPwH06qnQmmGHRWXChhnA0RSdD5/bOqMyXMMcgSHwNA=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=grB7fZSsFX0fp0v4FoURliRB45QBx+rCvJHFxiHkx9av2Ulj0B7dt7Zo6+JptvXAl i97XyOoPLk8elb91gErjDeH4duAPb3/kQz0MzXNfNP41pNtvPFKocNsHmPU7k4QYsP /hxGNtMqCC6PNbpjBTvPa19cfrzFcM1+mz6GLQ4c= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by sourceware.org (Postfix) with ESMTPS id C11C5398E46B for ; Fri, 2 Oct 2020 17:06:39 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org C11C5398E46B Received: by mail-qk1-x744.google.com with SMTP id s131so2120467qke.0 for ; Fri, 02 Oct 2020 10:06:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PPwH06qnQmmGHRWXChhnA0RSdD5/bOqMyXMMcgSHwNA=; b=WS6f/SH6KTd3RJJtFBL9wPgRlErTgea+ylhkw6uYh5ujd1XfBNXF2pwHK/TRggVFSV mvqxaPchIAcoNVHle18VUUdJ5IudQ6THaKrErjg97+fXCop8dnmYITKRUGw3ERmB4nih AwYtXH2Y3XW2qaaZFmwHz4Fr033bTGl8j/AV9brtQ/wpBsOjkBUer1/ORsdB73oUVLWt nVOCCnuMWN/n515ApHiujxoC404E3uQ0LzMKwYGA+x78twOZ4vqB6XKKsKEg9hiMgoGn VHfE6m8IeMBAEI1FWkyAGn3WKh7ssCdgdlb/sVCP2lTqESPuCpUN2GMxbKsqcEePkg1L emgQ== X-Gm-Message-State: AOAM533WmzdTLLZpKEnXzAhS8EhOmquLzy6GpyIgtZgmiDfo59+6C7ed mTeM3Dh+EVNFivXBt5QN+cq3JXAmQV6pjQ== X-Google-Smtp-Source: ABdhPJzkwvdQdBA30X6pdsMQXwCrIuXHj5T4IZUmQjVPO6I2pTwvNHExUHXayP9kcUIdsak7bUyIow== X-Received: by 2002:a05:620a:78b:: with SMTP id 11mr3155464qka.370.1601658398955; Fri, 02 Oct 2020 10:06:38 -0700 (PDT) Received: from localhost.localdomain ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id d10sm1436891qkk.1.2020.10.02.10.06.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Oct 2020 10:06:38 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v2 6/9] linux: Add __readdir64_unlocked Date: Fri, 2 Oct 2020 14:06:17 -0300 Message-Id: <20201002170620.1611673-7-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201002170620.1611673-1-adhemerval.zanella@linaro.org> References: <20201002170620.1611673-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-13.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Cc: James Clarke , John Paul Adrian Glaubitz Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" And use it on readdir_r implementation. Checked on i686-linux-gnu. --- include/dirent.h | 1 + sysdeps/unix/sysv/linux/readdir64.c | 20 +++++-- sysdeps/unix/sysv/linux/readdir64_r.c | 80 ++++++--------------------- 3 files changed, 33 insertions(+), 68 deletions(-) diff --git a/include/dirent.h b/include/dirent.h index 8325a19e5f..79c8fce969 100644 --- a/include/dirent.h +++ b/include/dirent.h @@ -21,6 +21,7 @@ extern DIR *__fdopendir (int __fd) attribute_hidden; extern int __closedir (DIR *__dirp) attribute_hidden; extern struct dirent *__readdir (DIR *__dirp) attribute_hidden; extern struct dirent *__readdir_unlocked (DIR *__dirp) attribute_hidden; +extern struct dirent64 *__readdir64_unlocked (DIR *__dirp) attribute_hidden; extern struct dirent64 *__readdir64 (DIR *__dirp); libc_hidden_proto (__readdir64) extern int __readdir_r (DIR *__dirp, struct dirent *__entry, diff --git a/sysdeps/unix/sysv/linux/readdir64.c b/sysdeps/unix/sysv/linux/readdir64.c index 1aa6e2664f..a774f7c6e3 100644 --- a/sysdeps/unix/sysv/linux/readdir64.c +++ b/sysdeps/unix/sysv/linux/readdir64.c @@ -28,15 +28,11 @@ /* Read a directory entry from DIRP. */ struct dirent64 * -__readdir64 (DIR *dirp) +__readdir64_unlocked (DIR *dirp) { struct dirent64 *dp; int saved_errno = errno; -#if IS_IN (libc) - __libc_lock_lock (dirp->lock); -#endif - do { size_t reclen; @@ -80,6 +76,20 @@ __readdir64 (DIR *dirp) /* Skip deleted files. */ } while (dp->d_ino == 0); + return dp; +} + +struct dirent64 * +__readdir64 (DIR *dirp) +{ + struct dirent64 *dp; + +#if IS_IN (libc) + __libc_lock_lock (dirp->lock); +#endif + + dp = __readdir64_unlocked (dirp); + #if IS_IN (libc) __libc_lock_unlock (dirp->lock); #endif diff --git a/sysdeps/unix/sysv/linux/readdir64_r.c b/sysdeps/unix/sysv/linux/readdir64_r.c index c587787417..e5ef7be4c2 100644 --- a/sysdeps/unix/sysv/linux/readdir64_r.c +++ b/sysdeps/unix/sysv/linux/readdir64_r.c @@ -32,89 +32,43 @@ __readdir64_r (DIR *dirp, struct dirent64 *entry, struct dirent64 **result) { struct dirent64 *dp; size_t reclen; - const int saved_errno = errno; - int ret; __libc_lock_lock (dirp->lock); - - do + while (1) { - if (dirp->offset >= dirp->size) - { - /* We've emptied out our buffer. Refill it. */ - - size_t maxread = dirp->allocation; - ssize_t bytes; - - maxread = dirp->allocation; - - bytes = __getdents64 (dirp->fd, dirp->data, maxread); - if (bytes <= 0) - { - /* On some systems getdents fails with ENOENT when the - open directory has been rmdir'd already. POSIX.1 - requires that we treat this condition like normal EOF. */ - if (bytes < 0 && errno == ENOENT) - { - bytes = 0; - __set_errno (saved_errno); - } - if (bytes < 0) - dirp->errcode = errno; - - dp = NULL; - break; - } - dirp->size = (size_t) bytes; - - /* Reset the offset into the buffer. */ - dirp->offset = 0; - } - - dp = (struct dirent64 *) &dirp->data[dirp->offset]; + dp = __readdir64_unlocked (dirp); + if (dp == NULL) + break; reclen = dp->d_reclen; + if (reclen <= offsetof (struct dirent64, d_name) + NAME_MAX + 1) + break; - dirp->offset += reclen; - - dirp->filepos = dp->d_off; - - if (reclen > offsetof (struct dirent64, d_name) + NAME_MAX + 1) + /* The record is very long. It could still fit into the caller-supplied + buffer if we can skip padding at the end. */ + size_t namelen = _D_EXACT_NAMLEN (dp); + if (namelen <= NAME_MAX) { - /* The record is very long. It could still fit into the - caller-supplied buffer if we can skip padding at the - end. */ - size_t namelen = _D_EXACT_NAMLEN (dp); - if (namelen <= NAME_MAX) - reclen = offsetof (struct dirent64, d_name) + namelen + 1; - else - { - /* The name is too long. Ignore this file. */ - dirp->errcode = ENAMETOOLONG; - dp->d_ino = 0; - continue; - } + reclen = offsetof (struct dirent64, d_name) + namelen + 1; + break; } - /* Skip deleted and ignored files. */ + /* The name is too long. Ignore this file. */ + dirp->errcode = ENAMETOOLONG; + dp->d_ino = 0; } - while (dp->d_ino == 0); if (dp != NULL) { *result = memcpy (entry, dp, reclen); entry->d_reclen = reclen; - ret = 0; } else - { - *result = NULL; - ret = dirp->errcode; - } + *result = NULL; __libc_lock_unlock (dirp->lock); - return ret; + return dp != NULL ? 0 : dirp->errcode; } From patchwork Fri Oct 2 17:06:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 40651 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 84E51398E466; Fri, 2 Oct 2020 17:06:44 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 84E51398E466 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1601658404; bh=hcYfMn8gDBQBig5xNKCq/Vw8vFAmnPW3oVn3Vmb9JTQ=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=d2DaZY6YSyeTX5TivYT2zbbInSBiJcc0fAoL0LMhMTtP6M9hgwCJfihCqwcoS+Drv dEB+lUlYWCsWoIy4irot/JkD8gW8rY2zWhwMYswN+2KRv+WDkuehxZrG2dYvQQn+yB 7n5JW0UFdV+PAsi5su7LxLkvGU9O8UURxirBB5W4= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by sourceware.org (Postfix) with ESMTPS id ABE90398E460 for ; Fri, 2 Oct 2020 17:06:41 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org ABE90398E460 Received: by mail-qk1-x744.google.com with SMTP id w12so2089805qki.6 for ; Fri, 02 Oct 2020 10:06:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hcYfMn8gDBQBig5xNKCq/Vw8vFAmnPW3oVn3Vmb9JTQ=; b=Tzzehn+ktcPLvKXTuQ59pS+9+LGYYhEG7mw9d3Ad4xFMrES0yAGgHMIN7aB5PRrGkx uDA6/apqCwz+WT0ti/Mr5pXb4eJbO3BeE6AWlKc2nWhYzHbE1FzTIu6yZSxa40bbUE6L 0YsNOZx1esjmIdUXrwV8pbUEdpBZ0GAxkXIUPgWDCAxICvhdZyRLi6fQ64WiEzZKFu6S Jb2KP8uJgPQBn3/vvE7d3aM4em2FHx8fSj4MNrXITgQjRr8nO/7kVA/gNEt5e+PQj9KE qgg8UvFojFQ+hVd1WjHgxs2BVywmP69Lh+fFbFbmQ/itpMNqDNqh6DXpMDjzZFmVBYeQ Xiqw== X-Gm-Message-State: AOAM5333iQ0N0HfmvxmHmImLbzP0FeE1thRYYtnW6tZgZoliN7VuPSCy Mapv9L4+bWKZ2XyTHivvS0C7SU0YQ9hAOg== X-Google-Smtp-Source: ABdhPJyuk6lbqG7DGui8252/QLCYHsrBLRbk4GemE0GeT2JLRodtykuf4M4hF7An8Ye+qmPxU52eXQ== X-Received: by 2002:a37:6393:: with SMTP id x141mr2995945qkb.238.1601658400974; Fri, 02 Oct 2020 10:06:40 -0700 (PDT) Received: from localhost.localdomain ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id d10sm1436891qkk.1.2020.10.02.10.06.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Oct 2020 10:06:40 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v2 7/9] linux: Add __old_readdir64_unlocked Date: Fri, 2 Oct 2020 14:06:18 -0300 Message-Id: <20201002170620.1611673-8-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201002170620.1611673-1-adhemerval.zanella@linaro.org> References: <20201002170620.1611673-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-13.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Cc: James Clarke , John Paul Adrian Glaubitz Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" And use it __old_readdir64_r. Checked on i686-linux-gnu. --- sysdeps/unix/sysv/linux/olddirent.h | 2 + sysdeps/unix/sysv/linux/readdir64.c | 21 +++++-- sysdeps/unix/sysv/linux/readdir64_r.c | 79 ++++++--------------------- 3 files changed, 35 insertions(+), 67 deletions(-) diff --git a/sysdeps/unix/sysv/linux/olddirent.h b/sysdeps/unix/sysv/linux/olddirent.h index 2d2559c1e9..b118b0ef96 100644 --- a/sysdeps/unix/sysv/linux/olddirent.h +++ b/sysdeps/unix/sysv/linux/olddirent.h @@ -32,6 +32,8 @@ struct __old_dirent64 /* Now define the internal interfaces. */ extern struct __old_dirent64 *__old_readdir64 (DIR *__dirp); libc_hidden_proto (__old_readdir64); +extern struct __old_dirent64 *__old_readdir64_unlocked (DIR *__dirp) + attribute_hidden; extern int __old_readdir64_r (DIR *__dirp, struct __old_dirent64 *__entry, struct __old_dirent64 **__result); extern __ssize_t __old_getdents64 (int __fd, char *__buf, size_t __nbytes) diff --git a/sysdeps/unix/sysv/linux/readdir64.c b/sysdeps/unix/sysv/linux/readdir64.c index a774f7c6e3..ce50067637 100644 --- a/sysdeps/unix/sysv/linux/readdir64.c +++ b/sysdeps/unix/sysv/linux/readdir64.c @@ -116,15 +116,11 @@ versioned_symbol (libc, __readdir64, readdir64, GLIBC_2_2); attribute_compat_text_section struct __old_dirent64 * -__old_readdir64 (DIR *dirp) +__old_readdir64_unlocked (DIR *dirp) { struct __old_dirent64 *dp; int saved_errno = errno; -#if IS_IN (libc) - __libc_lock_lock (dirp->lock); -#endif - do { size_t reclen; @@ -168,6 +164,21 @@ __old_readdir64 (DIR *dirp) /* Skip deleted files. */ } while (dp->d_ino == 0); + return dp; +} + +attribute_compat_text_section +struct __old_dirent64 * +__old_readdir64 (DIR *dirp) +{ + struct __old_dirent64 *dp; + +#if IS_IN (libc) + __libc_lock_lock (dirp->lock); +#endif + + dp = __old_readdir64_unlocked (dirp); + #if IS_IN (libc) __libc_lock_unlock (dirp->lock); #endif diff --git a/sysdeps/unix/sysv/linux/readdir64_r.c b/sysdeps/unix/sysv/linux/readdir64_r.c index e5ef7be4c2..133c534a69 100644 --- a/sysdeps/unix/sysv/linux/readdir64_r.c +++ b/sysdeps/unix/sysv/linux/readdir64_r.c @@ -91,89 +91,44 @@ __old_readdir64_r (DIR *dirp, struct __old_dirent64 *entry, { struct __old_dirent64 *dp; size_t reclen; - const int saved_errno = errno; - int ret; __libc_lock_lock (dirp->lock); - do + while (1) { - if (dirp->offset >= dirp->size) - { - /* We've emptied out our buffer. Refill it. */ - - size_t maxread = dirp->allocation; - ssize_t bytes; - - maxread = dirp->allocation; - - bytes = __old_getdents64 (dirp->fd, dirp->data, maxread); - if (bytes <= 0) - { - /* On some systems getdents fails with ENOENT when the - open directory has been rmdir'd already. POSIX.1 - requires that we treat this condition like normal EOF. */ - if (bytes < 0 && errno == ENOENT) - { - bytes = 0; - __set_errno (saved_errno); - } - if (bytes < 0) - dirp->errcode = errno; - - dp = NULL; - break; - } - dirp->size = (size_t) bytes; - - /* Reset the offset into the buffer. */ - dirp->offset = 0; - } - - dp = (struct __old_dirent64 *) &dirp->data[dirp->offset]; + dp = __old_readdir64_unlocked (dirp); + if (dp == NULL) + break; reclen = dp->d_reclen; + if (reclen <= offsetof (struct __old_dirent64, d_name) + NAME_MAX + 1) + break; - dirp->offset += reclen; - - dirp->filepos = dp->d_off; - - if (reclen > offsetof (struct __old_dirent64, d_name) + NAME_MAX + 1) + /* The record is very long. It could still fit into the caller-supplied + buffer if we can skip padding at the end. */ + size_t namelen = _D_EXACT_NAMLEN (dp); + if (namelen <= NAME_MAX) { - /* The record is very long. It could still fit into the - caller-supplied buffer if we can skip padding at the - end. */ - size_t namelen = _D_EXACT_NAMLEN (dp); - if (namelen <= NAME_MAX) - reclen = offsetof (struct __old_dirent64, d_name) + namelen + 1; - else - { - /* The name is too long. Ignore this file. */ - dirp->errcode = ENAMETOOLONG; - dp->d_ino = 0; - continue; - } + reclen = offsetof (struct dirent64, d_name) + namelen + 1; + break; } - /* Skip deleted and ignored files. */ + /* The name is too long. Ignore this file. */ + dirp->errcode = ENAMETOOLONG; + dp->d_ino = 0; } - while (dp->d_ino == 0); if (dp != NULL) { *result = memcpy (entry, dp, reclen); entry->d_reclen = reclen; - ret = 0; } else - { - *result = NULL; - ret = dirp->errcode; - } + *result = NULL; __libc_lock_unlock (dirp->lock); - return ret; + return dp != NULL ? 0 : dirp->errcode; } compat_symbol (libc, __old_readdir64_r, readdir64_r, GLIBC_2_1); From patchwork Fri Oct 2 17:06:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 40652 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id B845C398E477; Fri, 2 Oct 2020 17:06:47 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org B845C398E477 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1601658407; bh=HeNI1z7FIexcJ86bYdzRjxj+bPU26XzFK+zaYtYhcQs=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=yqCRhaEERUPpFFv0n3XvjyyWVWAsPNtGD27wFRhl7TCq9XsItPWK7P3W6YyoFRO0O PyovVkpaYm8ZreruHr810dcg2nNVe1cr93XNVaDVY4lYwrkO8XBUablyOEc8Z6hfo+ MdL3+ndN/kvZRzyR8CB0Ne8CT0ePS1IVjXEBCBaU= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by sourceware.org (Postfix) with ESMTPS id 92415398E477 for ; Fri, 2 Oct 2020 17:06:44 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 92415398E477 Received: by mail-qk1-x744.google.com with SMTP id q5so2107409qkc.2 for ; Fri, 02 Oct 2020 10:06:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HeNI1z7FIexcJ86bYdzRjxj+bPU26XzFK+zaYtYhcQs=; b=CsD3+XDBBvvyWQR2Io1WoImP6tjL0jDiSHnWN7ecIxNU8oodRvBt1BLopocR6Dbp+w BsnLg7SeyD6dBHRNKbjak9YYH0f80w0ti7H1PEWu5s1rIkK57TSfwhV0oQ1m288tr9rH U72vjpFyH6kQzlsPmm/i3ti5cH6wzkgLEoJ6ApfedzvvwqR+YB4H/PHArE5AVWlUUAuZ RQOcY2ZSMm3YsdaVkYXOAgcf6qrhtKSylh1SKSOSFY9BLEUNJtrHOLakNQmVSkkvVFdT dYEgrR0T8cprVA8YA/HOml7SZgaaiJlKwR6mrzEGZf2PBp0AcOMP7HjfD2JYZ34KxvxN 3Nuw== X-Gm-Message-State: AOAM531P04ifzAK3Tt+kjTx+J25mKnslc9iYZFSbGEg12u/C9JOhDBGa oLdh8UJj+IohC0Ldu+1a/4MnBAAtO3SkRw== X-Google-Smtp-Source: ABdhPJxHbdLjjH4WPpdxrGI+RM2g35ywvNPV65RNnHeauIs5qdyCs9uJXE5KXhBCZzridFlD1CpF0g== X-Received: by 2002:a05:620a:c97:: with SMTP id q23mr3127630qki.168.1601658403298; Fri, 02 Oct 2020 10:06:43 -0700 (PDT) Received: from localhost.localdomain ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id d10sm1436891qkk.1.2020.10.02.10.06.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Oct 2020 10:06:42 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v2 8/9] linux: Use getdents64 on readdir64 compat implementation Date: Fri, 2 Oct 2020 14:06:19 -0300 Message-Id: <20201002170620.1611673-9-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201002170620.1611673-1-adhemerval.zanella@linaro.org> References: <20201002170620.1611673-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-13.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Cc: James Clarke , John Paul Adrian Glaubitz Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" It uses a similar strategy from the non-LFS readdir that also uses getdents64 internally and reserves some space on the allocated internal DIR buffer to be used as a temporary buffer. The kernel obtained dirent64 data are copied to the temporary buffer for each readdir compat call. It allows to remove __old_getdents64. Checked on i686-linux-gnu. --- sysdeps/unix/sysv/linux/getdents64.c | 95 +--------------------------- sysdeps/unix/sysv/linux/olddirent.h | 2 - sysdeps/unix/sysv/linux/readdir.h | 47 +++++++++++++- sysdeps/unix/sysv/linux/readdir64.c | 27 ++++---- 4 files changed, 58 insertions(+), 113 deletions(-) diff --git a/sysdeps/unix/sysv/linux/getdents64.c b/sysdeps/unix/sysv/linux/getdents64.c index 75892c2823..f40dfacbfe 100644 --- a/sysdeps/unix/sysv/linux/getdents64.c +++ b/sysdeps/unix/sysv/linux/getdents64.c @@ -36,97 +36,4 @@ weak_alias (__getdents64, getdents64) #if _DIRENT_MATCHES_DIRENT64 strong_alias (__getdents64, __getdents) -#else -# include - -# if SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2) -# include -# include - -static ssize_t -handle_overflow (int fd, __off64_t offset, ssize_t count) -{ - /* If this is the first entry in the buffer, we can report the - error. */ - if (offset == 0) - { - __set_errno (EOVERFLOW); - return -1; - } - - /* Otherwise, seek to the overflowing entry, so that the next call - will report the error, and return the data read so far. */ - if (__lseek64 (fd, offset, SEEK_SET) != 0) - return -1; - return count; -} - -ssize_t -__old_getdents64 (int fd, char *buf, size_t nbytes) -{ - /* We do not move the individual directory entries. This is only - possible if the target type (struct __old_dirent64) is smaller - than the source type. */ - _Static_assert (offsetof (struct __old_dirent64, d_name) - <= offsetof (struct dirent64, d_name), - "__old_dirent64 is larger than dirent64"); - _Static_assert (__alignof__ (struct __old_dirent64) - <= __alignof__ (struct dirent64), - "alignment of __old_dirent64 is larger than dirent64"); - - ssize_t retval = INLINE_SYSCALL_CALL (getdents64, fd, buf, nbytes); - if (retval > 0) - { - /* This is the marker for the first entry. Offset 0 is reserved - for the first entry (see rewinddir). Here, we use it as a - marker for the first entry in the buffer. We never actually - seek to offset 0 because handle_overflow reports the error - directly, so it does not matter that the offset is incorrect - if entries have been read from the descriptor before (so that - the descriptor is not actually at offset 0). */ - __off64_t previous_offset = 0; - - char *p = buf; - char *end = buf + retval; - while (p < end) - { - struct dirent64 *source = (struct dirent64 *) p; - - /* Copy out the fixed-size data. */ - __ino_t ino = source->d_ino; - __off64_t offset = source->d_off; - unsigned int reclen = source->d_reclen; - unsigned char type = source->d_type; - - /* Check for ino_t overflow. */ - if (__glibc_unlikely (ino != source->d_ino)) - return handle_overflow (fd, previous_offset, p - buf); - - /* Convert to the target layout. Use a separate struct and - memcpy to side-step aliasing issues. */ - struct __old_dirent64 result; - result.d_ino = ino; - result.d_off = offset; - result.d_reclen = reclen; - result.d_type = type; - - /* Write the fixed-sized part of the result to the - buffer. */ - size_t result_name_offset = offsetof (struct __old_dirent64, d_name); - memcpy (p, &result, result_name_offset); - - /* Adjust the position of the name if necessary. Copy - everything until the end of the record, including the - terminating NUL byte. */ - if (result_name_offset != offsetof (struct dirent64, d_name)) - memmove (p + result_name_offset, source->d_name, - reclen - offsetof (struct dirent64, d_name)); - - p += reclen; - previous_offset = offset; - } - } - return retval; -} -# endif /* SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2) */ -#endif /* _DIRENT_MATCHES_DIRENT64 */ +#endif diff --git a/sysdeps/unix/sysv/linux/olddirent.h b/sysdeps/unix/sysv/linux/olddirent.h index b118b0ef96..63d8e97868 100644 --- a/sysdeps/unix/sysv/linux/olddirent.h +++ b/sysdeps/unix/sysv/linux/olddirent.h @@ -36,8 +36,6 @@ extern struct __old_dirent64 *__old_readdir64_unlocked (DIR *__dirp) attribute_hidden; extern int __old_readdir64_r (DIR *__dirp, struct __old_dirent64 *__entry, struct __old_dirent64 **__result); -extern __ssize_t __old_getdents64 (int __fd, char *__buf, size_t __nbytes) - attribute_hidden; int __old_scandir64 (const char * __dir, struct __old_dirent64 *** __namelist, int (*__selector) (const struct __old_dirent64 *), diff --git a/sysdeps/unix/sysv/linux/readdir.h b/sysdeps/unix/sysv/linux/readdir.h index 7f6cf3783d..234a7f26e6 100644 --- a/sysdeps/unix/sysv/linux/readdir.h +++ b/sysdeps/unix/sysv/linux/readdir.h @@ -21,6 +21,7 @@ #if !_DIRENT_MATCHES_DIRENT64 # include +# include /* getdents64 is used internally for both LFS and non-LFS implementations. The non-LFS interface reserves part of the allocated buffer to return the @@ -106,9 +107,53 @@ dirstream_ret_entry (struct __dirstream *ds) return dp; } + +/* Return the allocated buffer used on LFS compat readdir call. */ +static inline struct __old_dirent64 * +dirstream_ret64_compat (struct __dirstream *ds) +{ + return (struct __old_dirent64 *) ds->data; +} + +static inline struct __old_dirent64 * +dirstream_ret_entry64_compat (struct __dirstream *ds) +{ + struct dirent64 *dp64 = dirstream_entry (ds); + struct __old_dirent64 *dp64_compat = dirstream_ret64_compat (ds); + + dp64_compat->d_ino = dp64->d_ino; + if (dp64_compat->d_ino != dp64->d_ino) + /* Overflow. */ + return NULL; + + dp64_compat->d_off = dp64->d_off; + + const size_t size_diff = (offsetof (struct dirent64, d_name) + - offsetof (struct __old_dirent64, d_name)); + const size_t alignment = _Alignof (struct __old_dirent64); + size_t new_reclen = (dp64->d_reclen - size_diff + alignment - 1) + & ~(alignment - 1); + if (new_reclen > return_buffer_size) + /* Overflow. */ + return NULL; + + /* The compat symbol report the kernel obtained d_reclen, even though + it has an incompatible dirent layout. */ + dp64_compat->d_reclen = dp64->d_reclen; + + dp64_compat->d_type = dp64->d_type; + + memcpy (dp64_compat->d_name, dp64->d_name, + dp64->d_reclen - offsetof (struct dirent64, d_name)); + + ds->offset += dp64->d_reclen; + ds->filepos = dp64->d_off; + + return dp64_compat; +} + #else /* No need to reserve an buffer space if dirent has already LFS support. */ enum { return_buffer_size = 0 }; #endif /* _DIRENT_MATCHES_DIRENT64 */ - #endif diff --git a/sysdeps/unix/sysv/linux/readdir64.c b/sysdeps/unix/sysv/linux/readdir64.c index ce50067637..79a2027fd1 100644 --- a/sysdeps/unix/sysv/linux/readdir64.c +++ b/sysdeps/unix/sysv/linux/readdir64.c @@ -112,27 +112,23 @@ weak_alias (__readdir64, readdir64) versioned_symbol (libc, __readdir64, readdir64, GLIBC_2_2); # endif # if SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2) -# include +# include +# include attribute_compat_text_section struct __old_dirent64 * __old_readdir64_unlocked (DIR *dirp) { struct __old_dirent64 *dp; - int saved_errno = errno; + const int saved_errno = errno; do { - size_t reclen; - if (dirp->offset >= dirp->size) { /* We've emptied out our buffer. Refill it. */ - - size_t maxread = dirp->allocation; - ssize_t bytes; - - bytes = __old_getdents64 (dirp->fd, dirp->data, maxread); + ssize_t bytes = __getdents64 (dirp->fd, dirstream_data (dirp), + dirstream_alloc_size (dirp)); if (bytes <= 0) { /* On some systems getdents fails with ENOENT when the @@ -153,13 +149,12 @@ __old_readdir64_unlocked (DIR *dirp) dirp->offset = 0; } - dp = (struct __old_dirent64 *) &dirp->data[dirp->offset]; - - reclen = dp->d_reclen; - - dirp->offset += reclen; - - dirp->filepos = dp->d_off; + dp = dirstream_ret_entry64_compat (dirp); + if (dp == NULL) + { + __set_errno (EOVERFLOW); + break; + } /* Skip deleted files. */ } while (dp->d_ino == 0); From patchwork Fri Oct 2 17:06:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 40653 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 4BB56398E478; Fri, 2 Oct 2020 17:06:49 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 4BB56398E478 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1601658409; bh=+ah46WTCXqxdiJQqFvLewS8EU7dq4ZW+Eufwr66uj2E=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=Ict3f+GNXCFE8fJFaiXlRAR9KfGE1viL15n8Cp+PcgyHqfPdGAP26kfGyoYPMgSfL 5eC7EGiGOc1FN4DtbnEk21kQGnIQc9APqGtiAwl9RgmY7E5d3iWT32IXsSvLN0g7kq Eyr9j5oX5uSlpNIuAsWkJ+nSE1ZQUxOoubcsw45g= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qk1-x741.google.com (mail-qk1-x741.google.com [IPv6:2607:f8b0:4864:20::741]) by sourceware.org (Postfix) with ESMTPS id 4E8BF398E469 for ; Fri, 2 Oct 2020 17:06:46 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 4E8BF398E469 Received: by mail-qk1-x741.google.com with SMTP id c2so2074919qkf.10 for ; Fri, 02 Oct 2020 10:06:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+ah46WTCXqxdiJQqFvLewS8EU7dq4ZW+Eufwr66uj2E=; b=ujB4p68YHgz4C6QtnyF6Sb1yk0/mjRVns4g9BxQnQM0j5OH9edHbCXueXGjVPAao+Z I6gnVreHR5EQ0dwJKcHC15uYsbYd5LtZRNjvh+N5ohMLJOP8erz3hgn0ye0xD0PZL+kj U7+9OKNAtVh3EcZq/AssmSqpMVkji1fySIAm7FKYkCY12IFFg23k7QUxwu49I6i9GptP 7Y7tFU4jBZtjH8F56k0gPPGzl9Wd0XIw0xyLnMngpKXu0RQnI07Un2LXrYTdg+gbd3Wx oRYPyOKzyYVtGP6kGdCuqU+bvh6MElcvyEBpNLhQUrXMSGxZ/B2yxBVgWZyfYkJPyW9c 0Bcw== X-Gm-Message-State: AOAM532dGoAKB+lZApyy4p+e5WWeKIhg49JNPExd4Dv3tYpMhhOyNj69 O8WizIpiQbghJ07OsXe31k0VHlBGqC/F8w== X-Google-Smtp-Source: ABdhPJxvRZcRAOaJhYS9u4Hved03gOympUQgzjcNVHBDxN7i4C1ycP2D6wACb1x5lLtZH1Cx41oxzw== X-Received: by 2002:a37:745:: with SMTP id 66mr2977846qkh.344.1601658405664; Fri, 02 Oct 2020 10:06:45 -0700 (PDT) Received: from localhost.localdomain ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id d10sm1436891qkk.1.2020.10.02.10.06.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Oct 2020 10:06:45 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v2 9/9] dirent: Deprecate getdirentries Date: Fri, 2 Oct 2020 14:06:20 -0300 Message-Id: <20201002170620.1611673-10-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201002170620.1611673-1-adhemerval.zanella@linaro.org> References: <20201002170620.1611673-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-13.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Cc: James Clarke , John Paul Adrian Glaubitz Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" The interface has some issues: 1. It is build on top getdents on Linux and requires handling non-LFS call using LFS getdents. 2. It is not wildly used and the non-LFS support is as problematic as non-LFS readdir. glibc only exports the LFS getdents. 3. It is not a direct replacement over BSD since on some plataform its signature has changed (FreeBSD 11, for instance, used to set the offset as a 'long' and changed to 'off_t' on version 12). The idea is to eventually move the symbols to compat ones. --- NEWS | 3 +++ dirent/dirent.h | 11 +++++++---- sysdeps/unix/sysv/linux/Makefile | 3 +++ 3 files changed, 13 insertions(+), 4 deletions(-) diff --git a/NEWS b/NEWS index ce05d05b16..43dab4d519 100644 --- a/NEWS +++ b/NEWS @@ -33,6 +33,9 @@ Deprecated and removed features, and other changes affecting compatibility: * The mallinfo function is marked deprecated. Callers should call mallinfo2 instead. +* The function getdirentries is now deprecated, applications should use + either getdents64, readdir64 or readdir. + Changes to build and runtime requirements: [Add changes to build and runtime requirements here] diff --git a/dirent/dirent.h b/dirent/dirent.h index 92d0925047..1e93f2fbcf 100644 --- a/dirent/dirent.h +++ b/dirent/dirent.h @@ -348,29 +348,32 @@ extern int alphasort64 (const struct dirent64 **__e1, /* Read directory entries from FD into BUF, reading at most NBYTES. Reading starts at offset *BASEP, and *BASEP is updated with the new position after reading. Returns the number of bytes read; zero when at - end of directory; or -1 for errors. */ + end of directory; or -1 for errors. + This is deprecated and getdents64 or readdir should be used instead. */ # ifndef __USE_FILE_OFFSET64 extern __ssize_t getdirentries (int __fd, char *__restrict __buf, size_t __nbytes, __off_t *__restrict __basep) - __THROW __nonnull ((2, 4)); + __THROW __nonnull ((2, 4)) __attribute_deprecated__; # else # ifdef __REDIRECT extern __ssize_t __REDIRECT_NTH (getdirentries, (int __fd, char *__restrict __buf, size_t __nbytes, __off64_t *__restrict __basep), - getdirentries64) __nonnull ((2, 4)); + getdirentries64) + __THROW __nonnull ((2, 4)) __attribute_deprecated__; # else # define getdirentries getdirentries64 # endif # endif # ifdef __USE_LARGEFILE64 +/* This is deprecated and getdents64 or readdir64 should be used instead. */ extern __ssize_t getdirentries64 (int __fd, char *__restrict __buf, size_t __nbytes, __off64_t *__restrict __basep) - __THROW __nonnull ((2, 4)); + __THROW __nonnull ((2, 4)) __attribute_deprecated__; # endif #endif /* Use misc. */ diff --git a/sysdeps/unix/sysv/linux/Makefile b/sysdeps/unix/sysv/linux/Makefile index 3bd3106ef9..55b8df59c5 100644 --- a/sysdeps/unix/sysv/linux/Makefile +++ b/sysdeps/unix/sysv/linux/Makefile @@ -260,6 +260,9 @@ ifeq ($(subdir),dirent) sysdep_routines += getdirentries getdirentries64 tests += tst-getdents64 tests-internal += tst-readdir64-compat + +# Avoid the warning for the weak_alias for _DIRENT_MATCHES_DIRENT64 +CFLAGS-getdirentries64.c = -Wno-deprecated-declarations endif ifeq ($(subdir),nis)