From patchwork Wed Oct 21 14:15:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella X-Patchwork-Id: 40799 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A5F203953CF3; Wed, 21 Oct 2020 14:15:54 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org A5F203953CF3 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1603289754; bh=2UWYOnNWO1WrdIrUEplNyIb6gfP0wBN8L2vEo63nJ10=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=DnsM/uAJQE+m1GxXyQu4Mv8QalDeE+8k6uOyGmCAcxONXdcnTcX3fkpVEPfrUCHzD ZXDhn+fs2YrTjHmV9pQEVOgcqMGqojs6LfaKngxcFx+luffygRqWrvkiZHIf2/bT8A Dw7LIYoiNaNuk8SzAgrTlu4Q2hRSsu/5t0AQzgNI= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by sourceware.org (Postfix) with ESMTPS id 0FC843953CF3 for ; Wed, 21 Oct 2020 14:15:52 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 0FC843953CF3 Received: by mail-qk1-x744.google.com with SMTP id i22so2551190qkn.9 for ; Wed, 21 Oct 2020 07:15:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2UWYOnNWO1WrdIrUEplNyIb6gfP0wBN8L2vEo63nJ10=; b=jqu5AzC1fVcwyJtrH0IyHy5q+NtjCsNoE1Nik6hEESWXR5l/WAFP0xyXQYZfCw7K0C 8LdQGqZtuUXOnGYh6NvNCTit8oIWwhgEzO/CG/K/Ap3MjXJ6JkUgvRROEJZF9BfaIqcg ikDhzNCvOn1XtT1VQy+DudYWbq/7a+26Ys5jqPPO/R4XZ4klySaziShR0v7yIhgogt0X w0oR7c9jebPmn+cZmJ8oznvwIhrw//1BHAeWq+r4qzqp/MawiWN7HnZUkckLshAWKO22 b8Fn4iJJQkSTGlbMwhwfuLBKi+R+954sHXcAYEpGZGExYDnsOg+0xvU3Ciq/LbL0uS9N gjOg== X-Gm-Message-State: AOAM533jLQ/v+KXxW6KGbzF47d2bfn/3LhI7i7PLfnaas5QbCtIJtj6x quM9daLRnxnBQ/QA9Dl1Ze3yPIfk6MGW6w== X-Google-Smtp-Source: ABdhPJx/6qWgHdLG5gNdX2k/Jedb68eQapX0l8lRxFcERch5XoruaJ7aWOT/iDcvwuXJgxk58IEX/g== X-Received: by 2002:a37:6790:: with SMTP id b138mr3426268qkc.355.1603289751329; Wed, 21 Oct 2020 07:15:51 -0700 (PDT) Received: from localhost.localdomain ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id e17sm1187952qts.54.2020.10.21.07.15.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Oct 2020 07:15:50 -0700 (PDT) To: libc-alpha@sourceware.org, Florian Weimer Subject: [PATCH v3 1/7] linux: Do not skip entries with zero d_ino values [BZ #12165] Date: Wed, 21 Oct 2020 11:15:36 -0300 Message-Id: <20201021141542.2003377-2-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201021141542.2003377-1-adhemerval.zanella@linaro.org> References: <20201021141542.2003377-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-14.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Reply-To: Adhemerval Zanella Cc: James Clarke , John Paul Adrian Glaubitz Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" According to Linux commit 2adc376c55194 (vfs: avoid creation of inode number 0 in get_next_ino) Linux did not treat d_ino == 0 as a special case (it is a valid inode number). This patch fixes readdir{64} by not ignoring entried with d_ino being 0. Checked on x86_64-linux-gnu and i686-linux-gnu. --- sysdeps/unix/sysv/linux/readdir.c | 59 +++++++++++------------------ sysdeps/unix/sysv/linux/readdir64.c | 59 +++++++++++------------------ 2 files changed, 44 insertions(+), 74 deletions(-) diff --git a/sysdeps/unix/sysv/linux/readdir.c b/sysdeps/unix/sysv/linux/readdir.c index ca2a8964e9..1a9820e475 100644 --- a/sysdeps/unix/sysv/linux/readdir.c +++ b/sysdeps/unix/sysv/linux/readdir.c @@ -25,51 +25,36 @@ struct dirent * __readdir_unlocked (DIR *dirp) { - struct dirent *dp; - int saved_errno = errno; + const int saved_errno = errno; - do + if (dirp->offset >= dirp->size) { - size_t reclen; - - if (dirp->offset >= dirp->size) + /* We've emptied out our buffer. Refill it. */ + ssize_t bytes = __getdents (dirp->fd, dirp->data, dirp->allocation); + if (bytes <= 0) { - /* We've emptied out our buffer. Refill it. */ - - size_t maxread = dirp->allocation; - ssize_t bytes; - - bytes = __getdents (dirp->fd, dirp->data, maxread); - if (bytes <= 0) - { - /* On some systems getdents fails with ENOENT when the - open directory has been rmdir'd already. POSIX.1 - requires that we treat this condition like normal EOF. */ - if (bytes < 0 && errno == ENOENT) - bytes = 0; - - /* Don't modifiy errno when reaching EOF. */ - if (bytes == 0) - __set_errno (saved_errno); - dp = NULL; - break; - } - dirp->size = (size_t) bytes; - - /* Reset the offset into the buffer. */ - dirp->offset = 0; + /* On some systems getdents fails with ENOENT when the + open directory has been rmdir'd already. POSIX.1 + requires that we treat this condition like normal EOF. */ + if (bytes < 0 && errno == ENOENT) + bytes = 0; + + /* Don't modifiy errno when reaching EOF. */ + if (bytes == 0) + __set_errno (saved_errno); + return NULL; } + dirp->size = bytes; - dp = (struct dirent *) &dirp->data[dirp->offset]; - - reclen = dp->d_reclen; + /* Reset the offset into the buffer. */ + dirp->offset = 0; + } - dirp->offset += reclen; + struct dirent *dp = (struct dirent *) &dirp->data[dirp->offset]; - dirp->filepos = dp->d_off; + dirp->offset += dp->d_reclen; - /* Skip deleted files. */ - } while (dp->d_ino == 0); + dirp->filepos = dp->d_off; return dp; } diff --git a/sysdeps/unix/sysv/linux/readdir64.c b/sysdeps/unix/sysv/linux/readdir64.c index 1aa6e2664f..d35a4595f6 100644 --- a/sysdeps/unix/sysv/linux/readdir64.c +++ b/sysdeps/unix/sysv/linux/readdir64.c @@ -30,55 +30,40 @@ struct dirent64 * __readdir64 (DIR *dirp) { - struct dirent64 *dp; - int saved_errno = errno; + const int saved_errno = errno; #if IS_IN (libc) __libc_lock_lock (dirp->lock); #endif - do + if (dirp->offset >= dirp->size) { - size_t reclen; - - if (dirp->offset >= dirp->size) + /* We've emptied out our buffer. Refill it. */ + ssize_t bytes = __getdents64 (dirp->fd, dirp->data, dirp->allocation); + if (bytes <= 0) { - /* We've emptied out our buffer. Refill it. */ - - size_t maxread = dirp->allocation; - ssize_t bytes; - - bytes = __getdents64 (dirp->fd, dirp->data, maxread); - if (bytes <= 0) - { - /* On some systems getdents fails with ENOENT when the - open directory has been rmdir'd already. POSIX.1 - requires that we treat this condition like normal EOF. */ - if (bytes < 0 && errno == ENOENT) - bytes = 0; - - /* Don't modifiy errno when reaching EOF. */ - if (bytes == 0) - __set_errno (saved_errno); - dp = NULL; - break; - } - dirp->size = (size_t) bytes; - - /* Reset the offset into the buffer. */ - dirp->offset = 0; + /* On some systems getdents fails with ENOENT when the + open directory has been rmdir'd already. POSIX.1 + requires that we treat this condition like normal EOF. */ + if (bytes < 0 && errno == ENOENT) + bytes = 0; + + /* Don't modifiy errno when reaching EOF. */ + if (bytes == 0) + __set_errno (saved_errno); + return NULL; } + dirp->size = bytes; - dp = (struct dirent64 *) &dirp->data[dirp->offset]; - - reclen = dp->d_reclen; + /* Reset the offset into the buffer. */ + dirp->offset = 0; + } - dirp->offset += reclen; + struct dirent64 *dp = (struct dirent64 *) &dirp->data[dirp->offset]; - dirp->filepos = dp->d_off; + dirp->offset += dp->d_reclen; - /* Skip deleted files. */ - } while (dp->d_ino == 0); + dirp->filepos = dp->d_off; #if IS_IN (libc) __libc_lock_unlock (dirp->lock); From patchwork Wed Oct 21 14:15:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella X-Patchwork-Id: 40800 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 3963B3857022; Wed, 21 Oct 2020 14:15:57 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 3963B3857022 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1603289757; bh=k87iBi3edRIxG+dOjfP7/nZKftIYy42px0Aq4c6T8os=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ijvLpRcd+RfMvL3jiWhkHmEI+VCzd9hdMGgFb4cKd+tnycCfp29kVuj8BCCUzXCZV ijvNDJlh9vmGRQcnHq3MczzWvoTBI4bKntmQp+SDh9QdoQZIddVVrg83zvn4g2Sofv 8cRpzgJkqKgvPE+j7/HTBCRHllYAjiy0R+5QXwmg= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qk1-x741.google.com (mail-qk1-x741.google.com [IPv6:2607:f8b0:4864:20::741]) by sourceware.org (Postfix) with ESMTPS id 442B03857022 for ; Wed, 21 Oct 2020 14:15:54 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 442B03857022 Received: by mail-qk1-x741.google.com with SMTP id v200so2641890qka.0 for ; Wed, 21 Oct 2020 07:15:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=k87iBi3edRIxG+dOjfP7/nZKftIYy42px0Aq4c6T8os=; b=WCMjetbKObJW/yZv78+TEHxiSiNCxoYbhYr5nauSpgRlaZlKe+6fNyEMjbTbX5apvu ykw/e9R47zX3XqP9EF8M7+e74GuTzpcvdYVx00l40nIWeuvk7BoC4+hPYTyyIKC9Z+l5 Y/qA4xbpPo+WAS9e5ployyCo8FaRDEXm+lFHzBDO/hVAW/+lL1Hsy0a+JiQQo2er8xUW ONK2YcaxHYthG3TlAJjpSncewbuR19L0Pd/dvBXtMv/UYnYb4jTsPM+dQvhevM3252Nv wS9H2fOR0EQr6rfzKwujuVKZqbr+Q4Hilnv0jlO8lciPeRJwXveYPUFnqk2kz4J5/cVT r+gw== X-Gm-Message-State: AOAM533WpiX5v3Hgkz0ZoI5V+lInNFzoTdIBNb5gJhsT/za30lhfawGy UMiREVNndNNteDBUBp9kYLHw2v36Rt4Qpw== X-Google-Smtp-Source: ABdhPJxoi00CHxOrXkJO0UfEFPD8nBhGh4e8577ghZQVIEARTobxcfGKinhxmV4TgBNhLqfcAweW8A== X-Received: by 2002:a37:9045:: with SMTP id s66mr3488704qkd.223.1603289753507; Wed, 21 Oct 2020 07:15:53 -0700 (PDT) Received: from localhost.localdomain ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id e17sm1187952qts.54.2020.10.21.07.15.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Oct 2020 07:15:53 -0700 (PDT) To: libc-alpha@sourceware.org, Florian Weimer Subject: [PATCH v3 2/7] linux: Use getdents64 on non-LFS readdir Date: Wed, 21 Oct 2020 11:15:37 -0300 Message-Id: <20201021141542.2003377-3-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201021141542.2003377-1-adhemerval.zanella@linaro.org> References: <20201021141542.2003377-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-14.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Reply-To: Adhemerval Zanella Cc: James Clarke , John Paul Adrian Glaubitz Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" The opendir allocates a translation buffer to be used to return the non-LFS readdir entry. The obtained dirent64 struct is translated to the temporary buffer on each readdir call. Entries that overflow d_off/d_ino and the buffer reallocation failure (in case of large d_name) are ignored. Checked on x86_64-linux-gnu and i686-linux-gnu. --- sysdeps/unix/sysv/linux/closedir.c | 4 ++ sysdeps/unix/sysv/linux/dirstream.h | 5 ++ sysdeps/unix/sysv/linux/opendir.c | 21 +++++++ sysdeps/unix/sysv/linux/readdir.c | 97 +++++++++++++++++++++-------- 4 files changed, 101 insertions(+), 26 deletions(-) diff --git a/sysdeps/unix/sysv/linux/closedir.c b/sysdeps/unix/sysv/linux/closedir.c index ccc19eefcd..d67b590d5f 100644 --- a/sysdeps/unix/sysv/linux/closedir.c +++ b/sysdeps/unix/sysv/linux/closedir.c @@ -47,6 +47,10 @@ __closedir (DIR *dirp) __libc_lock_fini (dirp->lock); #endif +#if !_DIRENT_MATCHES_DIRENT64 + free (dirp->tbuffer); +#endif + free ((void *) dirp); return __close_nocancel (fd); diff --git a/sysdeps/unix/sysv/linux/dirstream.h b/sysdeps/unix/sysv/linux/dirstream.h index a3ea2b7197..da9315d353 100644 --- a/sysdeps/unix/sysv/linux/dirstream.h +++ b/sysdeps/unix/sysv/linux/dirstream.h @@ -41,6 +41,11 @@ struct __dirstream int errcode; /* Delayed error code. */ +#if !defined __OFF_T_MATCHES_OFF64_T || !defined __INO_T_MATCHES_INO64_T + char *tbuffer; /* Translation buffer for non-LFS calls. */ + size_t tbuffer_size; /* Size of translation buffer. */ +#endif + /* Directory block. We must make sure that this block starts at an address that is aligned adequately enough to store dirent entries. Using the alignment of "void *" is not diff --git a/sysdeps/unix/sysv/linux/opendir.c b/sysdeps/unix/sysv/linux/opendir.c index 2198224588..329575e809 100644 --- a/sysdeps/unix/sysv/linux/opendir.c +++ b/sysdeps/unix/sysv/linux/opendir.c @@ -119,6 +119,27 @@ __alloc_dir (int fd, bool close_fd, int flags, const struct stat64 *statp) return NULL; } +#if !_DIRENT_MATCHES_DIRENT64 + /* Allocates a translation buffer to use as the returned 'struct direct' + for non-LFS 'readdir' calls. + + The initial NAME_MAX size should handle most cases, while readdir might + expand the buffer if required. */ + enum + { + tbuffer_size = sizeof (struct dirent) + NAME_MAX + 1 + }; + dirp->tbuffer = malloc (tbuffer_size); + if (dirp->tbuffer == NULL) + { + free (dirp); + if (close_fd) + __close_nocancel_nostatus (fd); + return NULL; + } + dirp->tbuffer_size = tbuffer_size; +#endif + dirp->fd = fd; #if IS_IN (libc) __libc_lock_init (dirp->lock); diff --git a/sysdeps/unix/sysv/linux/readdir.c b/sysdeps/unix/sysv/linux/readdir.c index 1a9820e475..9af4561a8c 100644 --- a/sysdeps/unix/sysv/linux/readdir.c +++ b/sysdeps/unix/sysv/linux/readdir.c @@ -21,42 +21,87 @@ #if !_DIRENT_MATCHES_DIRENT64 #include +/* Translate the DP64 entry to the non-LFS one in the translation buffer + at dirstream DS. Return true is the translation was possible or + false if either an internal fields can be represented in the non-LFS + entry or if the translation can not be resized. */ +static bool +dirstream_entry (struct __dirstream *ds, const struct dirent64 *dp64) +{ + off_t d_off = dp64->d_off; + if (d_off != dp64->d_off) + return false; + ino_t d_ino = dp64->d_ino; + if (d_ino != dp64->d_ino) + return false; + + /* Expand the translation buffer to hold the new name size. */ + size_t new_reclen = sizeof (struct dirent) + + dp64->d_reclen - offsetof (struct dirent64, d_name); + if (new_reclen > ds->tbuffer_size) + { + char *newbuffer = realloc (ds->tbuffer, new_reclen); + if (newbuffer == NULL) + return false; + ds->tbuffer = newbuffer; + ds->tbuffer_size = new_reclen; + } + + struct dirent *dp = (struct dirent *) ds->tbuffer; + + dp->d_off = d_off; + dp->d_ino = d_ino; + dp->d_reclen = new_reclen; + dp->d_type = dp64->d_type; + memcpy (dp->d_name, dp64->d_name, + dp64->d_reclen - offsetof (struct dirent64, d_name)); + + return true; +} + /* Read a directory entry from DIRP. */ struct dirent * __readdir_unlocked (DIR *dirp) { const int saved_errno = errno; - if (dirp->offset >= dirp->size) + while (1) { - /* We've emptied out our buffer. Refill it. */ - ssize_t bytes = __getdents (dirp->fd, dirp->data, dirp->allocation); - if (bytes <= 0) + if (dirp->offset >= dirp->size) { - /* On some systems getdents fails with ENOENT when the - open directory has been rmdir'd already. POSIX.1 - requires that we treat this condition like normal EOF. */ - if (bytes < 0 && errno == ENOENT) - bytes = 0; - - /* Don't modifiy errno when reaching EOF. */ - if (bytes == 0) - __set_errno (saved_errno); - return NULL; + /* We've emptied out our buffer. Refill it. */ + ssize_t bytes = __getdents64 (dirp->fd, dirp->data, + dirp->allocation); + if (bytes <= 0) + { + /* On some systems getdents fails with ENOENT when the + open directory has been rmdir'd already. POSIX.1 + requires that we treat this condition like normal EOF. */ + if (bytes < 0 && errno == ENOENT) + bytes = 0; + + /* Don't modifiy errno when reaching EOF. */ + if (bytes == 0) + __set_errno (saved_errno); + return NULL; + } + dirp->size = bytes; + + /* Reset the offset into the buffer. */ + dirp->offset = 0; + } + + struct dirent64 *dp64 = (struct dirent64 *) &dirp->data[dirp->offset]; + dirp->offset += dp64->d_reclen; + + /* Skip entries which might overflow d_off/d_ino or if the translation + buffer can't be resized. */ + if (dirstream_entry (dirp, dp64)) + { + dirp->filepos = dp64->d_off; + return (struct dirent *) dirp->tbuffer; } - dirp->size = bytes; - - /* Reset the offset into the buffer. */ - dirp->offset = 0; } - - struct dirent *dp = (struct dirent *) &dirp->data[dirp->offset]; - - dirp->offset += dp->d_reclen; - - dirp->filepos = dp->d_off; - - return dp; } struct dirent * From patchwork Wed Oct 21 14:15:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella X-Patchwork-Id: 40801 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 6A1143959E42; Wed, 21 Oct 2020 14:16:00 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 6A1143959E42 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1603289760; bh=KO0LyoRnRBc2eFz+9JJn5JV4QwtIr9orVAwnw4yX/uw=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=vCRhQICBFUVscKzWwgdQvNLSaM1i/jhYof85J8VWyhGdTEIV41Bnt7pZ4sLYhVVnY 1mcR2eDUqSLXAZeduoB6NpRF6nXopdUaeN4rAZAZ1NOiVm6NmvjzB6aygIzSx+nWZ3 5vWTzvfzrOyUP0ljvEVl/dTEEUDyDJ3acLezUYyw= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) by sourceware.org (Postfix) with ESMTPS id E237F3959CB6 for ; Wed, 21 Oct 2020 14:15:56 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org E237F3959CB6 Received: by mail-qt1-x841.google.com with SMTP id c15so2233687qtc.2 for ; Wed, 21 Oct 2020 07:15:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KO0LyoRnRBc2eFz+9JJn5JV4QwtIr9orVAwnw4yX/uw=; b=KjwJfdOGnplKjRVJl4I4m2YboPCFib48znaX+PYkTtjbdJBc7YbbaoIbzoy8YtDg2G i9McI9NAu3l9llppYTUDIYJQMEm1n4HosUO22mYnSsaY3DC6eErL/DNIRmlk/KX5MHdO bTrjDC1qHtJ7qZADqFyGrr3mcTPpBQdMFxWjY6tIF+CKBdHQ6kowE5sGZW2Gxx86a4gL 1M++ZHxi86QFeGq8RxGr7V8celYTIUB5RCk8w41K1/Ewr1kAezjds+S6Rg852f6ThMSZ BkCHgF7GO3M4AYpZGQdwS6W13nkMXaVCKvMowrXNTDJuhSIDdxQ7HILzFFUs/ouSglSg 0N8A== X-Gm-Message-State: AOAM530mk6GMe4XsVPpV5+4VenVZqQqfN+4aV25jEOgaOhjXvaS9ShGY 4j4zYb5PnvXpn9axz3fnA37dNdrqFvuMHw== X-Google-Smtp-Source: ABdhPJwlC2hh+GAhYfgu0fbsWQKpntqq7sWEwR+rgtT4sFzOI/145lBtSfdeGdCSsO8QGmpExCHypg== X-Received: by 2002:ac8:1346:: with SMTP id f6mr3282504qtj.137.1603289755645; Wed, 21 Oct 2020 07:15:55 -0700 (PDT) Received: from localhost.localdomain ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id e17sm1187952qts.54.2020.10.21.07.15.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Oct 2020 07:15:55 -0700 (PDT) To: libc-alpha@sourceware.org, Florian Weimer Subject: [PATCH v3 3/7] linux: Set internal DIR filepos as off64_t [BZ #23960, BZ #24050] Date: Wed, 21 Oct 2020 11:15:38 -0300 Message-Id: <20201021141542.2003377-4-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201021141542.2003377-1-adhemerval.zanella@linaro.org> References: <20201021141542.2003377-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-14.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Reply-To: Adhemerval Zanella Cc: James Clarke , John Paul Adrian Glaubitz Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" It allows to obtain the expected entry offset on telldir and set it correctly on seekdir on platforms where long int is smaller than off64_t. On such cases telldir will mantain an internal list that maps the DIR object off64_t offsets to the returned long int (the function return value). The seekdir will then set the correct offset from the internal list using the telldir as the list key. It also removes the overflow check on readdir and the returned value will be truncated by the non-LFS off_t size. As Joseph has noted in BZ #23960 comment #22, d_off is an opaque value and since telldir/seekdir works regardless of the returned dirent d_off value. Finally it removed the requirement to check for overflow values on telldir (BZ #24050). Checked on x86_64-linux-gnu, i686-linux-gnu, powerpc-linux-gnu, and arm-linux-gnueabihf. --- dirent/Makefile | 2 +- dirent/tst-seekdir2.c | 158 ++++++++++++++++++++++++++++ sysdeps/unix/sysv/linux/closedir.c | 4 + sysdeps/unix/sysv/linux/dirstream.h | 6 +- sysdeps/unix/sysv/linux/opendir.c | 3 + sysdeps/unix/sysv/linux/readdir.c | 1 + sysdeps/unix/sysv/linux/rewinddir.c | 5 + sysdeps/unix/sysv/linux/seekdir.c | 36 ++++++- sysdeps/unix/sysv/linux/telldir.c | 47 ++++++++- sysdeps/unix/sysv/linux/telldir.h | 64 +++++++++++ 10 files changed, 317 insertions(+), 9 deletions(-) create mode 100644 dirent/tst-seekdir2.c create mode 100644 sysdeps/unix/sysv/linux/telldir.h diff --git a/dirent/Makefile b/dirent/Makefile index e917d5ceab..ba9755da64 100644 --- a/dirent/Makefile +++ b/dirent/Makefile @@ -31,7 +31,7 @@ routines := opendir closedir readdir readdir_r rewinddir \ scandir-cancel scandir-tail scandir64-tail tests := list tst-seekdir opendir-tst1 bug-readdir1 tst-fdopendir \ - tst-fdopendir2 tst-scandir tst-scandir64 + tst-fdopendir2 tst-scandir tst-scandir64 tst-seekdir2 CFLAGS-scandir.c += $(uses-callbacks) CFLAGS-scandir64.c += $(uses-callbacks) diff --git a/dirent/tst-seekdir2.c b/dirent/tst-seekdir2.c new file mode 100644 index 0000000000..3e01b361e5 --- /dev/null +++ b/dirent/tst-seekdir2.c @@ -0,0 +1,158 @@ +/* Check multiple telldir and seekdir. + Copyright (C) 2020 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include +#include +#include + +#include +#include +#include + +/* Some filesystems returns a arbitrary value for d_off direnty entry (ext4 + for instance, where the value is an internal hash key). The idea of + create a large number of file is to try trigger a overflow d_off value + in a entry to check if telldir/seekdir does work corretly in such + case. */ +static const char *dirname; +static const size_t nfiles = 10240; + +static void +do_prepare (int argc, char *argv[]) +{ + dirname = support_create_temp_directory ("tst-seekdir2-"); + + for (size_t i = 0; i < nfiles; i++) + { + int fd = create_temp_file_in_dir ("tempfile.", dirname, NULL); + TEST_VERIFY_EXIT (fd > 0); + close (fd); + } +} +#define PREPARE do_prepare + +/* Check for old non Large File Support (LFS). */ +static int +do_test_not_lfs (void) +{ + DIR *dirp = opendir (dirname); + TEST_VERIFY_EXIT (dirp != NULL); + + size_t dirp_count = 0; + for (struct dirent *dp = readdir (dirp); + dp != NULL; + dp = readdir (dirp)) + dirp_count++; + + /* The 2 extra files are '.' and '..'. */ + TEST_COMPARE (dirp_count, nfiles + 2); + + rewinddir (dirp); + + long *tdirp = xmalloc (dirp_count * sizeof (long)); + struct dirent **ddirp = xmalloc (dirp_count * sizeof (struct dirent *)); + + size_t i = 0; + do + { + tdirp[i] = telldir (dirp); + struct dirent *dp = readdir (dirp); + TEST_VERIFY_EXIT (dp != NULL); + ddirp[i] = xmalloc (dp->d_reclen); + memcpy (ddirp[i], dp, dp->d_reclen); + } while (++i < dirp_count); + + for (i = 0; i < dirp_count - 1; i++) + { + seekdir (dirp, tdirp[i]); + struct dirent *dp = readdir (dirp); + TEST_COMPARE (strcmp (dp->d_name, ddirp[i]->d_name), 0); + TEST_COMPARE (dp->d_ino, ddirp[i]->d_ino); + TEST_COMPARE (dp->d_off, ddirp[i]->d_off); + } + + closedir (dirp); + free (tdirp); + for (i = 0; i < dirp_count; i++) + free (ddirp[i]); + free (ddirp); + + return 0; +} + +/* Same as before but with LFS support. */ +static int +do_test_lfs (void) +{ + DIR *dirp = opendir (dirname); + TEST_VERIFY_EXIT (dirp != NULL); + + size_t dirp_count = 0; + for (struct dirent64 * dp = readdir64 (dirp); + dp != NULL; + dp = readdir64 (dirp)) + dirp_count++; + + /* The 2 extra files are '.' and '..'. */ + TEST_COMPARE (dirp_count, nfiles + 2); + + rewinddir (dirp); + + long *tdirp = xmalloc (dirp_count * sizeof (long)); + struct dirent64 **ddirp = xmalloc (dirp_count * sizeof (struct dirent64 *)); + + size_t i = 0; + do + { + tdirp[i] = telldir (dirp); + struct dirent64 *dp = readdir64 (dirp); + TEST_VERIFY_EXIT (dp != NULL); + ddirp[i] = xmalloc (dp->d_reclen); + memcpy (ddirp[i], dp, dp->d_reclen); + } while (++i < dirp_count); + + for (i = 0; i < dirp_count - 1; i++) + { + seekdir (dirp, tdirp[i]); + struct dirent64 *dp = readdir64 (dirp); + TEST_COMPARE (strcmp (dp->d_name, ddirp[i]->d_name), 0); + TEST_COMPARE (dp->d_ino, ddirp[i]->d_ino); + TEST_COMPARE (dp->d_off, ddirp[i]->d_off); + } + + closedir (dirp); + free (tdirp); + for (i = 0; i < dirp_count; i++) + free (ddirp[i]); + free (ddirp); + + return 0; +} + +static int +do_test (void) +{ + do_test_not_lfs (); + do_test_lfs (); + + return 0; +} + +#include diff --git a/sysdeps/unix/sysv/linux/closedir.c b/sysdeps/unix/sysv/linux/closedir.c index d67b590d5f..3436780519 100644 --- a/sysdeps/unix/sysv/linux/closedir.c +++ b/sysdeps/unix/sysv/linux/closedir.c @@ -43,6 +43,10 @@ __closedir (DIR *dirp) fd = dirp->fd; +#ifndef __LP64__ + dirstream_loc_clear (&dirp->locs); +#endif + #if IS_IN (libc) __libc_lock_fini (dirp->lock); #endif diff --git a/sysdeps/unix/sysv/linux/dirstream.h b/sysdeps/unix/sysv/linux/dirstream.h index da9315d353..369d0a697c 100644 --- a/sysdeps/unix/sysv/linux/dirstream.h +++ b/sysdeps/unix/sysv/linux/dirstream.h @@ -21,6 +21,7 @@ #include #include +#include /* Directory stream type. @@ -37,7 +38,7 @@ struct __dirstream size_t size; /* Total valid data in the block. */ size_t offset; /* Current offset into the block. */ - off_t filepos; /* Position of next entry to read. */ + off64_t filepos; /* Position of next entry to read. */ int errcode; /* Delayed error code. */ @@ -45,6 +46,9 @@ struct __dirstream char *tbuffer; /* Translation buffer for non-LFS calls. */ size_t tbuffer_size; /* Size of translation buffer. */ #endif +#ifndef __LP64__ + struct dirstream_loc_t locs; /* off64_t to long int map for telldir. */ +#endif /* Directory block. We must make sure that this block starts at an address that is aligned adequately enough to store diff --git a/sysdeps/unix/sysv/linux/opendir.c b/sysdeps/unix/sysv/linux/opendir.c index 329575e809..28fd55341e 100644 --- a/sysdeps/unix/sysv/linux/opendir.c +++ b/sysdeps/unix/sysv/linux/opendir.c @@ -149,6 +149,9 @@ __alloc_dir (int fd, bool close_fd, int flags, const struct stat64 *statp) dirp->offset = 0; dirp->filepos = 0; dirp->errcode = 0; +#ifndef __LP64__ + dirstream_loc_init (&dirp->locs); +#endif return dirp; } diff --git a/sysdeps/unix/sysv/linux/readdir.c b/sysdeps/unix/sysv/linux/readdir.c index 9af4561a8c..656672c706 100644 --- a/sysdeps/unix/sysv/linux/readdir.c +++ b/sysdeps/unix/sysv/linux/readdir.c @@ -17,6 +17,7 @@ . */ #include +#include #if !_DIRENT_MATCHES_DIRENT64 #include diff --git a/sysdeps/unix/sysv/linux/rewinddir.c b/sysdeps/unix/sysv/linux/rewinddir.c index 860bfda004..8db0d0be4a 100644 --- a/sysdeps/unix/sysv/linux/rewinddir.c +++ b/sysdeps/unix/sysv/linux/rewinddir.c @@ -33,6 +33,11 @@ __rewinddir (DIR *dirp) dirp->offset = 0; dirp->size = 0; dirp->errcode = 0; + +#ifndef __LP64__ + dirstream_loc_clear (&dirp->locs); +#endif + #if IS_IN (libc) __libc_lock_unlock (dirp->lock); #endif diff --git a/sysdeps/unix/sysv/linux/seekdir.c b/sysdeps/unix/sysv/linux/seekdir.c index 3c30520928..5f39ef9eef 100644 --- a/sysdeps/unix/sysv/linux/seekdir.c +++ b/sysdeps/unix/sysv/linux/seekdir.c @@ -22,14 +22,40 @@ #include /* Seek to position POS in DIRP. */ -/* XXX should be __seekdir ? */ void seekdir (DIR *dirp, long int pos) { + off64_t filepos; + __libc_lock_lock (dirp->lock); - (void) __lseek (dirp->fd, pos, SEEK_SET); - dirp->size = 0; - dirp->offset = 0; - dirp->filepos = pos; + +#ifndef __LP64__ + union dirstream_packed dsp; + + dsp.l = pos; + + if (dsp.p.is_packed == 1) + filepos = dsp.p.info; + else + { + size_t index = dsp.p.info; + + if (index >= dirstream_loc_size (&dirp->locs)) + return; + struct dirstream_loc *loc = dirstream_loc_at (&dirp->locs, index); + filepos = loc->filepos; + } +#else + filepos = pos; +#endif + + if (dirp->filepos != filepos) + { + __lseek64 (dirp->fd, filepos, SEEK_SET); + dirp->filepos = filepos; + dirp->offset = 0; + dirp->size = 0; + } + __libc_lock_unlock (dirp->lock); } diff --git a/sysdeps/unix/sysv/linux/telldir.c b/sysdeps/unix/sysv/linux/telldir.c index 57d435ed21..bb33626fa4 100644 --- a/sysdeps/unix/sysv/linux/telldir.c +++ b/sysdeps/unix/sysv/linux/telldir.c @@ -18,16 +18,59 @@ #include #include +#include /* Return the current position of DIRP. */ long int telldir (DIR *dirp) { - long int ret; +#ifndef __LP64__ + /* If the directory position fits in the packet structure returns it. + Otherwise, check if the position is already been recorded in the + dynamic array. If not, add the new record. */ + + union dirstream_packed dsp; + size_t i; __libc_lock_lock (dirp->lock); - ret = dirp->filepos; + + if (dirp->filepos < (1U << 31)) + { + dsp.p.is_packed = 1; + dsp.p.info = dirp->filepos; + goto out; + } + + dsp.l = -1; + + for (i = 0; i < dirstream_loc_size (&dirp->locs); i++) + { + struct dirstream_loc *loc = dirstream_loc_at (&dirp->locs, i); + if (loc->filepos == dirp->filepos) + break; + } + if (i == dirstream_loc_size (&dirp->locs)) + { + dirstream_loc_add (&dirp->locs, + (struct dirstream_loc) { dirp->filepos }); + if (dirstream_loc_has_failed (&dirp->locs)) + goto out; + } + + dsp.p.is_packed = 0; + /* This assignment might overflow, however most likely ENOMEM would happen + long before. */ + dsp.p.info = i; + +out: __libc_lock_unlock (dirp->lock); + return dsp.l; +#else + long int ret; + __libc_lock_lock (dirp->lock); + ret = dirp->filepos; + __libc_lock_unlock (dirp->lock); return ret; +#endif } diff --git a/sysdeps/unix/sysv/linux/telldir.h b/sysdeps/unix/sysv/linux/telldir.h new file mode 100644 index 0000000000..7c45886341 --- /dev/null +++ b/sysdeps/unix/sysv/linux/telldir.h @@ -0,0 +1,64 @@ +/* Linux internal telldir definitions. + Copyright (C) 2020 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef _TELLDIR_H +#define _TELLDIR_H 1 + +#ifndef __LP64__ + +/* On platforms where long int is smaller than off64_t this is how the + returned value is encoded and returned by 'telldir'. If the directory + offset can be enconded in 31 bits it is returned in the 'info' member + with 'is_packed' set to 1. + + Otherwise, the 'info' member describes an index in a dynamic array at + 'DIR' structure. */ + +union dirstream_packed +{ + long int l; + struct + { + unsigned long is_packed:1; + unsigned long info:31; + } p; +}; + +_Static_assert (sizeof (long int) == sizeof (union dirstream_packed), + "sizeof (long int) != sizeof (union dirstream_packed)"); + +/* telldir will mantain a list of offsets that describe the obtained diretory + position if it can fit this information in the returned 'dirstream_packed' + struct. */ + +struct dirstream_loc +{ + off64_t filepos; +}; + +# define DYNARRAY_STRUCT dirstream_loc_t +# define DYNARRAY_ELEMENT struct dirstream_loc +# define DYNARRAY_PREFIX dirstream_loc_ +# include +#else + +_Static_assert (sizeof (long int) == sizeof (off64_t), + "sizeof (long int) != sizeof (off64_t)"); +#endif /* __LP64__ */ + +#endif /* _TELLDIR_H */ From patchwork Wed Oct 21 14:15:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella X-Patchwork-Id: 40802 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id ED9893959E4F; Wed, 21 Oct 2020 14:16:01 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org ED9893959E4F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1603289762; bh=tpvAUaaQtgNtud+2HUXMFWH6HJFVsgmD23z8BGr9hxs=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=kTKmVNvd0eNXBp2baecX4Rxg/JQ+kjdGt8caeQPTFDdZ20uPsruob7B/kwugjXvpa zzYTvMvedG6BjCEM8mssck/7Y4STEx83JPZDq9Y4ovLfWzmtPngklq7UnX/ITZwJm6 EUWMZHWNWsTj3uxeWDE922nD2PN8aIWml5bRWsp4= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com [IPv6:2607:f8b0:4864:20::742]) by sourceware.org (Postfix) with ESMTPS id 109FA3959CB9 for ; Wed, 21 Oct 2020 14:15:59 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 109FA3959CB9 Received: by mail-qk1-x742.google.com with SMTP id s14so2538435qkg.11 for ; Wed, 21 Oct 2020 07:15:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tpvAUaaQtgNtud+2HUXMFWH6HJFVsgmD23z8BGr9hxs=; b=Tk20Pzv1yzTm6fAgaPUpZ0Y/jCPRUDnZRKQr+OzZXkExBejXItX2S1fz2VbCo84oO5 eZY9LUr1U9m5bn4ZFe/Gir+uW49TDHDWq0DtP1Satt+RvQDApZ1EZqC/UUfsarlJbf7Z FOZw2EhSzJmkszhZ9C9gg9C9W7kIXaHvLLLHXsiPiCnoVAMBFtsSeLJjUSJJ7zvvuycx 1lAOtSaqk3+KAzygmOPwHFAdwkMZeaZQ3U8mzIze/AWOi539JMDsn/lfYtlX7GV3lbcL 9OVlZusEnq9hbBSZs9V+AF4NLU2s9t9gSN2rcd6bbdH5ydu/pg4VrUkNtItVW3Gkf1Os taOQ== X-Gm-Message-State: AOAM531qehbTLcPtXVrqqOze40AvorieG4Un0UU62QiGnnOzej8Y+vC1 HXe0H98hl5jzQWQXESD9joBiQ3cIjl7T5g== X-Google-Smtp-Source: ABdhPJzRIo7UxGh73hbdyUaediNx79Ua2Hyn487m68q3pCmPZaeAj9ZRRj82VjlkdwE8k2VIvulFJw== X-Received: by 2002:a05:620a:a06:: with SMTP id i6mr3344043qka.165.1603289758297; Wed, 21 Oct 2020 07:15:58 -0700 (PDT) Received: from localhost.localdomain ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id e17sm1187952qts.54.2020.10.21.07.15.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Oct 2020 07:15:57 -0700 (PDT) To: libc-alpha@sourceware.org, Florian Weimer Subject: [PATCH v3 4/7] linux: Add __readdir64_unlocked Date: Wed, 21 Oct 2020 11:15:39 -0300 Message-Id: <20201021141542.2003377-5-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201021141542.2003377-1-adhemerval.zanella@linaro.org> References: <20201021141542.2003377-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-14.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Reply-To: Adhemerval Zanella Cc: James Clarke , John Paul Adrian Glaubitz Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" And use it on readdir_r implementation. Checked on i686-linux-gnu. --- include/dirent.h | 1 + sysdeps/unix/sysv/linux/readdir64.c | 20 +++++-- sysdeps/unix/sysv/linux/readdir64_r.c | 80 ++++++--------------------- 3 files changed, 33 insertions(+), 68 deletions(-) diff --git a/include/dirent.h b/include/dirent.h index 8325a19e5f..79c8fce969 100644 --- a/include/dirent.h +++ b/include/dirent.h @@ -21,6 +21,7 @@ extern DIR *__fdopendir (int __fd) attribute_hidden; extern int __closedir (DIR *__dirp) attribute_hidden; extern struct dirent *__readdir (DIR *__dirp) attribute_hidden; extern struct dirent *__readdir_unlocked (DIR *__dirp) attribute_hidden; +extern struct dirent64 *__readdir64_unlocked (DIR *__dirp) attribute_hidden; extern struct dirent64 *__readdir64 (DIR *__dirp); libc_hidden_proto (__readdir64) extern int __readdir_r (DIR *__dirp, struct dirent *__entry, diff --git a/sysdeps/unix/sysv/linux/readdir64.c b/sysdeps/unix/sysv/linux/readdir64.c index d35a4595f6..ee41807b97 100644 --- a/sysdeps/unix/sysv/linux/readdir64.c +++ b/sysdeps/unix/sysv/linux/readdir64.c @@ -28,14 +28,10 @@ /* Read a directory entry from DIRP. */ struct dirent64 * -__readdir64 (DIR *dirp) +__readdir64_unlocked (DIR *dirp) { const int saved_errno = errno; -#if IS_IN (libc) - __libc_lock_lock (dirp->lock); -#endif - if (dirp->offset >= dirp->size) { /* We've emptied out our buffer. Refill it. */ @@ -65,6 +61,20 @@ __readdir64 (DIR *dirp) dirp->filepos = dp->d_off; + return dp; +} + +struct dirent64 * +__readdir64 (DIR *dirp) +{ + struct dirent64 *dp; + +#if IS_IN (libc) + __libc_lock_lock (dirp->lock); +#endif + + dp = __readdir64_unlocked (dirp); + #if IS_IN (libc) __libc_lock_unlock (dirp->lock); #endif diff --git a/sysdeps/unix/sysv/linux/readdir64_r.c b/sysdeps/unix/sysv/linux/readdir64_r.c index c587787417..e5ef7be4c2 100644 --- a/sysdeps/unix/sysv/linux/readdir64_r.c +++ b/sysdeps/unix/sysv/linux/readdir64_r.c @@ -32,89 +32,43 @@ __readdir64_r (DIR *dirp, struct dirent64 *entry, struct dirent64 **result) { struct dirent64 *dp; size_t reclen; - const int saved_errno = errno; - int ret; __libc_lock_lock (dirp->lock); - - do + while (1) { - if (dirp->offset >= dirp->size) - { - /* We've emptied out our buffer. Refill it. */ - - size_t maxread = dirp->allocation; - ssize_t bytes; - - maxread = dirp->allocation; - - bytes = __getdents64 (dirp->fd, dirp->data, maxread); - if (bytes <= 0) - { - /* On some systems getdents fails with ENOENT when the - open directory has been rmdir'd already. POSIX.1 - requires that we treat this condition like normal EOF. */ - if (bytes < 0 && errno == ENOENT) - { - bytes = 0; - __set_errno (saved_errno); - } - if (bytes < 0) - dirp->errcode = errno; - - dp = NULL; - break; - } - dirp->size = (size_t) bytes; - - /* Reset the offset into the buffer. */ - dirp->offset = 0; - } - - dp = (struct dirent64 *) &dirp->data[dirp->offset]; + dp = __readdir64_unlocked (dirp); + if (dp == NULL) + break; reclen = dp->d_reclen; + if (reclen <= offsetof (struct dirent64, d_name) + NAME_MAX + 1) + break; - dirp->offset += reclen; - - dirp->filepos = dp->d_off; - - if (reclen > offsetof (struct dirent64, d_name) + NAME_MAX + 1) + /* The record is very long. It could still fit into the caller-supplied + buffer if we can skip padding at the end. */ + size_t namelen = _D_EXACT_NAMLEN (dp); + if (namelen <= NAME_MAX) { - /* The record is very long. It could still fit into the - caller-supplied buffer if we can skip padding at the - end. */ - size_t namelen = _D_EXACT_NAMLEN (dp); - if (namelen <= NAME_MAX) - reclen = offsetof (struct dirent64, d_name) + namelen + 1; - else - { - /* The name is too long. Ignore this file. */ - dirp->errcode = ENAMETOOLONG; - dp->d_ino = 0; - continue; - } + reclen = offsetof (struct dirent64, d_name) + namelen + 1; + break; } - /* Skip deleted and ignored files. */ + /* The name is too long. Ignore this file. */ + dirp->errcode = ENAMETOOLONG; + dp->d_ino = 0; } - while (dp->d_ino == 0); if (dp != NULL) { *result = memcpy (entry, dp, reclen); entry->d_reclen = reclen; - ret = 0; } else - { - *result = NULL; - ret = dirp->errcode; - } + *result = NULL; __libc_lock_unlock (dirp->lock); - return ret; + return dp != NULL ? 0 : dirp->errcode; } From patchwork Wed Oct 21 14:15:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella X-Patchwork-Id: 40803 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 9389A3959E4D; Wed, 21 Oct 2020 14:16:03 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 9389A3959E4D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1603289763; bh=OvaHi4zqVRdwY/J/FajvpE4pV+IZ4YZSnfjvw4ArdSQ=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=W2XjsfHSzH6keEvHBJb+LEY9sJm7Wok5nvIRW+6dqHCABeTK6HOScSYYpNoP/9Wc5 SHC7xfAbU8EB7aqq8Ue8ExD8mGdCYfMN/UTJg0JLKQMGs+KwDY5/EDnfn5aOR/BClS IuqoqePSyc3XzwVroXcrKaha5ZEXDD7SVLXbn6AM= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by sourceware.org (Postfix) with ESMTPS id 3A6FA3959E4B for ; Wed, 21 Oct 2020 14:16:01 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 3A6FA3959E4B Received: by mail-qk1-x744.google.com with SMTP id 140so2618684qko.2 for ; Wed, 21 Oct 2020 07:16:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OvaHi4zqVRdwY/J/FajvpE4pV+IZ4YZSnfjvw4ArdSQ=; b=V98XrfJsQuqKmFOs/GUmZlXpu02ZtdiyLyI4b8fTH1rgkO290Bhd4qbxdbqGnd0tom R/+89ysxZyRZ7I9CrhSGB9Omi+orW/UcMAegOMFTBQaENoZ+n2y+D4y86I3nWLvrjOIZ hZGRprhPcWka1lunQbEswecbELAvip0znTNLmglSL1obTjusvNIol8v9OkuvVoF3FFy5 6Zv0TfQCFqyF8vhZxPGLLmlLxT57x7T0YM5OclsP8StsR562d85WbNqc9VOeIAFY6LQA TrbQcLg7A0SLRrA7EvE9DyhXomsaZOw68nRKVNni5G2PnWHrz6OFQoUbwI1L5cONLcH/ VUOQ== X-Gm-Message-State: AOAM5311fErk1xtmfA3oLBNnErDivzXX5/f/FfxRA01ue20Idv54mBvX GhswBX0gxQwWfGCxCXIpPgOr5TtlZ2kMzw== X-Google-Smtp-Source: ABdhPJy143qNL3l8lM6BSlQXPUOWQESyKhF/grVSR4EQsymSB9Q+L+UZzrW+on2A1qdCjLCMmayqWg== X-Received: by 2002:a37:9f0a:: with SMTP id i10mr3220636qke.61.1603289760480; Wed, 21 Oct 2020 07:16:00 -0700 (PDT) Received: from localhost.localdomain ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id e17sm1187952qts.54.2020.10.21.07.15.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Oct 2020 07:16:00 -0700 (PDT) To: libc-alpha@sourceware.org, Florian Weimer Subject: [PATCH v3 5/7] linux: Add __old_readdir64_unlocked Date: Wed, 21 Oct 2020 11:15:40 -0300 Message-Id: <20201021141542.2003377-6-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201021141542.2003377-1-adhemerval.zanella@linaro.org> References: <20201021141542.2003377-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-14.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Reply-To: Adhemerval Zanella Cc: James Clarke , John Paul Adrian Glaubitz Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" And use it __old_readdir64_r. Checked on i686-linux-gnu. --- sysdeps/unix/sysv/linux/olddirent.h | 2 + sysdeps/unix/sysv/linux/readdir64.c | 21 +++++-- sysdeps/unix/sysv/linux/readdir64_r.c | 79 ++++++--------------------- 3 files changed, 35 insertions(+), 67 deletions(-) diff --git a/sysdeps/unix/sysv/linux/olddirent.h b/sysdeps/unix/sysv/linux/olddirent.h index 2d2559c1e9..b118b0ef96 100644 --- a/sysdeps/unix/sysv/linux/olddirent.h +++ b/sysdeps/unix/sysv/linux/olddirent.h @@ -32,6 +32,8 @@ struct __old_dirent64 /* Now define the internal interfaces. */ extern struct __old_dirent64 *__old_readdir64 (DIR *__dirp); libc_hidden_proto (__old_readdir64); +extern struct __old_dirent64 *__old_readdir64_unlocked (DIR *__dirp) + attribute_hidden; extern int __old_readdir64_r (DIR *__dirp, struct __old_dirent64 *__entry, struct __old_dirent64 **__result); extern __ssize_t __old_getdents64 (int __fd, char *__buf, size_t __nbytes) diff --git a/sysdeps/unix/sysv/linux/readdir64.c b/sysdeps/unix/sysv/linux/readdir64.c index ee41807b97..acd190733c 100644 --- a/sysdeps/unix/sysv/linux/readdir64.c +++ b/sysdeps/unix/sysv/linux/readdir64.c @@ -101,15 +101,11 @@ versioned_symbol (libc, __readdir64, readdir64, GLIBC_2_2); attribute_compat_text_section struct __old_dirent64 * -__old_readdir64 (DIR *dirp) +__old_readdir64_unlocked (DIR *dirp) { struct __old_dirent64 *dp; int saved_errno = errno; -#if IS_IN (libc) - __libc_lock_lock (dirp->lock); -#endif - do { size_t reclen; @@ -153,6 +149,21 @@ __old_readdir64 (DIR *dirp) /* Skip deleted files. */ } while (dp->d_ino == 0); + return dp; +} + +attribute_compat_text_section +struct __old_dirent64 * +__old_readdir64 (DIR *dirp) +{ + struct __old_dirent64 *dp; + +#if IS_IN (libc) + __libc_lock_lock (dirp->lock); +#endif + + dp = __old_readdir64_unlocked (dirp); + #if IS_IN (libc) __libc_lock_unlock (dirp->lock); #endif diff --git a/sysdeps/unix/sysv/linux/readdir64_r.c b/sysdeps/unix/sysv/linux/readdir64_r.c index e5ef7be4c2..133c534a69 100644 --- a/sysdeps/unix/sysv/linux/readdir64_r.c +++ b/sysdeps/unix/sysv/linux/readdir64_r.c @@ -91,89 +91,44 @@ __old_readdir64_r (DIR *dirp, struct __old_dirent64 *entry, { struct __old_dirent64 *dp; size_t reclen; - const int saved_errno = errno; - int ret; __libc_lock_lock (dirp->lock); - do + while (1) { - if (dirp->offset >= dirp->size) - { - /* We've emptied out our buffer. Refill it. */ - - size_t maxread = dirp->allocation; - ssize_t bytes; - - maxread = dirp->allocation; - - bytes = __old_getdents64 (dirp->fd, dirp->data, maxread); - if (bytes <= 0) - { - /* On some systems getdents fails with ENOENT when the - open directory has been rmdir'd already. POSIX.1 - requires that we treat this condition like normal EOF. */ - if (bytes < 0 && errno == ENOENT) - { - bytes = 0; - __set_errno (saved_errno); - } - if (bytes < 0) - dirp->errcode = errno; - - dp = NULL; - break; - } - dirp->size = (size_t) bytes; - - /* Reset the offset into the buffer. */ - dirp->offset = 0; - } - - dp = (struct __old_dirent64 *) &dirp->data[dirp->offset]; + dp = __old_readdir64_unlocked (dirp); + if (dp == NULL) + break; reclen = dp->d_reclen; + if (reclen <= offsetof (struct __old_dirent64, d_name) + NAME_MAX + 1) + break; - dirp->offset += reclen; - - dirp->filepos = dp->d_off; - - if (reclen > offsetof (struct __old_dirent64, d_name) + NAME_MAX + 1) + /* The record is very long. It could still fit into the caller-supplied + buffer if we can skip padding at the end. */ + size_t namelen = _D_EXACT_NAMLEN (dp); + if (namelen <= NAME_MAX) { - /* The record is very long. It could still fit into the - caller-supplied buffer if we can skip padding at the - end. */ - size_t namelen = _D_EXACT_NAMLEN (dp); - if (namelen <= NAME_MAX) - reclen = offsetof (struct __old_dirent64, d_name) + namelen + 1; - else - { - /* The name is too long. Ignore this file. */ - dirp->errcode = ENAMETOOLONG; - dp->d_ino = 0; - continue; - } + reclen = offsetof (struct dirent64, d_name) + namelen + 1; + break; } - /* Skip deleted and ignored files. */ + /* The name is too long. Ignore this file. */ + dirp->errcode = ENAMETOOLONG; + dp->d_ino = 0; } - while (dp->d_ino == 0); if (dp != NULL) { *result = memcpy (entry, dp, reclen); entry->d_reclen = reclen; - ret = 0; } else - { - *result = NULL; - ret = dirp->errcode; - } + *result = NULL; __libc_lock_unlock (dirp->lock); - return ret; + return dp != NULL ? 0 : dirp->errcode; } compat_symbol (libc, __old_readdir64_r, readdir64_r, GLIBC_2_1); From patchwork Wed Oct 21 14:15:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella X-Patchwork-Id: 40804 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 990113959E6D; Wed, 21 Oct 2020 14:16:06 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 990113959E6D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1603289766; bh=Afx4r27ZhBdDfqA5xdcHNZulmcYtfxVVB1wONBAKKSE=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=rkWXbCa2P19ynfUi/uZrxXQY6GZvZpzF9xu9yaAGGhgsQL6qhMQtIGGkZ1I4HYNj/ VI2qfmZvlYDo1CXscTCpEMiOeDJpI8NKOiiS5+kRj72YOnsAx6/M0y8FYKCnEX3xdn AKJIWJJcZsOrq/s4A3WnrYiqj0Dq/6VssNWgGu7Q= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by sourceware.org (Postfix) with ESMTPS id BB2373959E59 for ; Wed, 21 Oct 2020 14:16:03 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org BB2373959E59 Received: by mail-qt1-x842.google.com with SMTP id r8so2145699qtp.13 for ; Wed, 21 Oct 2020 07:16:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Afx4r27ZhBdDfqA5xdcHNZulmcYtfxVVB1wONBAKKSE=; b=JglqZNtabUsY3/6+m1T1uDsbCGKaFcG7/fiQnr3hK927au23Wjzh1pnooKaDKlDx4Z hunLxRYfuyG0xeEJyDMnDS1UCKmDj2YL7EcnwVX2L1pee4bJwPQ3QPoC1m6ClfSmN3JL nFyDtirqYE7A4Kx9wbxfdY9n0dzyIDLtTtGup+NyXuY2nLOoUbqF73HK6T2J3deHcE2n BwCrv4HNgO/TUMgG5n+cXKR5tKX0Q5TawAr0/humBD/dTr5VF+499TAwkw4+CtcL0iiw h3cPZafSpnT5PzA0PucLZJpyygTXJLNVBUkWJjagbiLNnDuq2roy1xqC6Fiq765J+Ubw hQ+w== X-Gm-Message-State: AOAM5316ZIJqYUlqfBBeDNQ/EjIbxKq0+sCqtJkAk+DeFUUBtcx8f1wz Peo2hW1ug7zKshVMZBjz4Krui+yqhU6/2w== X-Google-Smtp-Source: ABdhPJw9Jb7IRJfyvxmxgq8jl70WY33WVs/mJjxxbVGEFSDtE/3oOVEe1hKoyVUzrns3l9zkYVtf/w== X-Received: by 2002:ac8:36ca:: with SMTP id b10mr3242825qtc.135.1603289762913; Wed, 21 Oct 2020 07:16:02 -0700 (PDT) Received: from localhost.localdomain ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id e17sm1187952qts.54.2020.10.21.07.16.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Oct 2020 07:16:02 -0700 (PDT) To: libc-alpha@sourceware.org, Florian Weimer Subject: [PATCH v3 6/7] linux: Use getdents64 on readdir64 compat implementation Date: Wed, 21 Oct 2020 11:15:41 -0300 Message-Id: <20201021141542.2003377-7-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201021141542.2003377-1-adhemerval.zanella@linaro.org> References: <20201021141542.2003377-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-14.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Reply-To: Adhemerval Zanella Cc: James Clarke , John Paul Adrian Glaubitz Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" It uses a similar strategy from the non-LFS readdir that also uses getdents64 internally and uses a translation buffer to return the compat readdir64 entry. It allows to remove __old_getdents64. Checked on i686-linux-gnu. --- sysdeps/unix/sysv/linux/getdents64.c | 93 ------------------------ sysdeps/unix/sysv/linux/olddirent.h | 2 - sysdeps/unix/sysv/linux/opendir.c | 15 +++- sysdeps/unix/sysv/linux/readdir64.c | 104 +++++++++++++++++---------- 4 files changed, 79 insertions(+), 135 deletions(-) diff --git a/sysdeps/unix/sysv/linux/getdents64.c b/sysdeps/unix/sysv/linux/getdents64.c index 75892c2823..b8682c3472 100644 --- a/sysdeps/unix/sysv/linux/getdents64.c +++ b/sysdeps/unix/sysv/linux/getdents64.c @@ -36,97 +36,4 @@ weak_alias (__getdents64, getdents64) #if _DIRENT_MATCHES_DIRENT64 strong_alias (__getdents64, __getdents) -#else -# include - -# if SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2) -# include -# include - -static ssize_t -handle_overflow (int fd, __off64_t offset, ssize_t count) -{ - /* If this is the first entry in the buffer, we can report the - error. */ - if (offset == 0) - { - __set_errno (EOVERFLOW); - return -1; - } - - /* Otherwise, seek to the overflowing entry, so that the next call - will report the error, and return the data read so far. */ - if (__lseek64 (fd, offset, SEEK_SET) != 0) - return -1; - return count; -} - -ssize_t -__old_getdents64 (int fd, char *buf, size_t nbytes) -{ - /* We do not move the individual directory entries. This is only - possible if the target type (struct __old_dirent64) is smaller - than the source type. */ - _Static_assert (offsetof (struct __old_dirent64, d_name) - <= offsetof (struct dirent64, d_name), - "__old_dirent64 is larger than dirent64"); - _Static_assert (__alignof__ (struct __old_dirent64) - <= __alignof__ (struct dirent64), - "alignment of __old_dirent64 is larger than dirent64"); - - ssize_t retval = INLINE_SYSCALL_CALL (getdents64, fd, buf, nbytes); - if (retval > 0) - { - /* This is the marker for the first entry. Offset 0 is reserved - for the first entry (see rewinddir). Here, we use it as a - marker for the first entry in the buffer. We never actually - seek to offset 0 because handle_overflow reports the error - directly, so it does not matter that the offset is incorrect - if entries have been read from the descriptor before (so that - the descriptor is not actually at offset 0). */ - __off64_t previous_offset = 0; - - char *p = buf; - char *end = buf + retval; - while (p < end) - { - struct dirent64 *source = (struct dirent64 *) p; - - /* Copy out the fixed-size data. */ - __ino_t ino = source->d_ino; - __off64_t offset = source->d_off; - unsigned int reclen = source->d_reclen; - unsigned char type = source->d_type; - - /* Check for ino_t overflow. */ - if (__glibc_unlikely (ino != source->d_ino)) - return handle_overflow (fd, previous_offset, p - buf); - - /* Convert to the target layout. Use a separate struct and - memcpy to side-step aliasing issues. */ - struct __old_dirent64 result; - result.d_ino = ino; - result.d_off = offset; - result.d_reclen = reclen; - result.d_type = type; - - /* Write the fixed-sized part of the result to the - buffer. */ - size_t result_name_offset = offsetof (struct __old_dirent64, d_name); - memcpy (p, &result, result_name_offset); - - /* Adjust the position of the name if necessary. Copy - everything until the end of the record, including the - terminating NUL byte. */ - if (result_name_offset != offsetof (struct dirent64, d_name)) - memmove (p + result_name_offset, source->d_name, - reclen - offsetof (struct dirent64, d_name)); - - p += reclen; - previous_offset = offset; - } - } - return retval; -} -# endif /* SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2) */ #endif /* _DIRENT_MATCHES_DIRENT64 */ diff --git a/sysdeps/unix/sysv/linux/olddirent.h b/sysdeps/unix/sysv/linux/olddirent.h index b118b0ef96..63d8e97868 100644 --- a/sysdeps/unix/sysv/linux/olddirent.h +++ b/sysdeps/unix/sysv/linux/olddirent.h @@ -36,8 +36,6 @@ extern struct __old_dirent64 *__old_readdir64_unlocked (DIR *__dirp) attribute_hidden; extern int __old_readdir64_r (DIR *__dirp, struct __old_dirent64 *__entry, struct __old_dirent64 **__result); -extern __ssize_t __old_getdents64 (int __fd, char *__buf, size_t __nbytes) - attribute_hidden; int __old_scandir64 (const char * __dir, struct __old_dirent64 *** __namelist, int (*__selector) (const struct __old_dirent64 *), diff --git a/sysdeps/unix/sysv/linux/opendir.c b/sysdeps/unix/sysv/linux/opendir.c index 28fd55341e..3a08b76c0a 100644 --- a/sysdeps/unix/sysv/linux/opendir.c +++ b/sysdeps/unix/sysv/linux/opendir.c @@ -23,6 +23,11 @@ #include +#include +#if SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2) +# include +#endif + enum { opendir_oflags = O_RDONLY|O_NDELAY|O_DIRECTORY|O_LARGEFILE|O_CLOEXEC }; @@ -127,7 +132,15 @@ __alloc_dir (int fd, bool close_fd, int flags, const struct stat64 *statp) expand the buffer if required. */ enum { - tbuffer_size = sizeof (struct dirent) + NAME_MAX + 1 + tbuffer_size = +# if SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2) + /* This is used on compat readdir64. */ + MAX (sizeof (struct dirent), + sizeof (struct __old_dirent64)) +# else + sizeof (struct dirent) +# endif + + NAME_MAX + 1 }; dirp->tbuffer = malloc (tbuffer_size); if (dirp->tbuffer == NULL) diff --git a/sysdeps/unix/sysv/linux/readdir64.c b/sysdeps/unix/sysv/linux/readdir64.c index acd190733c..799708174a 100644 --- a/sysdeps/unix/sysv/linux/readdir64.c +++ b/sysdeps/unix/sysv/linux/readdir64.c @@ -99,57 +99,83 @@ versioned_symbol (libc, __readdir64, readdir64, GLIBC_2_2); # if SHLIB_COMPAT(libc, GLIBC_2_1, GLIBC_2_2) # include +/* Translate the DP64 entry to the old LFS one in the translation buffer + at dirstream DS. Return true is the translation was possible or + false if either an internal fields can be represented in the non-LFS + entry or if the translation can not be resized. */ +static bool +dirstream_old_entry (struct __dirstream *ds, const struct dirent64 *dp64) +{ + /* Check for overflow. */ + ino_t d_ino = dp64->d_ino; + if (d_ino != dp64->d_ino) + return false; + + /* Expand the translation buffer to hold the new namesize. */ + size_t d_reclen = sizeof (struct __old_dirent64) + + dp64->d_reclen - offsetof (struct dirent64, d_name); + if (d_reclen > ds->tbuffer_size) + { + char *newbuffer = realloc (ds->tbuffer, d_reclen); + if (newbuffer == NULL) + return false; + ds->tbuffer = newbuffer; + ds->tbuffer_size = d_reclen; + } + + struct __old_dirent64 *olddp64 = (struct __old_dirent64 *) ds->tbuffer; + + olddp64->d_off = dp64->d_off; + olddp64->d_ino = dp64->d_ino; + olddp64->d_reclen = dp64->d_reclen; + olddp64->d_type = dp64->d_type; + memcpy (olddp64->d_name, dp64->d_name, + dp64->d_reclen - offsetof (struct dirent64, d_name)); + + return true; +} + attribute_compat_text_section struct __old_dirent64 * __old_readdir64_unlocked (DIR *dirp) { - struct __old_dirent64 *dp; - int saved_errno = errno; + const int saved_errno = errno; - do + if (dirp->offset >= dirp->size) { - size_t reclen; - - if (dirp->offset >= dirp->size) + /* We've emptied out our buffer. Refill it. */ + ssize_t bytes = __getdents64 (dirp->fd, dirp->data, dirp->allocation); + if (bytes <= 0) { - /* We've emptied out our buffer. Refill it. */ - - size_t maxread = dirp->allocation; - ssize_t bytes; - - bytes = __old_getdents64 (dirp->fd, dirp->data, maxread); - if (bytes <= 0) - { - /* On some systems getdents fails with ENOENT when the - open directory has been rmdir'd already. POSIX.1 - requires that we treat this condition like normal EOF. */ - if (bytes < 0 && errno == ENOENT) - bytes = 0; - - /* Don't modifiy errno when reaching EOF. */ - if (bytes == 0) - __set_errno (saved_errno); - dp = NULL; - break; - } - dirp->size = (size_t) bytes; - - /* Reset the offset into the buffer. */ - dirp->offset = 0; - } - - dp = (struct __old_dirent64 *) &dirp->data[dirp->offset]; + /* On some systems getdents fails with ENOENT when the + open directory has been rmdir'd already. POSIX.1 + requires that we treat this condition like normal EOF. */ + if (bytes < 0 && errno == ENOENT) + bytes = 0; - reclen = dp->d_reclen; + /* Don't modifiy errno when reaching EOF. */ + if (bytes == 0) + __set_errno (saved_errno); + return NULL; + } + dirp->size = bytes; - dirp->offset += reclen; + /* Reset the offset into the buffer. */ + dirp->offset = 0; + } - dirp->filepos = dp->d_off; + struct dirent64 *dp64 = (struct dirent64 *) &dirp->data[dirp->offset]; + dirp->offset += dp64->d_reclen; - /* Skip deleted files. */ - } while (dp->d_ino == 0); + /* Skip entries which might overflow d_ino or for memory allocation failure + in case of large file names. */ + if (dirstream_old_entry (dirp, dp64)) + { + dirp->filepos = dp64->d_off; + return (struct __old_dirent64 *) dirp->tbuffer; + } - return dp; + return NULL; } attribute_compat_text_section From patchwork Wed Oct 21 14:15:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella X-Patchwork-Id: 40805 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 2A3323959CB7; Wed, 21 Oct 2020 14:16:13 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 2A3323959CB7 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1603289773; bh=KE1CoT3rnYOZe59ux/O/6ANspSJGMetPj7G/N6EnyFs=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ElAmQzwTPVUvrztVHr8PaG734EQY6AfBYz2KWpq4ER0Om8usDimzLjtSR9FXOW4qe VWqo4cclCZemySdWFZVhsjkkr3Ul0EHj4SdJ1TdkvjtiSg4/qTLvqQ0kxHGHhfkW0c 47SjfkpCyaJ3IEyx8asW1ZaJTL9diFh30mTA+usw= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) by sourceware.org (Postfix) with ESMTPS id E54773959E47 for ; Wed, 21 Oct 2020 14:16:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org E54773959E47 Received: by mail-qk1-x72e.google.com with SMTP id s14so2538915qkg.11 for ; Wed, 21 Oct 2020 07:16:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KE1CoT3rnYOZe59ux/O/6ANspSJGMetPj7G/N6EnyFs=; b=SHnu8Xy2dn60Q9Pl4UyJIVqfo5ykLXeed1OG4xQ9AX7YNiXXx2Kd/aoRGMPDSH8cdA ZJ3WdEVj5XgJuG9FnTy6z5ga6/BptThJ2whw3Iz05mKsXhSmCVTFWxViS83Ul2NqMinN /mUCGbOYaVslmKnPXOSBCoauahWe2zsL6bpaF2IA2UrKRvns6WX1MIO6L3tf8pUQk5wp cZOBYkY02KubzIBPyxrxHhhSURgxeYwcTUpj9wnte0yBzVpVGywhD3hnL1Sq0znRgNwa 8U3/gM9FKXckB2IW2OWs6Jv5/FuAj5pACrQMq+kA8POzG0MOkVnZbw4EZUIIyn7n9S/T cwYw== X-Gm-Message-State: AOAM531tFY7QaLuy+INr2XuAZ/hfQaq0Vp8ZXXqpmnSkvlT6y3Nww5VM fHcVuJeJo4TNi6nWZNzUYeSQdMEfZyLkdQ== X-Google-Smtp-Source: ABdhPJxoOAhh5LY3uCRaKdPMHjCGijtN7kLqklfNGftyKQY+bcUJDplSngxxnR5JpokJE68DExcUyg== X-Received: by 2002:a37:6790:: with SMTP id b138mr3427281qkc.355.1603289765218; Wed, 21 Oct 2020 07:16:05 -0700 (PDT) Received: from localhost.localdomain ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id e17sm1187952qts.54.2020.10.21.07.16.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Oct 2020 07:16:04 -0700 (PDT) To: libc-alpha@sourceware.org, Florian Weimer Subject: [PATCH v3 7/7] dirent: Deprecate getdirentries Date: Wed, 21 Oct 2020 11:15:42 -0300 Message-Id: <20201021141542.2003377-8-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201021141542.2003377-1-adhemerval.zanella@linaro.org> References: <20201021141542.2003377-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-14.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Reply-To: Adhemerval Zanella Cc: James Clarke , John Paul Adrian Glaubitz Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" The interface has some issues: 1. It is build on top getdents on Linux and requires handling non-LFS call using LFS getdents. 2. It is not wildly used and the non-LFS support is as problematic as non-LFS readdir. glibc only exports the LFS getdents. 3. It is not a direct replacement over BSD since on some plataform its signature has changed (FreeBSD 11, for instance, used to set the offset as a 'long' and changed to 'off_t' on version 12). The idea is to eventually move the symbols to compat ones. --- NEWS | 3 +++ dirent/dirent.h | 14 ++++++++++---- sysdeps/unix/sysv/linux/Makefile | 3 +++ 3 files changed, 16 insertions(+), 4 deletions(-) diff --git a/NEWS b/NEWS index 3c1e509744..63623dcee2 100644 --- a/NEWS +++ b/NEWS @@ -47,6 +47,9 @@ Deprecated and removed features, and other changes affecting compatibility: as a compatibility symbol. Applications should use the getrlimit or prlimit. +* The function getdirentries is now deprecated, applications should use + either getdents64, readdir64 or readdir. + Changes to build and runtime requirements: * On Linux, the system administrator needs to configure /dev/pts with diff --git a/dirent/dirent.h b/dirent/dirent.h index 92d0925047..62bb77b4e1 100644 --- a/dirent/dirent.h +++ b/dirent/dirent.h @@ -348,29 +348,35 @@ extern int alphasort64 (const struct dirent64 **__e1, /* Read directory entries from FD into BUF, reading at most NBYTES. Reading starts at offset *BASEP, and *BASEP is updated with the new position after reading. Returns the number of bytes read; zero when at - end of directory; or -1 for errors. */ + end of directory; or -1 for errors. + This is deprecated and getdents64 or readdir should be used instead. */ # ifndef __USE_FILE_OFFSET64 extern __ssize_t getdirentries (int __fd, char *__restrict __buf, size_t __nbytes, __off_t *__restrict __basep) - __THROW __nonnull ((2, 4)); + __THROW __nonnull ((2, 4)) + __attribute_deprecated_msg__ ("Use getdents64 instead"); # else # ifdef __REDIRECT extern __ssize_t __REDIRECT_NTH (getdirentries, (int __fd, char *__restrict __buf, size_t __nbytes, __off64_t *__restrict __basep), - getdirentries64) __nonnull ((2, 4)); + getdirentries64) + __THROW __nonnull ((2, 4)) + __attribute_deprecated_msg__ ("Use getdents64 instead"); # else # define getdirentries getdirentries64 # endif # endif # ifdef __USE_LARGEFILE64 +/* This is deprecated and getdents64 or readdir64 should be used instead. */ extern __ssize_t getdirentries64 (int __fd, char *__restrict __buf, size_t __nbytes, __off64_t *__restrict __basep) - __THROW __nonnull ((2, 4)); + __THROW __nonnull ((2, 4)) + __attribute_deprecated_msg__ ("Use getdents64 instead"); # endif #endif /* Use misc. */ diff --git a/sysdeps/unix/sysv/linux/Makefile b/sysdeps/unix/sysv/linux/Makefile index 09604e128b..242b07734f 100644 --- a/sysdeps/unix/sysv/linux/Makefile +++ b/sysdeps/unix/sysv/linux/Makefile @@ -262,6 +262,9 @@ ifeq ($(subdir),dirent) sysdep_routines += getdirentries getdirentries64 tests += tst-getdents64 tests-internal += tst-readdir64-compat + +# Avoid the warning for the weak_alias for _DIRENT_MATCHES_DIRENT64 +CFLAGS-getdirentries64.c = -Wno-deprecated-declarations endif ifeq ($(subdir),nis)