Message ID | 20201021141542.2003377-1-adhemerval.zanella@linaro.org |
---|---|
Headers |
Return-Path: <libc-alpha-bounces@sourceware.org> X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 232913953D23; Wed, 21 Oct 2020 14:15:52 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 232913953D23 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1603289752; bh=NP3SR8QK+T7ERgmtmd/L3IHEOlWcB8R1Ls9ExZiF07Q=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=vuxdvgfk7LkbBZtaT87WARkL7G9wKxPR79UuuWFuOvzJg5GUZ3aWd1JRRQwyDTRqG s/dwYM/t+7WMQuQTzVlsqdENW2qwjKnvFfWb7gPYbKKYnBEkNcxeN3Hwa0k6ok1FBN bIK0slpWLod3l/YuAZaTea/bDVFws7tFSsiwkbbQ= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qv1-xf41.google.com (mail-qv1-xf41.google.com [IPv6:2607:f8b0:4864:20::f41]) by sourceware.org (Postfix) with ESMTPS id D61973857022 for <libc-alpha@sourceware.org>; Wed, 21 Oct 2020 14:15:49 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org D61973857022 Received: by mail-qv1-xf41.google.com with SMTP id h11so1169488qvq.7 for <libc-alpha@sourceware.org>; Wed, 21 Oct 2020 07:15:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=NP3SR8QK+T7ERgmtmd/L3IHEOlWcB8R1Ls9ExZiF07Q=; b=Doi2h22YHjcnynF2cBaukvBVU8wohU5p37SV8MiRWjrtzxokzc2BvEIJLBI5Gt/w/H jNbW0ECXxJLc9yT3ycqdieVtmpviUeO/aduGxu5Eh0Mxx6FLpQanmzemO7mAgFLvL11s OrSi3/YIFxitlQQ9fJpP02UAoLkERo1PGqGDEevQzAlenfvUD4sEOsvxf84PNJ6H0iZG UxC/OIc5fsC96d1p/zMiZSMf6omlVv4pfOl9v0I+IDzVPXk2PhnlsK2FB42kPDjKSwrQ VRk2P/kOHOmmNgGi2XGbbGzEFNhoQI/fGSR6BQ6Abq3UmmWreN4o+OmWBxwxXeR75MHn tzBg== X-Gm-Message-State: AOAM531X+/77Ytb4q+xJI12vhZLYrqX3hcPpTEBwA4hINyozOIO0IGha lHja09mN5gcqqGjAuBIZs5ifFNsPY+g3gQ== X-Google-Smtp-Source: ABdhPJychY1Sv8d3S2hWPempvFFikb4RzJ8ADcwHogyRCxXD+KYEx9wx9+vskwJ+cSG0V/xnoM89PQ== X-Received: by 2002:a0c:ea2a:: with SMTP id t10mr3105377qvp.37.1603289749197; Wed, 21 Oct 2020 07:15:49 -0700 (PDT) Received: from localhost.localdomain ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id e17sm1187952qts.54.2020.10.21.07.15.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Oct 2020 07:15:48 -0700 (PDT) To: libc-alpha@sourceware.org, Florian Weimer <fweimer@redhat.com> Subject: [PATCH v3 0/7] Fix getdents{64} regression on some FS Date: Wed, 21 Oct 2020 11:15:35 -0300 Message-Id: <20201021141542.2003377-1-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-8.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list <libc-alpha.sourceware.org> List-Unsubscribe: <https://sourceware.org/mailman/options/libc-alpha>, <mailto:libc-alpha-request@sourceware.org?subject=unsubscribe> List-Archive: <https://sourceware.org/pipermail/libc-alpha/> List-Post: <mailto:libc-alpha@sourceware.org> List-Help: <mailto:libc-alpha-request@sourceware.org?subject=help> List-Subscribe: <https://sourceware.org/mailman/listinfo/libc-alpha>, <mailto:libc-alpha-request@sourceware.org?subject=subscribe> From: Adhemerval Zanella via Libc-alpha <libc-alpha@sourceware.org> Reply-To: Adhemerval Zanella <adhemerval.zanella@linaro.org> Cc: James Clarke <jrtc27@debian.org>, John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" <libc-alpha-bounces@sourceware.org> |
Series |
Fix getdents{64} regression on some FS
|
|
Message
Adhemerval Zanella Netto
Oct. 21, 2020, 2:15 p.m. UTC
This is updated version [1], rebased against master add with some suggestion from previous review. The main changes are: - The translation buffer is allocated as an auxiliarry one (instead of a reserved spacefrom the dirstream) and readdir might reallocate it if it finds a large entry. - Failures in non-LFS readdir (such as d_ino/d_off overflow or failure in expand the translation buffer) is ignored. This allows code that does not handle possible failure in readdir to successful read the rest of entries in the directory. - So minor fixes in dirent/tst-seekdir2.c - Added __attribute_deprecated_msg__ on getdirentries to redirect to getdents64. I have checked on x86_64-linux-gnu, powerpc64-linux-gnu and on some 32-bit ABIs arm-linux-gnueabihf, powerpc-linux-gnu, and sparcv9-linux-gnu. [1] https://sourceware.org/pipermail/libc-alpha/2020-October/118274.html Adhemerval Zanella (7): linux: Do not skip entries with zero d_ino values [BZ #12165] linux: Use getdents64 on non-LFS readdir linux: Set internal DIR filepos as off64_t [BZ #23960, BZ #24050] linux: Add __readdir64_unlocked linux: Add __old_readdir64_unlocked linux: Use getdents64 on readdir64 compat implementation dirent: Deprecate getdirentries NEWS | 3 + dirent/Makefile | 2 +- dirent/dirent.h | 14 +- dirent/tst-seekdir2.c | 158 +++++++++++++++++++++ include/dirent.h | 1 + sysdeps/unix/sysv/linux/Makefile | 3 + sysdeps/unix/sysv/linux/closedir.c | 8 ++ sysdeps/unix/sysv/linux/dirstream.h | 11 +- sysdeps/unix/sysv/linux/getdents64.c | 93 ------------ sysdeps/unix/sysv/linux/olddirent.h | 4 +- sysdeps/unix/sysv/linux/opendir.c | 37 +++++ sysdeps/unix/sysv/linux/readdir.c | 83 +++++++---- sysdeps/unix/sysv/linux/readdir64.c | 196 +++++++++++++++----------- sysdeps/unix/sysv/linux/readdir64_r.c | 159 +++++---------------- sysdeps/unix/sysv/linux/rewinddir.c | 5 + sysdeps/unix/sysv/linux/seekdir.c | 36 ++++- sysdeps/unix/sysv/linux/telldir.c | 47 +++++- sysdeps/unix/sysv/linux/telldir.h | 64 +++++++++ 18 files changed, 583 insertions(+), 341 deletions(-) create mode 100644 dirent/tst-seekdir2.c create mode 100644 sysdeps/unix/sysv/linux/telldir.h
Comments
Hi! On 10/21/20 4:15 PM, Adhemerval Zanella wrote: > This is updated version [1], rebased against master add with some > suggestion from previous review. The main changes are: > > - The translation buffer is allocated as an auxiliarry one (instead > of a reserved spacefrom the dirstream) and readdir might reallocate > it if it finds a large entry. > > - Failures in non-LFS readdir (such as d_ino/d_off overflow or failure > in expand the translation buffer) is ignored. This allows code that > does not handle possible failure in readdir to successful read the > rest of entries in the directory. > > - So minor fixes in dirent/tst-seekdir2.c > > - Added __attribute_deprecated_msg__ on getdirentries to redirect to > getdents64. > > I have checked on x86_64-linux-gnu, powerpc64-linux-gnu and on some > 32-bit ABIs arm-linux-gnueabihf, powerpc-linux-gnu, and sparcv9-linux-gnu. Are there any news on this patch series? The bug is still present as far as I know. Thanks, Adrian
Hello! On 10/21/20 4:15 PM, Adhemerval Zanella wrote: > This is updated version [1], rebased against master add with some > suggestion from previous review. The main changes are: > > - The translation buffer is allocated as an auxiliarry one (instead > of a reserved spacefrom the dirstream) and readdir might reallocate > it if it finds a large entry. > > - Failures in non-LFS readdir (such as d_ino/d_off overflow or failure > in expand the translation buffer) is ignored. This allows code that > does not handle possible failure in readdir to successful read the > rest of entries in the directory. > > - So minor fixes in dirent/tst-seekdir2.c > > - Added __attribute_deprecated_msg__ on getdirentries to redirect to > getdents64. > > I have checked on x86_64-linux-gnu, powerpc64-linux-gnu and on some > 32-bit ABIs arm-linux-gnueabihf, powerpc-linux-gnu, and sparcv9-linux-gnu. Are there any news on this patch series besides the v3 version? [1] Thanks, Adrian > [1] https://patchwork.ozlabs.org/project/glibc/list/?series=209257
On 19/08/2021 13:07, John Paul Adrian Glaubitz wrote: > Hello! > > On 10/21/20 4:15 PM, Adhemerval Zanella wrote: >> This is updated version [1], rebased against master add with some >> suggestion from previous review. The main changes are: >> >> - The translation buffer is allocated as an auxiliarry one (instead >> of a reserved spacefrom the dirstream) and readdir might reallocate >> it if it finds a large entry. >> >> - Failures in non-LFS readdir (such as d_ino/d_off overflow or failure >> in expand the translation buffer) is ignored. This allows code that >> does not handle possible failure in readdir to successful read the >> rest of entries in the directory. >> >> - So minor fixes in dirent/tst-seekdir2.c >> >> - Added __attribute_deprecated_msg__ on getdirentries to redirect to >> getdents64. >> >> I have checked on x86_64-linux-gnu, powerpc64-linux-gnu and on some >> 32-bit ABIs arm-linux-gnueabihf, powerpc-linux-gnu, and sparcv9-linux-gnu. > > Are there any news on this patch series besides the v3 version? [1] Unfortunately it is requires reviews. Since it only affects legacy ABIs there is no much traction for fixing it. > > Thanks, > Adrian > >> [1] https://patchwork.ozlabs.org/project/glibc/list/?series=209257 >
Hi! On 8/19/21 6:19 PM, Adhemerval Zanella wrote: >> Are there any news on this patch series besides the v3 version? [1] > > Unfortunately it is requires reviews. Since it only affects legacy > ABIs there is no much traction for fixing it. Would it help if I put a bounty on this bug report? I still think this is an important fix as it breaks 32-bit qemu-user guests on 64-bit systems and currently, I have to build the glibc package manually for Debian whenever it's updated in the archives. Adrian
On 19/08/2021 13:50, John Paul Adrian Glaubitz wrote: > Hi! > > On 8/19/21 6:19 PM, Adhemerval Zanella wrote: >>> Are there any news on this patch series besides the v3 version? [1] >> >> Unfortunately it is requires reviews. Since it only affects legacy >> ABIs there is no much traction for fixing it. > > Would it help if I put a bounty on this bug report? I am not sure in fact; the main problem is no one is really interested in revise if my approach is the expected one and if the patch is what we want. > > I still think this is an important fix as it breaks 32-bit qemu-user guests > on 64-bit systems and currently, I have to build the glibc package manually > for Debian whenever it's updated in the archives. > > Adrian >
On 8/19/21 6:52 PM, Adhemerval Zanella wrote:> On 19/08/2021 13:50, John Paul Adrian Glaubitz wrote: >> Hi! >> >> On 8/19/21 6:19 PM, Adhemerval Zanella wrote: >>>> Are there any news on this patch series besides the v3 version? [1] >>> >>> Unfortunately it is requires reviews. Since it only affects legacy >>> ABIs there is no much traction for fixing it. >> >> Would it help if I put a bounty on this bug report? > > I am not sure in fact; the main problem is no one is really interested in > revise if my approach is the expected one and if the patch is what we want. Well, maybe a little incentive can help which is why I asked :-). I don't have the knowledge myself to help with the issue, but I would greatly appreciate it if it could finally get fixed since it has been a burden ever since the 2.29 release when this API got changed. Adrian
* John Paul Adrian Glaubitz: > On 8/19/21 6:52 PM, Adhemerval Zanella wrote:> On 19/08/2021 13:50, John Paul Adrian Glaubitz wrote: >>> Hi! >>> >>> On 8/19/21 6:19 PM, Adhemerval Zanella wrote: >>>>> Are there any news on this patch series besides the v3 version? [1] >>>> >>>> Unfortunately it is requires reviews. Since it only affects legacy >>>> ABIs there is no much traction for fixing it. >>> >>> Would it help if I put a bounty on this bug report? >> >> I am not sure in fact; the main problem is no one is really interested in >> revise if my approach is the expected one and if the patch is what we want. > > Well, maybe a little incentive can help which is why I asked :-). I can't speak for everyone, but for me offers of payment raise complicated compliance issues. It's hard to avoid the appearance of a conflict of interest once third-party payments are involved. That is even true if the people involved never intended to claim any money. Thanks, Florian
On 8/19/21 7:15 PM, Florian Weimer wrote: >> Well, maybe a little incentive can help which is why I asked :-). > > I can't speak for everyone, but for me offers of payment raise > complicated compliance issues. It's hard to avoid the appearance of a > conflict of interest once third-party payments are involved. That is > even true if the people involved never intended to claim any money. So, what would be the best way to get this bug fixed then? I'm open to any suggested path way. It's an important fix for anyone using qemu-user in a common use case. Adrian
Hi all, It seems that Peter Maydell of qemu fame thinks it could be solved in qemu if/when the kernel people add an explicit 32bit mode to the filesystem API. See the bug here: https://gitlab.com/qemu-project/qemu/-/issues/263 And specifically, he mentions that Linus Walleij has submitted a preliminary patch, which has yet to be looked at. https://lore.kernel.org/qemu- devel/20201117233928.255671-1-linus.walleij@linaro.org/ Not sure if this is an easier path or not, but it might be worth pinging the kernel ML. Best regards, Dave On Thu, Aug 19, 2021 at 5:30 PM John Paul Adrian Glaubitz < glaubitz@physik.fu-berlin.de> wrote: > On 8/19/21 7:15 PM, Florian Weimer wrote: > >> Well, maybe a little incentive can help which is why I asked :-). > > > > I can't speak for everyone, but for me offers of payment raise > > complicated compliance issues. It's hard to avoid the appearance of a > > conflict of interest once third-party payments are involved. That is > > even true if the people involved never intended to claim any money. > > So, what would be the best way to get this bug fixed then? I'm open to > any suggested path way. > > It's an important fix for anyone using qemu-user in a common use case. > > Adrian > > -- > .''`. John Paul Adrian Glaubitz > : :' : Debian Developer - glaubitz@debian.org > `. `' Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de > `- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 > >