From patchwork Mon Jan 22 19:30:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xi Ruoyao X-Patchwork-Id: 84574 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5C5EE3858414 for ; Mon, 22 Jan 2024 19:32:50 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from xry111.site (xry111.site [89.208.246.23]) by sourceware.org (Postfix) with ESMTPS id 80C023858D20 for ; Mon, 22 Jan 2024 19:32:19 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 80C023858D20 Authentication-Results: sourceware.org; dmarc=pass (p=reject dis=none) header.from=xry111.site Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=xry111.site ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 80C023858D20 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=89.208.246.23 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1705951949; cv=none; b=EepivkUi+YkoH1hQ5B7dac/mcAzotonDNab6eC6YLnDsI9refg7qU3IAfFwoKKn1Ktpd0IUVvNlZ7HqLk53wavZjfan8rthxyjSSeQwcKS944vfQYpl2Jx8z2FGu1GfBTPen1IIqYod/IlFvT86C8aG0wX0G8LzZPIzVax0hHQ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1705951949; c=relaxed/simple; bh=+6TQASv9SpayodHhqfkBWPiH0nam/HpYy94K1EcCQP0=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=UsjYB3r+oDX9RDA266xgY1nHuBaMABvmWF99PN+vgh66cb9Xv46yQ5T4yRd1WX/wmlOPi9ryD/0pErHIL8u1OK7DCjZgrD8VqGTdw7YpNTBvXG14zlkiTEA4xS2kIvMqGFEUbqg1X2kB4+0AEfXDii4Owfr4SsNejINipaLPiko= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=xry111.site; s=default; t=1705951938; bh=+6TQASv9SpayodHhqfkBWPiH0nam/HpYy94K1EcCQP0=; h=From:To:Cc:Subject:Date:From; b=FUa6EIabC9jS/X42i1ZalSiwaZff3KYYvBm6GEA8aJ0z4oQQiWeXvxj24cO/nhv7e 6JuhqbDnYFGr7MafcZHHp1khIx8nP8SN9cr5fKwMt8vDlYUPCBa0J8vAYac65JZhdQ JIVIxWysK09BNB56XFb8mFPQl4b2ALuwZVBfnoQw= Received: from stargazer.. (unknown [IPv6:240e:358:119a:ce00:dc73:854d:832e:2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (Client did not present a certificate) (Authenticated sender: xry111@xry111.site) by xry111.site (Postfix) with ESMTPSA id 22C0D66F47; Mon, 22 Jan 2024 14:32:15 -0500 (EST) From: Xi Ruoyao To: libc-alpha@sourceware.org Cc: Adhemerval Zanella Netto , "Andreas K . Huettel" , Xi Ruoyao Subject: [PATCH] qsort: Fix a typo causing unnecessary malloc/free Date: Tue, 23 Jan 2024 03:30:55 +0800 Message-ID: <20240122193150.2731985-1-xry111@xry111.site> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 X-Spam-Status: No, score=-9.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, LIKELY_SPAM_FROM, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org In qsort_r we allocate a buffer sized QSORT_STACK_SIZE (1024) on stack and we intend to use it if all elements can fit into it. But there is a typo: if (total_size < sizeof buf) buf = tmp; else /* allocate a buffer on heap and use it ... */ Here "buf" is a pointer, thus sizeof buf is just 4 or 8, instead of 1024. This bug is detected debugging some strange heap corruption running the Ruby-3.3.0 test suite (on an experimental Linux From Scratch build using Binutils-2.41.90 and Glibc trunk, and also Fedora Rawhide [1]). It seems Ruby is doing some wild "optimization" by jumping into somewhere in qsort_r instead of calling it normally, resulting in a double free of buf if we allocate it on heap. The issue can be reproduced deterministically with: LD_PRELOAD=/usr/lib/libc_malloc_debug.so MALLOC_CHECK_=3 \ LD_LIBRARY_PATH=. ./ruby test/runner.rb test/ruby/test_enum.rb in Ruby-3.3.0 tree after building it. This change would hide the issue for Ruby, but Ruby is likely still buggy (if using this "optimization" sorting larger arrays). [1]:https://kojipkgs.fedoraproject.org//work/tasks/9729/111889729/build.log Signed-off-by: Xi Ruoyao --- stdlib/qsort.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stdlib/qsort.c b/stdlib/qsort.c index 7f5a00fb33..1c1505e7b2 100644 --- a/stdlib/qsort.c +++ b/stdlib/qsort.c @@ -353,7 +353,7 @@ __qsort_r (void *const pbase, size_t total_elems, size_t size, if (size > INDIRECT_SORT_SIZE_THRES) total_size = 2 * total_elems * sizeof (void *) + size; - if (total_size < sizeof buf) + if (total_size < sizeof tmp) buf = tmp; else {