From patchwork Thu Mar 4 16:33:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Szabolcs Nagy X-Patchwork-Id: 42253 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A542F3AAA0AB; Thu, 4 Mar 2021 16:33:48 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org A542F3AAA0AB DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1614875628; bh=kBUcnFjZricgm0E4t6EsPdSBCvOWMltky5W3meiCiVk=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=KaFzOj4FBLovzt3o6Jhbv8O+pLjwqPnBTHbGCSbquyLLhtfOVITdsD5OPl/lny7Sl DwGgxDk0t9VhOK0jy2t8ZuZHw/nJtF5IwGtGsZ+mIpuISyjCqJ/j3oNsobHnxmEFge S9lYu1sElcJtnq8zBCQmQtKxO+AMD7Nm09o6gdHQ= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on062c.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe1e::62c]) by sourceware.org (Postfix) with ESMTPS id CA3E3389041C for ; Thu, 4 Mar 2021 16:33:44 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org CA3E3389041C Received: from AM6P191CA0008.EURP191.PROD.OUTLOOK.COM (2603:10a6:209:8b::21) by VI1PR0801MB1886.eurprd08.prod.outlook.com (2603:10a6:800:8b::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3890.20; Thu, 4 Mar 2021 16:33:42 +0000 Received: from VE1EUR03FT011.eop-EUR03.prod.protection.outlook.com (2603:10a6:209:8b:cafe::98) by AM6P191CA0008.outlook.office365.com (2603:10a6:209:8b::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.17 via Frontend Transport; Thu, 4 Mar 2021 16:33:42 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; sourceware.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;sourceware.org; dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT011.mail.protection.outlook.com (10.152.18.134) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.17 via Frontend Transport; Thu, 4 Mar 2021 16:33:41 +0000 Received: ("Tessian outbound 155e15b2e217:v71"); Thu, 04 Mar 2021 16:33:41 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: cbec6ed8ce047976 X-CR-MTA-TID: 64aa7808 Received: from 37c15e2de80a.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id EF7450BB-B3F2-433E-9205-BAF030799083.1; Thu, 04 Mar 2021 16:33:33 +0000 Received: from EUR04-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 37c15e2de80a.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 04 Mar 2021 16:33:33 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DzORoJ9W+t1sgwb1S8Dqii+XoSrQ8OMui4npZm2J/XUeTKOItJsGeaDdnVidqza/JVdgzUQxLEhvm/+/DKqldpsA9bHVIWC6yw+kva7an/zZgc4+XaRy6XQDDS1gtFei8ZSEkjCPvOXTUKrhWqzySjjmXHd/FniGle52W3DUCVaCOM/uAxMPVXDWMLL76vclsUVsSA0Lj5/w50Ur2F25JusSYn0/g4A7dMnq62ybIPlLlg9fqL0j04qSgcGf27NYFWTk2hZFShJZRxhkXjQyMnzFPGmSPllHxEseqoIpkS9EsYd1Y4M/a4zZkFsfmlpFsaquqa4uOQ96Ii9pS29zOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kBUcnFjZricgm0E4t6EsPdSBCvOWMltky5W3meiCiVk=; b=CLq6hc9xiK5dfNvmLoe90xmztnzBvW/NusZdeQ54pOnA1xGv1cL1eNbQcL+yzNHGwsP73xI5Z5i4eQQhZEoltwdbH/6dhoaBLXpspR4dD7tnmLSr9qxqrdQFS8IWBwCr8GrOs1I2yTAL2C/YoE2lQ7MLi+XqARHbh9KsOPhhsuUDCIlGOtKSYy9lx2hyDYd5aJXcVwjg9Bto7nP2YhWIxGdnG0ZlWNwhwgCpNmBiVMQlHX+EdVzSuBopwoFcUofuq/+zQ1XZdVpZO7qgSSlr77PIQYxvLoh1TUs39ErfLwdtay2wpim40sRWuKtZf+xN9LNxO0uo7FIqq+lO9JPZ+A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none Authentication-Results-Original: sourceware.org; dkim=none (message not signed) header.d=none;sourceware.org; dmarc=none action=none header.from=arm.com; Received: from PA4PR08MB6320.eurprd08.prod.outlook.com (2603:10a6:102:e5::9) by PA4PR08MB6064.eurprd08.prod.outlook.com (2603:10a6:102:e2::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3890.20; Thu, 4 Mar 2021 16:33:31 +0000 Received: from PA4PR08MB6320.eurprd08.prod.outlook.com ([fe80::60f0:3773:69b8:e336]) by PA4PR08MB6320.eurprd08.prod.outlook.com ([fe80::60f0:3773:69b8:e336%2]) with mapi id 15.20.3912.021; Thu, 4 Mar 2021 16:33:31 +0000 To: libc-alpha@sourceware.org, Richard.Earnshaw@arm.com, DJ Delorie Subject: [PATCH 11/16] malloc: Use branches instead of mtag_granule_mask Date: Thu, 4 Mar 2021 16:33:25 +0000 Message-Id: <93b2191bf171072a519f19e9f8e763af9156213f.1614874816.git.szabolcs.nagy@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: X-Originating-IP: [217.140.106.55] X-ClientProxiedBy: LO2P123CA0106.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:139::21) To PA4PR08MB6320.eurprd08.prod.outlook.com (2603:10a6:102:e5::9) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (217.140.106.55) by LO2P123CA0106.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:139::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3890.21 via Frontend Transport; Thu, 4 Mar 2021 16:33:31 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 82a70303-a8ed-41a1-0d84-08d8df2b45d8 X-MS-TrafficTypeDiagnostic: PA4PR08MB6064:|VI1PR0801MB1886: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: N2CDrS+mJYr2UqQ3i4dtLuwZDehNFCy9qinHMW3m+zduR9a6jB4KxyjWZi0yR/YrCQehFOrqqYvrxUNQuSAlY4cFBYPmI6afk1V/aLYKsEGkU5WTEgeuXIgMwCw6to4iNiQd9+wWaOKeUGjt85tLGKePXoHpv+mQ1/hG9/JD/CdFwBJLA9UFKnFSjpGp1L7ek4JO3A1kZIalbIMVGGnjPDWXOEUNk1n7gF7xzLS3XaLljgCQOJ3yvMsX21jnBkrhfXvVMIKJpy5G/CPX0XgpsAn2Ypv8Y9gj9pxUNKoMbyaK8dSFjC15dkd0o/Q6fv4/wIWQO5kCzqr+GuOShJ6YstJiNEJCyL6PQK8TuTzVaIkklqwdZWi6cmxU6XRXJfBYS3i21U5/COdlX5QzMi00yGNxSRzQM9tnjgXcFlDD5TPSrRtEYV0RJADv8l7gr28xErZEPfIpC2otqrHu/KHk6lCx2IO0ESENFyNk28crbycE0B8CcAJM5tQnSI+VQD98ICac+NlTDP+pxYbYOxgHLby70EEhbzq834C31IOLP2l6/asXHnAx9QgPlyG8Dw4hWPjD+Qw8nlTt0RG3DGNf1Q== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PA4PR08MB6320.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39850400004)(376002)(366004)(346002)(136003)(396003)(69590400012)(8676002)(6512007)(8936002)(478600001)(6666004)(36756003)(2616005)(52116002)(956004)(86362001)(6916009)(6506007)(5660300002)(26005)(6486002)(316002)(66476007)(66946007)(66556008)(186003)(2906002)(16526019)(44832011)(83380400001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: fbjyrtTHKBlqu4vKmUVaDLIiiMmKmquqJESGIhjDn1jyS9BxQRUIzjJkTb0rKLPABEbZb8aW3iFDm1uhOmraY9vdjsS+oH58ZuizoTxRBlu94eycDiG6kWTuRz8bx/PchQ9sg9fedTqBEIJavTGaWZFDvta1Yd53sulWmN/nqQYnvps+hMJ84tglKyParrYqYvOGau9kmZjqydWYk3fX7mzrndyHz34yrULkZ7J6WdV1fbi8SytsFPo5fE94/FAwLwmA8F78HA9/q5H/AabocO4iCZduz64rmDX9cLLfVMPoQL+eAlItc117XBsx0VAGpq6EAS19Wns2xiSRzUw49rSTwaZDwwSY4HUy06holroFh1q8CDBtbOgWt1We+O9kBbGgpCvKHYSPmU0f8d5ZPI/BIBcMDRmY21WIh+AbZp08XRMmraM+8KBYCPteDDZtIm1g4abUubfiBAiuoMq+WWijiOrvjLiAN2uhXJtJF6jNwJuWceh3a8vFVkrIzNq75jk8d7GvhO4NzCCk3bReBxHiGz0mROLcv9Wfq5EFMLtP4/4AFQPY2T9cpgrYhniIEBMgP7wpYrPV4FQZ4+6LE6JmlzFMyTAKgFiKskpmUaJH+FrpVUC6LoAQiWK48aSLxC4U6i/0QwXXs64JQ7KRikNTY27SLOSGd7OHwHItoC6y84xmWc8vReLrpJRIUbF+s1tdeUQps92KYYCleEBykcmLoo2IDPbloaVZXSwdueOMJWevvztpKhYmMFLs639ffsyzVNYcFnA51qAUFAvBN53TnSoXFSsmCyEFldbL9PrYOmqN5dQ4k3A9NnwbshUX6HjHGYBb/yEi9aF0wOR8x+KjHXqyjzrcxRiwuLbfr1l7LtFzgjHsxp3PXT3YGTJLii3MNgpUnbPELmK9R41/DiSwVvp1NchqZSON+QP2+xtPlgM800sxGvXBIky6DLGmYoYCPN7rDQgHY9CNXJe/ijsw+IDO6MfEVgOLVxkAOsuO66CqwaqPfPTdELwqUmW3My2RgN7wmsib2GA/0/mdfTWt+7hTv9yD2Gqr9D3z8sAqCU+6c8zN1qwFT5CtehO8Ld2GH9hQnzg+Ck2bwNHVUaZ8dVzLWVaAvVdBDUWiVL+NTpNBswqKkYqudp8dRBtv73zOt4Jr8IXeo458PeAd3c/8UZeglFf/lYSRhY+7O4vIoLA7uoQZiTQgXUGzf0ecQ/yNaL4OEopq5C+iJU/HNqiKU6ZqG9aVYYG6wrSHkuY2w4VmRHIAf0TsOo1a3oILR2tXHiv6VN215kJbI70muYfFMWv5uDwBVpf1czXSHGillzEY5Z2+K0G0YDo2S77g X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6064 Original-Authentication-Results: sourceware.org; dkim=none (message not signed) header.d=none; sourceware.org; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT011.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: 98cddca2-50b5-4bfb-a93c-08d8df2b3f83 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: SRTajAeRK/rosDgD21k0qwVgPQyft1afoVGAn0Z4jki2q7+KJgn1qMsuUZX0stQkeos5jvO6pCRZW89yQ25olfegwRKJX9GyNY7ulgMq9ZlDCxSKLqmMURJ2zAiVEx0dUkn00aYNloMsnCtGWjkcsKlq71csmOx0QMvy1ylTvrgsPmXtB7e37z8Mu6/YDXl2nNM34Scf1K8nmfHdQEhiz9Qgxp/x5am3PCCLqEkfC/nrRr11kMTgmAfXAjRvSK3AYgteWeVPlohBIUJ8To7c/cLKXX3kPUMLYYLS8OgTX590iFZHXGLzombPPoa18VssmZWnaaUimaPeXAa77oq0qlXrsy7rffUp+m1ZMb9xeJmeYXppRG36sYvRZLLjTP1gMpO84pue9qkGlb7a2rd6JQeaoAffRSXYxDAFnJuuchNE+BM7dTtB+JCigobbiQjB9iDxHA32mt+22MkqZ33V+pJTgbRkc3pGY99YEZjda4/o1K0GQHyCwon1sLuYdRj7oe8w8GFthNJUTcSrtyahOH8blVT2uP4voneTPCoIzbHwgRp/g2KYLCVqyQuU8dgUsYWHemjj8TMVVD+QVea0rZ2jL0TAuwox39PHOto2UBpsGLkg4dA1wYvZrieZYzX1r6LYqeLt6iZJvcn5Jk8VxoOZ9tExDkd3asS4Db+fIiFLM19s/XF7akTGWk3OSQal X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(136003)(346002)(376002)(39850400004)(396003)(46966006)(36840700001)(82740400003)(82310400003)(70586007)(70206006)(36756003)(316002)(86362001)(6862004)(356005)(47076005)(83380400001)(81166007)(8676002)(478600001)(8936002)(6506007)(5660300002)(44832011)(16526019)(336012)(69590400012)(2616005)(26005)(956004)(6486002)(36860700001)(2906002)(6512007)(6666004)(186003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Mar 2021 16:33:41.9940 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 82a70303-a8ed-41a1-0d84-08d8df2b45d8 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: VE1EUR03FT011.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB1886 X-Spam-Status: No, score=-13.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, MSGID_FROM_MTA_HEADER, SPF_HELO_PASS, SPF_PASS, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Szabolcs Nagy via Libc-alpha From: Szabolcs Nagy Reply-To: Szabolcs Nagy Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" The branches may be better optimized since mtag_enabled is widely used. Granule size larger than a chunk header is not supported since then we cannot have both the chunk header and user area granule aligned. To fix that for targets with large granule, the chunk layout has to change. So code that attempted to handle the granule mask generally was changed. This simplified CHUNK_AVAILABLE_SIZE and the logic in malloc_usable_size. --- malloc/arena.c | 1 - malloc/malloc.c | 34 ++++++++++++++-------------------- 2 files changed, 14 insertions(+), 21 deletions(-) diff --git a/malloc/arena.c b/malloc/arena.c index 1e83bb66bd..9fbbb38a15 100644 --- a/malloc/arena.c +++ b/malloc/arena.c @@ -306,7 +306,6 @@ ptmalloc_init (void) mtag_enabled = true; mtag_mmap_flags = __MTAG_MMAP_FLAGS; - mtag_granule_mask = ~(size_t)(__MTAG_GRANULE_SIZE - 1); } #endif diff --git a/malloc/malloc.c b/malloc/malloc.c index b1ee0f450b..8854afec88 100644 --- a/malloc/malloc.c +++ b/malloc/malloc.c @@ -442,7 +442,6 @@ void *(*__morecore)(ptrdiff_t) = __default_morecore; #ifdef USE_MTAG static bool mtag_enabled = false; static int mtag_mmap_flags = 0; -static size_t mtag_granule_mask = ~(size_t)0; #else # define mtag_enabled false # define mtag_mmap_flags 0 @@ -1333,15 +1332,16 @@ nextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ((req) + SIZE_SZ + MALLOC_ALIGN_MASK) & ~MALLOC_ALIGN_MASK) /* Available size of chunk. This is the size of the real usable data - in the chunk, plus the chunk header. */ -#ifdef USE_MTAG -#define CHUNK_AVAILABLE_SIZE(p) \ - ((chunksize (p) + (chunk_is_mmapped (p) ? 0 : SIZE_SZ)) \ - & mtag_granule_mask) -#else -#define CHUNK_AVAILABLE_SIZE(p) \ - (chunksize (p) + (chunk_is_mmapped (p) ? 0 : SIZE_SZ)) -#endif + in the chunk, plus the chunk header. Note: If memory tagging is + enabled the layout changes to accomodate the granule size, this is + wasteful for small allocations so not done by default. The logic + does not work if chunk headers are not granule aligned. */ +_Static_assert (__MTAG_GRANULE_SIZE <= CHUNK_HDR_SZ, + "memory tagging is not supported with large granule."); +#define CHUNK_AVAILABLE_SIZE(p) \ + (__MTAG_GRANULE_SIZE > SIZE_SZ && __glibc_unlikely (mtag_enabled) ? \ + chunksize (p) : \ + chunksize (p) + (chunk_is_mmapped (p) ? 0 : SIZE_SZ)) /* Check if REQ overflows when padded and aligned and if the resulting value is less than PTRDIFF_T. Returns TRUE and the requested size or MINSIZE in @@ -1353,7 +1353,6 @@ checked_request2size (size_t req, size_t *sz) __nonnull (1) if (__glibc_unlikely (req > PTRDIFF_MAX)) return false; -#ifdef USE_MTAG /* When using tagged memory, we cannot share the end of the user block with the header for the next chunk, so ensure that we allocate blocks that are rounded up to the granule size. Take @@ -1361,8 +1360,9 @@ checked_request2size (size_t req, size_t *sz) __nonnull (1) number. Ideally, this would be part of request2size(), but that must be a macro that produces a compile time constant if passed a constant literal. */ - req = (req + ~mtag_granule_mask) & mtag_granule_mask; -#endif + if (__glibc_unlikely (mtag_enabled)) + req = (req + (__MTAG_GRANULE_SIZE - 1)) & + ~(size_t)(__MTAG_GRANULE_SIZE - 1); *sz = request2size (req); return true; @@ -5112,14 +5112,8 @@ musable (void *mem) result = chunksize (p) - CHUNK_HDR_SZ; } else if (inuse (p)) - result = chunksize (p) - SIZE_SZ; + result = CHUNK_AVAILABLE_SIZE (p) - CHUNK_HDR_SZ; -#ifdef USE_MTAG - /* The usable space may be reduced if memory tagging is needed, - since we cannot share the user-space data with malloc's internal - data structure. */ - result &= mtag_granule_mask; -#endif return result; } return 0;