From patchwork Mon Aug 23 21:57:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 44767 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id E3454385781B for ; Mon, 23 Aug 2021 21:57:48 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org E3454385781B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1629755868; bh=Z4NPnYLsG+Qph/E7UO4YsOTVi72wreHvDYQTviT2Jy8=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=kPrhdZnuPVkhhs44qzO/kGAzH/7Wb6hurFrtNWZ3UQ/400tpBe0MhPw8YWtZtbVNo 7+oSAxuoFSU2qiZfFDDcdpdEtwxtS+PxgM0M12oydOgtdgEk6INUcMWZnbjBTRs3tu 7UC824f+NDQ5rje8g/0BIxyjB1V/xhUDjB8Z3Gqs= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-qk1-x731.google.com (mail-qk1-x731.google.com [IPv6:2607:f8b0:4864:20::731]) by sourceware.org (Postfix) with ESMTPS id 21A453858C3B for ; Mon, 23 Aug 2021 21:57:18 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 21A453858C3B Received: by mail-qk1-x731.google.com with SMTP id t190so20937771qke.7 for ; Mon, 23 Aug 2021 14:57:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=Z4NPnYLsG+Qph/E7UO4YsOTVi72wreHvDYQTviT2Jy8=; b=iiC/MkZbP2qlmVxeIzWXIZL6wtCfYm//0D/3PIxHM1ZE3TIG8Q9eO7Q6g/csv1iO56 W9uhegCq3AzK5VQPlbPe70+HMy3MEpaNrjpOMsWcNBkjWFVB9Q7jr7fU/KAQNcRxkDu0 WoYfSgp63KgeFkwGvnC6jWk2AXPyV13myZhy+EKkeeEhrAHg17EddKtKZFy2bopbfDRm Fl/VEaNJOhsKOARKWS2bCDwmqFLyAydY9XD8/RIga1NdbRdSWrTqyr6MSkGWgCFoDB1d kK42Ibf/V0UemVekCMBRqzFTofYnVEsCnoy5GBZVJiEuWKJI6QJ+J0OzoWwvZ8Kgi6sL WrXQ== X-Gm-Message-State: AOAM531ScfH0CdQI1k6lKbJe0ZMxka+Z8MUPs1u+7RVYDCNpMYWHzWzP LVCMlhiFr/34G/APFSHcza+kiGPbH1dICg== X-Google-Smtp-Source: ABdhPJx+UCvKFgpMrbAYkqKL7l2RNiej+LN1UUXTM64QW5Twc2vaF449qruFK0UoGHgW8iBt8uxBxQ== X-Received: by 2002:a05:620a:2a10:: with SMTP id o16mr10971792qkp.481.1629755837530; Mon, 23 Aug 2021 14:57:17 -0700 (PDT) Received: from birita.. ([2804:431:c7ca:cd83:c38b:b50d:5d9a:43d4]) by smtp.gmail.com with ESMTPSA id g1sm7444540qti.56.2021.08.23.14.57.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Aug 2021 14:57:17 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH v3 0/5] malloc: Improve Huge Page support Date: Mon, 23 Aug 2021 18:57:08 -0300 Message-Id: <20210823215713.3304523-1-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Cc: Norbert Manthey , Guillaume Morin , Siddhesh Poyarekar Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" Linux currently supports two ways to use Huge Pages: either by using specific flags directly with the syscall (MAP_HUGETLB for mmap(), or SHM_HUGETLB for shmget()), or by using Transparent Huge Pages (THP) where the kernel will try to move allocated anonymous pages to Huge Pages blocks transparent to application. Also, THP current support three different modes [1]: 'never', 'madvise', and 'always'. The 'never' is self-explanatory and 'always' will enable THP for all anonymous memory. However, 'madvise' is still the default for some systems and for such cases THP will be only used if the memory range is explicity advertise by the program through a madvise(MADV_HUGEPAGE) call. This patchset adds a two new tunables to improve malloc() support with Huge Page, 'glibc.malloc.hugetlb' with the supported values: - '0': default value, do not enable any huge page usage. - '1': instruct the system allocator to issue a madvise(MADV_HUGEPAGE) call after a mmap() one for sizes larger than the default huge page size and on sbrk() calls to extend the program data segment. - '2': instruct the system allocator to round allocation to huge page sizes along with the required flags (MAP_HUGETLB for Linux). If the memory allocation fails, the default system page size is used instead. - positive value: same as '2' but to use a specific huge page size. The value is checked against the supported one by the system. The 'glibc.malloc.hugetlb=2' aims to replace the 'morecore' removed callback from 2.34 for libhugetlbfs (where the library tries to leverage the huge pages usage instead to provide a system allocator). By implementing the support directly on the mmap() code patch there is no need to try emulate the morecore()/sbrk() semantic which simplifies the code and make memory shrink logic more straighforward. Changes from previous version: - Only use one tunable, 'glibc.malloc.hugetlb'. - Disable madvise if THP is set to always. - Add support to arenas. Adhemerval Zanella (5): malloc: Add madvise support for Transparent Huge Pages malloc: Add THP/madvise support for sbrk malloc: Move mmap logic to its own function malloc: Add Huge Page support for mmap() malloc: Add huge page support to arenas NEWS | 11 +- Rules | 36 +++ elf/dl-tunables.list | 4 + elf/tst-rtld-list-tunables.exp | 1 + include/libc-pointer-arith.h | 10 + malloc/Makefile | 18 ++ malloc/arena.c | 124 +++++++--- malloc/malloc-internal.h | 1 + malloc/malloc.c | 268 +++++++++++++++------ manual/tunables.texi | 16 ++ sysdeps/generic/Makefile | 8 + sysdeps/generic/malloc-hugepages.c | 38 +++ sysdeps/generic/malloc-hugepages.h | 44 ++++ sysdeps/unix/sysv/linux/malloc-hugepages.c | 202 ++++++++++++++++ 14 files changed, 660 insertions(+), 121 deletions(-) create mode 100644 sysdeps/generic/malloc-hugepages.c create mode 100644 sysdeps/generic/malloc-hugepages.h create mode 100644 sysdeps/unix/sysv/linux/malloc-hugepages.c