From patchwork Fri Aug 13 21:04:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 44664 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C739E3857424 for ; Fri, 13 Aug 2021 21:06:49 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org C739E3857424 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1628888809; bh=P/TcJ6r/PgTtmm016ijdlKo/YH3/X1qB+XmBSMDngPw=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=JKhJ9YTwEU3tn2i7hpWPCEhEtveIfYUQkWG6Zx0oKteluFdI8z8+smFL9PRgdu0gT b3dz79aVRoM0eIr9HQ5zs/UTIrn8WMpS+ATrbGHNTGiLTp3FHD56B6RLKXXfFEZ6k5 HJBuy9xF4JaDGEqHIn6ilrZTSFVX7OMWiWejWNg8= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com [IPv6:2607:f8b0:4864:20::1029]) by sourceware.org (Postfix) with ESMTPS id AD29D3889C30 for ; Fri, 13 Aug 2021 21:04:40 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org AD29D3889C30 Received: by mail-pj1-x1029.google.com with SMTP id 28-20020a17090a031cb0290178dcd8a4d1so12057388pje.0 for ; Fri, 13 Aug 2021 14:04:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=P/TcJ6r/PgTtmm016ijdlKo/YH3/X1qB+XmBSMDngPw=; b=DL2P8+yGdvyc9H/Db+7DeO/L0EOIfDFE/BmqJqhatVka5dBeIthQQqEwcA71ESWIGz D1FNz7NO5dir5TbixQn8K2Meqk4BAtcNDhBBbU5k/1+0W3OXl+4lmLk+EWW8eF+B3JB4 kbAlyRI2BbixTiqCv1cE60mXFLR444IaGgF7kLQuGhki3t2zjlcuegLIyFtfsQvGBhgl 20cy9hapr6AWwgfQLwJpMWNyrzEzJfmhEUF1snQLeVBuMjB4JQzT6B5KSwiOeuOU0/2j eKZB7pMNxOVPBUsQZ4iGAt5RB7UCJthoPz3WFsytbl8LfrXnayRvOMJ7lh3bEq01Vztd zhKw== X-Gm-Message-State: AOAM5317JvnFoyiVR3vdG6Laz8x5/vXPEfpz2UcettX7lho/QeyIctQt sLr96BvXHLcj9AxLTxTLjsjl2xatgeGD0w== X-Google-Smtp-Source: ABdhPJwky/dqL0sLXhp7bMIu0hx6OmtdYJpGunXP4t1cIF1IBHO+GG/UtmV80i5IeGw3f2lwiFqFVA== X-Received: by 2002:a17:903:244d:b029:12d:77ee:2dd1 with SMTP id l13-20020a170903244db029012d77ee2dd1mr3538079pls.72.1628888679503; Fri, 13 Aug 2021 14:04:39 -0700 (PDT) Received: from birita.. ([2804:431:c7cb:9dce:7cf8:4c28:b9e7:2767]) by smtp.gmail.com with ESMTPSA id v15sm3259462pff.105.2021.08.13.14.04.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 14:04:39 -0700 (PDT) To: libc-alpha@sourceware.org Subject: [PATCH 3/3] malloc: Add arch-specific malloc_verify_thp_pagesize for Linux Date: Fri, 13 Aug 2021 18:04:29 -0300 Message-Id: <20210813210429.1147112-4-adhemerval.zanella@linaro.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210813210429.1147112-1-adhemerval.zanella@linaro.org> References: <20210813210429.1147112-1-adhemerval.zanella@linaro.org> MIME-Version: 1.0 X-Spam-Status: No, score=-13.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Adhemerval Zanella via Libc-alpha From: Adhemerval Zanella Netto Reply-To: Adhemerval Zanella Cc: Norbert Manthey , Siddhesh Poyarekar , Guillaume Morin Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" Not all architectures have Transparent Huge Page (THP) support enabled by default, so this patch only adds support for the one that have HAVE_ARCH_TRANSPARENT_HUGEPAGE defined. Also, Linux THP only support one Huge Page size, so malloc_verify_thp_pagesize() returns only the default value for the architecture. The x86, sparc, and riscv are straightforward since they only support one possible value. AArch64 and mips, which supports multiple pages sizes, can use a direct map to the Large Page support. PowerPC is the only architecture where its THP size depends not only on the configured page size, but also on which MMU is used. For this case the sysfs file is used instead. Checked on x86_64-linux-gnu, aarch64-linux-gnu, and powerpc64le-linux-gnu. --- sysdeps/unix/sysv/linux/aarch64/malloc-thp.h | 40 ++++++++++++++ sysdeps/unix/sysv/linux/mips/malloc-thp.h | 39 ++++++++++++++ sysdeps/unix/sysv/linux/powerpc/malloc-thp.h | 56 ++++++++++++++++++++ sysdeps/unix/sysv/linux/riscv/malloc-thp.h | 32 +++++++++++ sysdeps/unix/sysv/linux/s390/malloc-thp.h | 33 ++++++++++++ sysdeps/unix/sysv/linux/sparc/malloc-thp.h | 36 +++++++++++++ sysdeps/unix/sysv/linux/x86/malloc-thp.h | 32 +++++++++++ 7 files changed, 268 insertions(+) create mode 100644 sysdeps/unix/sysv/linux/aarch64/malloc-thp.h create mode 100644 sysdeps/unix/sysv/linux/mips/malloc-thp.h create mode 100644 sysdeps/unix/sysv/linux/powerpc/malloc-thp.h create mode 100644 sysdeps/unix/sysv/linux/riscv/malloc-thp.h create mode 100644 sysdeps/unix/sysv/linux/s390/malloc-thp.h create mode 100644 sysdeps/unix/sysv/linux/sparc/malloc-thp.h create mode 100644 sysdeps/unix/sysv/linux/x86/malloc-thp.h diff --git a/sysdeps/unix/sysv/linux/aarch64/malloc-thp.h b/sysdeps/unix/sysv/linux/aarch64/malloc-thp.h new file mode 100644 index 0000000000..e2e65446f2 --- /dev/null +++ b/sysdeps/unix/sysv/linux/aarch64/malloc-thp.h @@ -0,0 +1,40 @@ +/* Transparent Huge Page support. AArch64 implementation. + Copyright (C) 2021 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public License as + published by the Free Software Foundation; either version 2.1 of the + License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; see the file COPYING.LIB. If + not, see . */ + +#ifndef _MALLOC_THP_H +#define _MALLOC_THP_H + +#include +#include + +/* Return the prefered large page size for the request PAGESIZE. The + requested value of 1 means the default size for the architecutre. + Returning 0 disables Large Parse usage. */ +static inline size_t +malloc_verify_thp_pagesize (size_t pagesize) +{ + /* AArch64 THP size depends of the default page size: + 4k -> 2m + 16k -> 32m + 64k -> 512m */ + int page_shift = __builtin_ctzl (GLRO(dl_pagesize)); + printf ("%s: page_shift=%d\n", __func__, page_shift); + return 1UL << ((page_shift - 3) * 2 + 3); +} + +#endif /* _MALLOC_THP_H */ diff --git a/sysdeps/unix/sysv/linux/mips/malloc-thp.h b/sysdeps/unix/sysv/linux/mips/malloc-thp.h new file mode 100644 index 0000000000..d8cdbd026d --- /dev/null +++ b/sysdeps/unix/sysv/linux/mips/malloc-thp.h @@ -0,0 +1,39 @@ +/* Transparent Huge Page support. Generic implementation. + Copyright (C) 2021 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public License as + published by the Free Software Foundation; either version 2.1 of the + License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; see the file COPYING.LIB. If + not, see . */ + +#ifndef _MALLOC_THP_H +#define _MALLOC_THP_H + +#include + +/* Return the prefered large page size for the request PAGESIZE. The + requested value of 1 means the default size for the architecture. */ +static inline size_t +malloc_verify_thp_pagesize (size_t pagesize) +{ + /* MIPS THP size depends of the default page size: + 4k -> 2m + 8k -> 8m + 16k -> 32m + 32k -> 128m + 64k -> 512m */ + int page_shift = __builtin_ctzl (GLRO(dl_pagesize)); + return 1UL << (page_shift + (page_shift - 3)); +} + +#endif /* _MALLOC_THP_H */ diff --git a/sysdeps/unix/sysv/linux/powerpc/malloc-thp.h b/sysdeps/unix/sysv/linux/powerpc/malloc-thp.h new file mode 100644 index 0000000000..c3fcfa3386 --- /dev/null +++ b/sysdeps/unix/sysv/linux/powerpc/malloc-thp.h @@ -0,0 +1,56 @@ +/* Transparent Huge Page support. PowerPC implementation. + Copyright (C) 2021 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public License as + published by the Free Software Foundation; either version 2.1 of the + License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; see the file COPYING.LIB. If + not, see . */ + +#ifndef _MALLOC_THP_H +#define _MALLOC_THP_H + +#include +#include + +/* Return the prefered large page size for the request PAGESIZE. The + requested value of 1 means the default size for the architecture. */ +static inline size_t +malloc_verify_thp_pagesize (size_t pagesize) +{ + /* PowerPC THP size depends of the default page size and which MMU hardware + is used. So no easy way to statically map it, query the kernel + instead. */ + int fd = __open64_nocancel ( + "/sys/kernel/mm/transparent_hugepage/hpage_pmd_size", O_RDONLY); + if (fd == -1) + return 0; + + size_t hps = 0; + + char str[INT_BUFSIZE_BOUND (size_t)]; + ssize_t r = __read_nocancel (fd, str, sizeof (str)); + if (r > 0) + for (ssize_t i = 0; i < r; i++) + { + if (str[i] == '\n') + break; + hps *= 10; + hps += str[i] - '0'; + } + + __close_nocancel (fd); + + return hps; +} + +#endif /* _MALLOC_THP_H */ diff --git a/sysdeps/unix/sysv/linux/riscv/malloc-thp.h b/sysdeps/unix/sysv/linux/riscv/malloc-thp.h new file mode 100644 index 0000000000..aa38ca6dd6 --- /dev/null +++ b/sysdeps/unix/sysv/linux/riscv/malloc-thp.h @@ -0,0 +1,32 @@ +/* Transparent Huge Page support. RISCV implementation. + Copyright (C) 2021 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public License as + published by the Free Software Foundation; either version 2.1 of the + License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; see the file COPYING.LIB. If + not, see . */ + +#ifndef _MALLOC_THP_H +#define _MALLOC_THP_H + +#include + +/* Return the prefered large page size for the request PAGESIZE. The + requested value of 1 means the default size for the architecture. */ +static inline size_t +malloc_verify_thp_pagesize (size_t pagesize) +{ + return 1UL << 21; +} + +#endif /* _MALLOC_THP_H */ diff --git a/sysdeps/unix/sysv/linux/s390/malloc-thp.h b/sysdeps/unix/sysv/linux/s390/malloc-thp.h new file mode 100644 index 0000000000..be6c26aa3b --- /dev/null +++ b/sysdeps/unix/sysv/linux/s390/malloc-thp.h @@ -0,0 +1,33 @@ +/* Transparent Huge Page support. Generic implementation. + Copyright (C) 2021 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public License as + published by the Free Software Foundation; either version 2.1 of the + License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; see the file COPYING.LIB. If + not, see . */ + +#ifndef _MALLOC_THP_H +#define _MALLOC_THP_H + +#include + +/* Return the prefered large page size for the request PAGESIZE. The + requested value of 1 means the default size for the architecture. */ +static inline size_t +malloc_verify_thp_pagesize (size_t pagesize) +{ + /* s390 uses 1M for THP. */ + return 1UL << 20; +} + +#endif /* _MALLOC_THP_H */ diff --git a/sysdeps/unix/sysv/linux/sparc/malloc-thp.h b/sysdeps/unix/sysv/linux/sparc/malloc-thp.h new file mode 100644 index 0000000000..83f6fdc114 --- /dev/null +++ b/sysdeps/unix/sysv/linux/sparc/malloc-thp.h @@ -0,0 +1,36 @@ +/* Transparent Huge Page support. SPARC implementation. + Copyright (C) 2021 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public License as + published by the Free Software Foundation; either version 2.1 of the + License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; see the file COPYING.LIB. If + not, see . */ + +#ifndef _MALLOC_THP_H +#define _MALLOC_THP_H + +#include + +/* Return the prefered large page size for the request PAGESIZE. The + requested value of 1 means the default size for the architecture. */ +static inline size_t +malloc_verify_thp_pagesize (size_t pagesize) +{ +#ifdef __arch64__ + return 1UL << 23; +#else + return 1UL << 18; +#endif +} + +#endif /* _MALLOC_THP_H */ diff --git a/sysdeps/unix/sysv/linux/x86/malloc-thp.h b/sysdeps/unix/sysv/linux/x86/malloc-thp.h new file mode 100644 index 0000000000..d94cb578c2 --- /dev/null +++ b/sysdeps/unix/sysv/linux/x86/malloc-thp.h @@ -0,0 +1,32 @@ +/* Transparent Huge Page support. Generic implementation. + Copyright (C) 2021 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public License as + published by the Free Software Foundation; either version 2.1 of the + License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; see the file COPYING.LIB. If + not, see . */ + +#ifndef _MALLOC_THP_H +#define _MALLOC_THP_H + +#include + +/* Return the prefered large page size for the request PAGESIZE. The + requested value of 1 means the default size for the architecture. */ +static inline size_t +malloc_verify_thp_pagesize (size_t pagesize) +{ + return 1UL << 21; +} + +#endif /* _MALLOC_THP_H */