From patchwork Mon Mar 6 08:04:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kratochvil X-Patchwork-Id: 66024 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 8E44B3858C5F for ; Mon, 6 Mar 2023 08:22:10 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 8E44B3858C5F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1678090930; bh=1O/KxpBKz3LYBrw/6vsbq3700Fgbz/KXc8i4O19NrrE=; h=Resent-From:Resent-Date:Resent-To:Date:Subject:To:Cc:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=JesOPKow0mxa2jSkklwWv1W5a23mw59W0bjq2Btu78/3Neqlcr1v2kAi5/ixJRLQu Pmab/zBRrHTi97J+3NbDCJAyOm+QTaoZIq82PtYDx+XEMAB/vIhc8qJcjXhCILr4Ec t0/uC6ndR1B4C+cp5GDBqPgxOhvOtVKE4L4JKvIU= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2080.outbound.protection.outlook.com [40.107.102.80]) by sourceware.org (Postfix) with ESMTPS id 66FBE3858D35 for ; Mon, 6 Mar 2023 08:21:38 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 66FBE3858D35 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jGcpgjLoOZ2fTo7Y93DpXpvys6cWO7MeSizGMaQdS5A9C3BLOHhE1RFU9R5EcDKhsLtyYLO+jLSPn3ocPzq3IILU4triIlJ7HymIrpAbVi///g1t9UG1jlv0J/uSKWjFf+xoR8hxmSJjEkLg6fVRU4agQCZvUEP1okVuYxdtrqPTQfZuAZQwHtmBZZ2kXvbmHm50u/pytuReIW/mvgekwqxwlha7xvWtk4vHk75YuH6fDQ9MkRLMHE7Hs+hz97mk76prVJeMjuTDVGbJXmXVb43iRm8I+3xxq7adYKSxO5h2BGD4X+fGbQTyKhtNsg9vpiYAiG6E7AgZoo/iwvLaJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1O/KxpBKz3LYBrw/6vsbq3700Fgbz/KXc8i4O19NrrE=; b=b0vE/YlsUb8A7Q5sEP7pSrsCogCxq8kd9i9inmHy+NAuRmyfXMFpyYJTHSsp3aIe2GvTRSxMPK7TgbeZYfS70Xi7rJ98JFdTXL+Gfy3f2DT/Un256OlT2zCzRR4O2saUHHP982QiIpvnzmWJ5+P79kEXRCVaJkyK2b3j7SWVZCCutJQALTm38YPyVsCk05IGy/uPMmFuRAtE1gVSUB0YiPqFIktB6VvK/kvTzX2Hjvle0uT+iFHor8PdFE9SEJ9xTnxGcwanWpla2vNtuPQczXEZ4SKvPQzMVTvLSEGQ+i6UkAXhdTSrTUP/vFBMb2hDf8H4bqjLFHr/j3lEPjcyPQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=azul.com; dmarc=pass action=none header.from=azul.com; dkim=pass header.d=azul.com; arc=none Received: from DM6PR11MB4073.namprd11.prod.outlook.com (2603:10b6:5:19f::22) by PH0PR11MB7448.namprd11.prod.outlook.com (2603:10b6:510:26c::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6156.28; Mon, 6 Mar 2023 08:21:33 +0000 Received: from DM6PR11MB4073.namprd11.prod.outlook.com ([fe80::8476:1cf6:afeb:c285]) by DM6PR11MB4073.namprd11.prod.outlook.com ([fe80::8476:1cf6:afeb:c285%3]) with mapi id 15.20.6156.028; Mon, 6 Mar 2023 08:21:32 +0000 Resent-From: Jan Kratochvil Resent-Date: Mon, 6 Mar 2023 16:21:24 +0800 Resent-Message-ID: Resent-To: libc-alpha@sourceware.org, akozlov@azul.com Date: Mon, 6 Mar 2023 09:04:34 +0100 Subject: [PATCH] RFC: Provide a function to reset IFUNC PLTs To: libc-alpha@sourceware.org Cc: Anton Kozlov X-ClientProxiedBy: VI1PR07CA0242.eurprd07.prod.outlook.com (2603:10a6:802:58::45) To DM6PR11MB4073.namprd11.prod.outlook.com (2603:10b6:5:19f::22) Message-ID: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR11MB4073:EE_|PH0PR11MB7448:EE_ X-MS-Office365-Filtering-Correlation-Id: db7b087d-1233-4bc2-2782-08db1e1bcb3a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 74uL3id5MNX1nWY/pYj05albBG6jIo2Vf5VLL1JCmnnQriij3xgNowOMjfRRrfdvef64QFXBLnzBUXrJAwPOoeZlF+NAaplo0HnZP8OIPUsEGS1cFM0l2DOZ+MevqZWs0gvb3dAseGm6qGjpHxdYxKSMJEwXsbrV0Ms3Djqi9wGjUigRHU0sfMGr6GfwnAdn7AVb0bCSFBfmYHObscoWLzuO953mCwnI2tM1XDF8a5DVHQ6pRWYqsKq4w94s9kwVIz3tEVz9/ifDEzx02ssijbd5pOrNsSsRYp6hfwwTxr0Rd5DgCBy4MmugoZROCK+QLyDlVnRzWndvXzwoqxubAf2Fu87868AicNnV64f1XBgjv+lrbFq+NejiPa9CbOLd3j1+9JVW3LiXOYZFxP4mZs+xK7w6SLsQKB6XjLZwv5jEa8S6IpZl11hnvJFf5ia+PhluPUB6leSscmyeeBTM0zb8Bm603X55i9A2sfK/NI6UTZEKXhqoJz5+nkRg3nj8OGdr32OmGD/5Zv71sfkbmeYgUyQyP5kKuYfoRSxCGCFwJ4uO1/Z9CivIKHHscUxy0uGJxdJTC82Fh/wnx6bdT8OavLtVijvyJS6PtEIRH9vBcqc6XfvugpKVjmewJeqK8avUqbcHhMY3SDW8QBDqqbb+uP8Q09a6FqFYCD8w8RHSDIRewcX/eF+XD3y7QJnZ X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR11MB4073.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230025)(4636009)(366004)(39830400003)(136003)(346002)(396003)(376002)(451199018)(33656002)(4326008)(41300700001)(66946007)(66556008)(8676002)(6916009)(8936002)(2906002)(30864003)(5660300002)(86362001)(38100700002)(66476007)(7696005)(107886003)(966005)(478600001)(55016003)(316002)(52536014)(83380400001)(6506007)(9686003)(186003)(2004002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: A1rI6zcdKnKHHq+nap8DEJxb6f86z4KiHwnovqUIzjG60f+Rb1rauUQ/49DYINRXhK3YthX9dC27OmQl1YEbFXtEj7Cs1E7yYdaHabZMiY+S/46+6iaBzMxf8yofPPSIN5/s7sfPaD6+pXi3k3JQmkjKvjancyCOj+5Pl0eAf/ax1s46PQ9PGLhlNG2qmZ0XS7Gizcn59HWvLHL3FRZoQJVFMD9pl7C/iP7raL5ZjzeM7VVuVnVqJuAY5G24gbVvXFki8o/WG8DEYWX5/n1StZYIp6DQVBls4jYv0qIUVe3xDgnYL7248tmTerK9lb4zV9WSMpU2csLU/SbZZ+cPrf+24ATkehqWJH1Y8JAzFoqs0xq8fH8jQRDWRdtQl8AELSCUelOsaACYrEy8kvenDeL9sUN2o+AJiWL+C/JOfMTJr9JSLg/Nlwx5Q9e/CMGXgQ1HSNLjhn96VTtYMHKK8OTWbfgEm4Eg7wCPFW5+lPmeB2PDfASiEn+5B/VcpDrFMelWF70DhiWnJ1Yc4VQ8mkXdjrNekPETZr1HlzUMykkwvkuH2vlqAEwntxQTCZgo/gNpIHHdcGoebKrVH4bbl+W/yyhEKlTkGguhGV+tqu7VopfAsNeesxoKDTo/EUrT0y2hLpRLFTbw8+GYjlRrep1Dg1c269LvcInat+B3Xz/iI85ZMaMdDnGZ63lXUmZvHWf2Cmvr8bVS+P0FNtsKvMaSgjw3PHakGNilLS+aSYM5of1sBS8zBWtrWGKMWSINy1AOCEwiKugwSE77EopHKONAJbZQcHThVSxiyq35IyhKVFqxSnLkcDAq0Ox1J9R/761T5MiEimDC9PvVk+JVdTJsxtSo9oaLqojvMeLfSK1vhHZJG+O+cTHOMthUKefP6ymC+5A5iwfhSQE/o4hAf2UzePIUAD3GzBQVtueczboJoXcUDtfF5jXAmxTgAYHsDahxowfp3+eDA9ppQ7cYCe25pe0S0xb41F9TIHBmU1HEVuKATmhJgGIDyvWrDh5fVJHCaIDmjRUrzrUTbGs1Mo8zoUH4SKFe2Jn71on8CwF5mtdyztPTe1SCZY4o4l1D+0EOjCSgIjVmJy7aHdpIEZULHYmUqiy3k3mSiIEUrEcztfC4Bj8+Ec5jKxEFygiO9uhylv4si7Tm50NKm7G7bYFTFL2CsOqhkrHxEiEK3zJqIz4gZo6tHl10M3IHi2lL9EggxlnGzyJNmYjXPw0jAax75wUGcujcWf+ClWzmI8BMvbszQ7tFVuSLnr0i1OzB6NhkdgbPUOvp09umDexFm84D4fmi5pYDXQGl7/nE1zFEh0OxIDWPMELRLIgI2Xf9QDllyUrZB4PmQ2qwA3fc7lKX1MBFhvmUVR7BQz9syx3n+m0q7frfVde4xNxweL6LKGaTiq0dje8mwKs+AVl0i2Ccjxkv/gUABUQaJVXI0lXuGDgC2nBxkzqkehsxUX04pfdEiAsQkg5l5hkTmcJtAOSUaGo9n4UT3tyZ+YmtQ5bHugb/Gom4seY6NWKUyatpi1z56Q2M8EyZ0RRoqgzzKtkchzFzYRJgl7Q8nDWgL1/dEbqSydN5pVoVHPNa8Q6yjDWxP8wnbz90dXWfGyXLBw== X-OriginatorOrg: azul.com X-MS-Exchange-CrossTenant-Network-Message-Id: db7b087d-1233-4bc2-2782-08db1e1bcb3a X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4073.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Mar 2023 08:21:32.8093 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: c480eb31-2b17-43d7-b4c7-9bcb20cc4bf2 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: qKGVn68LfoBsVQdNm4WLvv6rVRUXLibUAetCtKwQsK33iKn/OcWkstzd/P7k9KgKcKjtEVHmSSg1SxTPx9YjtA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB7448 X-Spam-Status: No, score=-11.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jan Kratochvil via Libc-alpha From: Jan Kratochvil Reply-To: Jan Kratochvil Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" Some projects snapshot+restore process images to migrate them to a different machine. The target machine may have different (particularly lower) set of CPU features. Restored process does crash in glibc IFUNC functions which have been already set in PLTs due to the former more rich CPU features on the snapshotting machine. Providing a glibc function which can be called during the restore. I understand the code may need more adjustments before its upstreaming but is this an acceptable approach? An example of the problem reproducibility with: http://criu.org int main(void) { setbuf(stdout, NULL); for (;;) { time_t t = time(NULL); fputs(ctime(&t), stdout); sleep(strcmp("hello", "world") != 0); } } newcpu$ gcc -o strcmper strcmper.c -Wall -g newcpu$ rm -rf dir;mkdir dir;./strcmper &p=$!;sleep 1;sudo criu dump --shell-job -t $p -D dir oldcpu$ sudo criu restore --shell-job -D dir Segmentation fault newcpu$ gcc -o strcmper strcmper.c -Wall -g glibc/ld-linux-x86-64.so.2 glibc/libc.so.6 # patched files are in glibc/ newcpu$ rm -rf dir;mkdir dir;LD_LIBRARY_PATH=glibc ./glibc/ld-linux-x86-64.so.2 ./strcmper &p=$!;sleep 1;sudo criu dump --shell-job -t $p -D dir oldcpu$ sudo criu restore --shell-job -D dir running... --- elf/Makefile | 2 + elf/Versions | 9 + elf/dl-reloc.c | 2 + elf/dl-reset-ifunc.c | 392 +++++++++++++++++++++++++++++ elf/dl-tunables.c | 14 ++ elf/dl-tunables.h | 4 + elf/link.h | 2 + elf/tst-reset-ifunc.c | 70 ++++++ sysdeps/generic/ldsodefs.h | 4 +- sysdeps/x86/dl-get-cpu-features.c | 1 + sysdeps/x86/include/cpu-features.h | 3 +- 11 files changed, 500 insertions(+), 3 deletions(-) create mode 100644 elf/dl-reset-ifunc.c create mode 100644 elf/tst-reset-ifunc.c diff --git a/elf/Makefile b/elf/Makefile index 0d19964d42..1948d51407 100644 --- a/elf/Makefile +++ b/elf/Makefile @@ -39,6 +39,7 @@ routines = \ dl-origin \ dl-profstub \ dl-reloc-static-pie \ + dl-reset-ifunc \ dl-support \ dl-sym \ dl-sysdep \ @@ -309,6 +310,7 @@ tests := \ tst-auxv \ tst-dl-hash \ tst-leaks1 \ + tst-reset-ifunc \ tst-stringtable \ tst-tls9 \ # tests diff --git a/elf/Versions b/elf/Versions index 4614acea3e..f04f81efbf 100644 --- a/elf/Versions +++ b/elf/Versions @@ -23,6 +23,9 @@ libc { GLIBC_2.35 { _dl_find_object; } + GLIBC_2.38 { + _dl_reset_ifunc; + } GLIBC_ABI_DT_RELR { # This symbol is used only for empty version map and will be removed # by scripts/versions.awk. @@ -78,5 +81,11 @@ ld { # Set value of a tunable. __tunable_get_val; + + # For dl-reset-ifunc.c. + __tunable_get_start; + __tunable_get_end; + _dl_relocate_object; + _dl_x86_init_cpu_features; } } diff --git a/elf/dl-reloc.c b/elf/dl-reloc.c index 1d558c1e0c..7aed2d44fa 100644 --- a/elf/dl-reloc.c +++ b/elf/dl-reloc.c @@ -349,6 +349,8 @@ _dl_relocate_object (struct link_map *l, struct r_scope_elem *scope[], _dl_protect_relro (l); } +rtld_hidden_def (_dl_relocate_object) + void _dl_protect_relro (struct link_map *l) diff --git a/elf/dl-reset-ifunc.c b/elf/dl-reset-ifunc.c new file mode 100644 index 0000000000..04a17088e4 --- /dev/null +++ b/elf/dl-reset-ifunc.c @@ -0,0 +1,392 @@ +/* Reset IFUNC symbols for their new detection. + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include +#include + +/* Very special case: This object is built into the static libc, but + must know the layout of _rtld_global_ro. */ +#undef SHARED +#define SHARED +#include + +#include + +#if HAVE_TUNABLES +# define TUNABLES_INTERNAL 1 +# include "dl-tunables.h" +static tunable_t tunable_list[] attribute_relro __attribute_maybe_unused__; +#endif // HAVE_TUNABLES + +#define strcmp strcmp_local +static int strcmp_local(const char *s1, const char *s2) { + const unsigned char *a = (const unsigned char *) s1; + const unsigned char *b = (const unsigned char *) s2; + for (;; ++a, ++b) { + if (*a != *b) + return *b > *a ? +1 : -1; + if (*a == 0) + return 0; + } +} + +#define strchr strchr_local +static char *strchr_local(const char *s, int c) { + for (; *s; ++s) + if ((uint8_t)*s == (uint8_t)c) + return (char *)s; + return NULL; +} + +#define strlen strlen_local +__attribute__ ((optimize(0))) // otherwise GCC will replace the function by system strlen() again +static size_t strlen_local(const char *cs) { + size_t retval = 0; + while (*cs++) + ++retval; + return retval; +} + +// FIXME: Why volatile? +#define memset memset_local +static volatile void *memset_local(volatile void *m, int c, size_t n) { + for (volatile uint8_t *d = (volatile uint8_t *)m; n--; ++d) + *d = c; + return m; +} + +static char *file_read(const char *fn) { + int fd = open(fn, O_RDONLY); + assert(fd != -1); + // realloc() calls memmove(). + size_t buf_size = 0x100000; + char *buf = (char *)malloc(buf_size); + assert(buf); + size_t buf_have = 0; + for (;;) { + assert(buf_have < buf_size); + ssize_t got = read(fd, buf + buf_have, buf_size - buf_have); + assert(got != -1); + if (got == 0) + break; + assert(got > 0); + assert(buf_have + got <= buf_size); + buf_have += got; + } + assert(buf_have < buf_size); + buf[buf_have] = 0; + assert(strlen(buf) == buf_have); + return buf; +} + +static uint64_t read_hex(const char **cs_p) { + uint64_t retval = 0; + for (;;) +#define cs (*cs_p) + switch (*cs) { + case '0' ... '9': + retval <<= 4; + retval |= *cs++ - '0'; + continue; + case 'a' ... 'f': + retval <<= 4; + retval |= *cs++ - 'a' + 0xa; + continue; + default: + return retval; + break; + } +#undef cs +} + +static int mprotect_read(const void *addr, const void **addr_end_return) { + uint64_t addr_u = (uintptr_t)addr; + char *file = file_read("/proc/self/maps"); + int retval = -1; + for (const char *cs = file; *cs;) { + // sscanf() calls rawmemchr(). + uint64_t start = read_hex(&cs); + assert(*cs == '-'); + ++cs; + uint64_t end = read_hex(&cs); + assert(*cs == ' '); + ++cs; + assert(start < end); + int rwxp = 0; + assert(*cs == 'r' || *cs == '-'); + if (*cs++ == 'r') + rwxp |= 04; + assert(*cs == 'w' || *cs == '-'); + if (*cs++ == 'w') + rwxp |= 02; + assert(*cs == 'x' || *cs == '-'); + if (*cs++ == 'x') + rwxp |= 01; + assert(*cs == 's' || *cs == 'p'); + ++cs; + assert(*cs == ' '); + ++cs; + if (start <= addr_u && addr_u < end) { + if (addr_end_return) + *addr_end_return = (const void *)(uintptr_t)end; + retval = rwxp; + break; + } + cs = strchr(cs, '\n'); + assert(cs); + ++cs; + } + if (retval == -1) { + fprintf(stderr, "Not found an address: %p\n", addr); + assert(0); + } + free(file); + return retval; +} + +static void verify_rwxp(const void *start, const void *end, int rwxp_want) { + assert((((uintptr_t)start) & (PAGE_SIZE - 1)) == 0); + assert((((uintptr_t)end ) & (PAGE_SIZE - 1)) == 0); + assert(start < end); + while (start < end) { + int rwxp_found = mprotect_read(start, &start); + if (rwxp_found != rwxp_want) { printf("sudo gdb -p %d\n",getpid()); pause(); } + assert(rwxp_found == rwxp_want); + } +} + +const struct link_map *phdr_info_to_link_map(struct dl_phdr_info *phdr_info) { + Dl_info info; + const struct link_map *link_map = NULL; + int err = dladdr1(phdr_info->dlpi_phdr, &info, (void **)&link_map, RTLD_DL_LINKMAP); + assert(err == 1); + assert(link_map); + return link_map; +} + +static void page_align(const void **start_p, const void **end_p) { + *start_p = (const void *)(((uintptr_t)*start_p) & -PAGE_SIZE); + assert(*start_p); + *end_p = (const void *)((((uintptr_t)*end_p) + PAGE_SIZE - 1) & -PAGE_SIZE); +} + +static void readonly_unset(const void *start, const void *end) { + assert((((uintptr_t)start) & (PAGE_SIZE - 1)) == 0); + assert((((uintptr_t)end ) & (PAGE_SIZE - 1)) == 0); + assert(start <= end); + if (start == end) + return; + verify_rwxp(start, end, 04/*r--*/); + int err = mprotect((void *)start, (const uint8_t *)end - (const uint8_t *)start, PROT_READ | PROT_WRITE); + assert(!err); + verify_rwxp(start, end, 06/*rw-*/); +} + +static void readonly_reset(const void *start, const void *end) { + assert((((uintptr_t)start) & (PAGE_SIZE - 1)) == 0); + assert((((uintptr_t)end ) & (PAGE_SIZE - 1)) == 0); + assert(start <= end); + if (start == end) + return; + verify_rwxp(start, end, 06/*rw-*/); + int err = mprotect((void *)start, (const uint8_t *)end - (const uint8_t *)start, PROT_READ); + assert(!err); + verify_rwxp(start, end, 04/*r--*/); +} + +static void swap(const void **a_p, const void **b_p) { + const void *p = *a_p; + *a_p = *b_p; + *b_p = p; +} + +static void intersect(const void **first_start_p, const void **first_end_p, const void **second_start_p, const void **second_end_p) { + assert((((uintptr_t)*first_start_p) & (PAGE_SIZE - 1)) == 0); + assert((((uintptr_t)*first_end_p ) & (PAGE_SIZE - 1)) == 0); + assert(*first_start_p <= *first_end_p); + assert((((uintptr_t)*second_start_p) & (PAGE_SIZE - 1)) == 0); + assert((((uintptr_t)*second_end_p ) & (PAGE_SIZE - 1)) == 0); + assert(*second_start_p <= *second_end_p); + if (*first_start_p > *second_start_p) { + swap(first_start_p, second_start_p); + swap(second_start_p, second_start_p); + } + if (*second_start_p < *first_end_p) { + *second_start_p = *first_end_p; + if (*second_start_p > *second_end_p) + *second_end_p = *second_start_p; + } +} + +static int reset_ifunc_iterate_phdr(struct dl_phdr_info *info, size_t size, void *data_unused) { + if (strcmp(info->dlpi_name, "/lib64/ld-linux-x86-64.so.2") == 0) // _dl_relocate_object would crash on scope == NULL. + return 0; // unused + const void *relro = NULL; + const void *relro_end = NULL; + assert(size >= offsetof(struct dl_phdr_info, dlpi_adds)); + for (size_t phdr_ix = 0; phdr_ix < info->dlpi_phnum; ++phdr_ix) { + const Elf64_Phdr *phdr = info->dlpi_phdr + phdr_ix; + if (phdr->p_type == PT_GNU_RELRO) { + // It does not apply: assert(phdr->p_offset == phdr->p_vaddr); + assert(phdr->p_paddr == phdr->p_vaddr); + // /lib64/libz.so.1: p_filesz=0x538 > p_memsz=0x550 + assert(!relro); + relro = (const void *)(uintptr_t)(phdr->p_vaddr + info->dlpi_addr); + relro_end = (const void *)(((const uint8_t *)relro) + phdr->p_memsz); + page_align(&relro, &relro_end); + assert(relro); + } + } + if (relro) + readonly_unset(relro, relro_end); + const struct link_map *map = phdr_info_to_link_map(info); + Elf64_Dyn *dynamic = map->l_ld; + Elf64_Xword *relxsz_p = NULL; + Elf64_Xword *relrsz_p = NULL; + Elf64_Xword *relxcount_p = NULL; + for (; dynamic->d_tag != DT_NULL; ++dynamic) + switch (dynamic->d_tag) { + case DT_RELASZ: + case DT_RELSZ: + assert(!relxsz_p); + relxsz_p = &dynamic->d_un.d_val; + break; + case DT_RELRSZ: + assert(!relrsz_p); + relrsz_p = &dynamic->d_un.d_val; + break; + case DT_RELCOUNT: + case DT_RELACOUNT: + assert(!relxcount_p); + relxcount_p = &dynamic->d_un.d_val; + break; + case DT_PLTREL: + // It is impossible to relocate DT_REL twice. + assert(dynamic->d_un.d_val == DT_RELA); + break; + } + Elf64_Xword relxsz_saved = -1; // may be used uninitialized [-Werror=maybe-uninitialized] + if (relxsz_p) { + relxsz_saved = *relxsz_p; + *relxsz_p = 0; + } + Elf64_Xword relrsz_saved = -1; // may be used uninitialized [-Werror=maybe-uninitialized] + if (relrsz_p) { + relrsz_saved = *relrsz_p; + *relrsz_p = 0; + } + Elf64_Xword relxcount_saved = -1; // may be used uninitialized [-Werror=maybe-uninitialized] + if (relxcount_p) { + relxcount_saved = *relxcount_p; + *relxcount_p = 0; + } + struct link_map *link_map_p = (struct link_map *)map; + assert(link_map_p->l_relocated); + link_map_p->l_relocated = 0; + // FIXME: skip ifuncs + _dl_relocate_object((struct link_map *)map, link_map_p->l_scope, 0/*lazy*/, 0/*consider_profiling*/); + // It was read/write before but _dl_relocate_object made it read-only. + const void *dynamic_start = NULL; + const void *dynamic_end; + if (relxsz_p) { + dynamic_start = relxsz_p; + dynamic_end = relxsz_p + 1; + } + if (relrsz_p) { + if (!dynamic_start) { + dynamic_start = relrsz_p; + dynamic_end = relrsz_p + 1; + } else { + if (dynamic_start > (const void *)relrsz_p) + dynamic_start = relrsz_p; + if (dynamic_end < (const void *)(relrsz_p + 1)) + dynamic_end = relrsz_p + 1; + } + } + if (relxcount_p) { + if (!dynamic_start) { + dynamic_start = relxcount_p; + dynamic_end = relxcount_p + 1; + } else { + if (dynamic_start > (const void *)relxcount_p) + dynamic_start = relxcount_p; + if (dynamic_end < (const void *)(relxcount_p + 1)) + dynamic_end = relxcount_p + 1; + } + } + if (dynamic_start) { + page_align(&dynamic_start, &dynamic_end); + // _dl_relocate_object made it already readonly: readonly_reset(relro, relro_end); + intersect(&relro, &relro_end, &dynamic_start, &dynamic_end); + readonly_unset(relro, relro_end); + readonly_unset(dynamic_start, dynamic_end); + } + if (relxsz_p) + *relxsz_p = relxsz_saved; + if (relrsz_p) + *relrsz_p = relrsz_saved; + if (relxcount_p) + *relxcount_p = relxcount_saved; + if (dynamic_start) + readonly_reset(dynamic_start, dynamic_end); + if (relro) + readonly_reset(relro, relro_end); + return 0; // unused +} + +static void reset_glibc(void) { + const void *rtld_global_ro = &_rtld_global_ro; + const void *rtld_global_ro_end = &_rtld_global_ro + 1; + page_align(&rtld_global_ro, &rtld_global_ro_end); +#if HAVE_TUNABLES + const void *tunable_list = __tunable_get_start(); + const void *tunable_list_end = __tunable_get_end(); + page_align(&tunable_list, &tunable_list_end); + intersect(&rtld_global_ro, &rtld_global_ro_end, &tunable_list, &tunable_list_end); + readonly_unset(tunable_list, tunable_list_end); +#endif + readonly_unset(rtld_global_ro, rtld_global_ro_end); + const struct cpu_features *cpu_features_const = &GLRO(dl_x86_cpu_features); + // FIXME: Why volatile? + volatile struct cpu_features *cpu_features = (struct cpu_features *)cpu_features_const; + assert(cpu_features->basic.kind != arch_kind_unknown); + unsigned long int xsave_state_size = cpu_features->xsave_state_size; + unsigned int xsave_state_full_size = cpu_features->xsave_state_full_size; + memset((volatile uint8_t *)cpu_features, 0, sizeof(*cpu_features)); + cpu_features->xsave_state_size = xsave_state_size; + cpu_features->xsave_state_full_size = xsave_state_full_size; + assert(cpu_features->basic.kind == arch_kind_unknown); + _dl_x86_init_cpu_features(); + assert(cpu_features->basic.kind != arch_kind_unknown); + readonly_reset(rtld_global_ro, rtld_global_ro_end); +#if HAVE_TUNABLES + readonly_reset(tunable_list, tunable_list_end); +#endif +} + +void +_dl_reset_ifunc(void) +{ + // _dl_relocate_object() from reset_ifunc_iterate_phdr may be calling glibc ifunc resolvers already. + reset_glibc(); + int i = dl_iterate_phdr(reset_ifunc_iterate_phdr, NULL/*data*/); + assert(!i); +} +rtld_hidden_def (_dl_reset_ifunc) diff --git a/elf/dl-tunables.c b/elf/dl-tunables.c index 327b9eb52f..d8c0617bd1 100644 --- a/elf/dl-tunables.c +++ b/elf/dl-tunables.c @@ -433,3 +433,17 @@ __tunable_get_val (tunable_id_t id, void *valp, tunable_callback_t callback) } rtld_hidden_def (__tunable_get_val) + +const void * +__tunable_get_start(void) +{ + return &tunable_list; +} +rtld_hidden_def (__tunable_get_start) + +const void * +__tunable_get_end(void) +{ + return tunable_list + sizeof (tunable_list) / sizeof (tunable_t); +} +rtld_hidden_def (__tunable_get_end) diff --git a/elf/dl-tunables.h b/elf/dl-tunables.h index ae6e014b95..3e3783b9f7 100644 --- a/elf/dl-tunables.h +++ b/elf/dl-tunables.h @@ -56,10 +56,14 @@ extern void __tunables_print (void); extern void __tunable_get_val (tunable_id_t, void *, tunable_callback_t); extern void __tunable_set_val (tunable_id_t, tunable_val_t *, tunable_num_t *, tunable_num_t *); +extern const void *__tunable_get_start (void); +extern const void *__tunable_get_end (void); rtld_hidden_proto (__tunables_init) rtld_hidden_proto (__tunables_print) rtld_hidden_proto (__tunable_get_val) rtld_hidden_proto (__tunable_set_val) +rtld_hidden_proto (__tunable_get_start) +rtld_hidden_proto (__tunable_get_end) /* Define TUNABLE_GET and TUNABLE_SET in short form if TOP_NAMESPACE and TUNABLE_NAMESPACE are defined. This is useful shorthand to get and set diff --git a/elf/link.h b/elf/link.h index 3b5954d981..b30a736191 100644 --- a/elf/link.h +++ b/elf/link.h @@ -185,6 +185,8 @@ extern int dl_iterate_phdr (int (*__callback) (struct dl_phdr_info *, size_t, void *), void *__data); +extern void _dl_reset_ifunc (void); + /* Prototypes for the ld.so auditing interfaces. These are not defined anywhere in ld.so but instead have to be provided by the diff --git a/elf/tst-reset-ifunc.c b/elf/tst-reset-ifunc.c new file mode 100644 index 0000000000..8c185e269d --- /dev/null +++ b/elf/tst-reset-ifunc.c @@ -0,0 +1,70 @@ +/* Test resetting of IFUNC symbols for their new detection. + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include +#include +#include + +static int func_count; + +void +my_testifunc (void) +{ + puts ("my_testifunc"); + ++func_count; +} + +static int resolve_count; + +static void (* +resolve_testifunc (void)) (void) +{ + puts ("resolve_testifunc"); + ++resolve_count; + return (void (*)(void)) my_testifunc; // we'll just always select this routine +} + +void testifunc (void) __attribute__ ((ifunc ("resolve_testifunc"))); + +static int +do_test (void) +{ + // resolver could be called or not yet. + assert (func_count == 0); + puts("testifunc"); + testifunc (); + assert (resolve_count == 1); + assert (func_count == 1); + puts("_dl_reset_ifunc"); + _dl_reset_ifunc (); + // resolver could be called or not yet. + assert (func_count == 1); + puts("testifunc"); + testifunc (); + assert (resolve_count == 2); + assert (func_count == 2); + puts("testifunc"); + testifunc (); + assert (resolve_count == 2); + assert (func_count == 3); + return 0; +} + +#include diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h index c99dad77cc..5cf15a797d 100644 --- a/sysdeps/generic/ldsodefs.h +++ b/sysdeps/generic/ldsodefs.h @@ -1015,8 +1015,8 @@ extern struct link_map *_dl_new_object (char *realname, const char *libname, If RTLD_LAZY is set in RELOC-MODE, don't relocate its PLT. */ extern void _dl_relocate_object (struct link_map *map, struct r_scope_elem *scope[], - int reloc_mode, int consider_profiling) - attribute_hidden; + int reloc_mode, int consider_profiling); +rtld_hidden_proto (_dl_relocate_object) /* Protect PT_GNU_RELRO area. */ extern void _dl_protect_relro (struct link_map *map) attribute_hidden; diff --git a/sysdeps/x86/dl-get-cpu-features.c b/sysdeps/x86/dl-get-cpu-features.c index 5e3bb04d11..46fba307f0 100644 --- a/sysdeps/x86/dl-get-cpu-features.c +++ b/sysdeps/x86/dl-get-cpu-features.c @@ -68,6 +68,7 @@ Fatal glibc error: CPU does not support x86-64-v%d\n", 4); } } +rtld_hidden_def (_dl_x86_init_cpu_features) __ifunc (__x86_cpu_features, __x86_cpu_features, NULL, void, _dl_x86_init_cpu_features); #endif diff --git a/sysdeps/x86/include/cpu-features.h b/sysdeps/x86/include/cpu-features.h index fa91a23129..ad2bcc883f 100644 --- a/sysdeps/x86/include/cpu-features.h +++ b/sysdeps/x86/include/cpu-features.h @@ -929,7 +929,8 @@ extern const struct cpu_features *_dl_x86_get_cpu_features (void) /* Unused for x86. */ # define INIT_ARCH() # define _dl_x86_get_cpu_features() (&GLRO(dl_x86_cpu_features)) -extern void _dl_x86_init_cpu_features (void) attribute_hidden; +extern void _dl_x86_init_cpu_features (void); +rtld_hidden_proto (_dl_x86_init_cpu_features) #endif #ifdef __x86_64__