From patchwork Thu Dec 2 19:55:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Iain Sandoe X-Patchwork-Id: 48433 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id D553F3858036 for ; Thu, 2 Dec 2021 19:56:18 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org D553F3858036 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1638474978; bh=nkicLLHOvqut9OGjbUVHSAg+nQHA0VzaGQX/i1tgPjI=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=ccxiIRb7kRjBUptrSGMPhMKHFfn/9n+pzH72WEC/MYZsvK7LO0iOqSsrfAYnHxZsm jwKehBQyo1arqY8edPoC/GpTNKp7Z9FilGTKWDQoHmFQCuVMpuYYLD5i5O4S/ry0x9 L6vyEvGCM48/95rcahY+QA/9fT47RmrRTMk0/pGk= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by sourceware.org (Postfix) with ESMTPS id 7DFE03858001 for ; Thu, 2 Dec 2021 19:55:48 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 7DFE03858001 Received: by mail-wr1-x432.google.com with SMTP id l16so1012570wrp.11 for ; Thu, 02 Dec 2021 11:55:48 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :mime-version:content-transfer-encoding; bh=nkicLLHOvqut9OGjbUVHSAg+nQHA0VzaGQX/i1tgPjI=; b=KoRmjq2XcFYTpW05YlbGuDA4w1/kKwatl3y7uhrKYIzwVAxt4CUZU316L6tzAWRsze yAvqxDR3+IdNOQZiJ/2dSn23jFqwcfr5yQn/bwT6oqeXmnOc+dO1AvyV94cXw3i8n5pm 7butLqL5obZcGB8IV28gLapSYDsQrhUV9pqLDBFhBbqGmtf3pvqYS6qaJHON8vfwaEJ8 ZWhwbBVP/D6CJ98i+P1LMvRrs9S+LXH5s8DREjFnnvyHd+0+pW0+9S2cZFcDAf3I/+wn a2EdRgS0YWGoc6/yGGhYlq6LPeEPRmK6hdLFXbLWfUUc1jjlibLdNzH9RPx6DcfIKOZE TDiA== X-Gm-Message-State: AOAM532BkrGDsVtPiC27rayjcXC6PGxnKqwZf5EdSvQNg3aEdQtShI6E HkeDl2uWIpU0jY8qxGMMd8i8GLAmt/4= X-Google-Smtp-Source: ABdhPJyVHsVEqC/YByvcvcfZrb+GkKqmwc0E0HTFpNr8UIlDfEFDbl4RHP5MAod0mEJx8tFmEP3TAg== X-Received: by 2002:adf:db04:: with SMTP id s4mr16215018wri.467.1638474947393; Thu, 02 Dec 2021 11:55:47 -0800 (PST) Received: from localhost.localdomain (host81-138-1-83.in-addr.btopenworld.com. [81.138.1.83]) by smtp.gmail.com with ESMTPSA id k8sm654527wrn.91.2021.12.02.11.55.46 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 02 Dec 2021 11:55:47 -0800 (PST) X-Google-Original-From: Iain Sandoe To: gcc-patches@gcc.gnu.org Subject: [pushed ] Darwin: Rewrite host PCH support [PR 55610]. Date: Thu, 2 Dec 2021 19:55:40 +0000 Message-Id: <20211202195540.72712-1-iain@sandoe.co.uk> X-Mailer: git-send-email 2.24.3 (Apple Git-128) MIME-Version: 1.0 X-Spam-Status: No, score=-8.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Iain Sandoe via Gcc-patches From: Iain Sandoe Reply-To: iain@sandoe.co.uk Cc: Iain Sandoe , jakub@redhat.com Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" We need to revise the PCH memory allocation scheme to enable support for PIE on aarch64. The rewrite uses a similar scheme to the one used on Linux. We attempt to identify VM segments for each arch/OS version that are always available to the compiler (note this is not general, it only needs to work for the cc1* exes). If we cannot find the preferred segment we fall back to allowing the kernel to supply one - this is more likely to fail when the PCH read-in occurs (but that is trapped). In doing this we obviate the need to unmap any part of the compiler __DATA segment - thus fixing PR 55610. Notes: For 64bit hosts there is quite a large portion of the VMA that is apparently unused in current OS versions. We avoid the areas that are already being mapped by the sanitizers. For 32bit hosts the situation is not so easy - fortunately the requirement here is not general (we do not need to find an unused space for any arbitrary executable - only for the cases found in the cc1* compilers - which have relatively few external library DSO dependencies). Initial experiments suggest that the 64Bit host choices are OK for executables with ASLR enabled (PIE). However, it is doubtful whether a totally reliable choice can be found found for 32 bit hosts and PIE executables would not be recommended there. tested across the Darwin range over several iterations of master. pushed to master, thanks Iain Signed-off-by: Iain Sandoe gcc/ChangeLog: PR target/55610 * config/host-darwin.c (TRY_EMPTY_VM_SPACE, SAFE_ALLOC_SIZE): New. (darwin_gt_pch_get_address): Rewrite to use nominated memory segments rather than part of the compiler __DATA segment. (darwin_gt_pch_use_address): Likewise. --- gcc/config/host-darwin.c | 191 +++++++++++++++++++++++++++++++-------- 1 file changed, 153 insertions(+), 38 deletions(-) diff --git a/gcc/config/host-darwin.c b/gcc/config/host-darwin.c index 14a01fe71f2..559b919f2ee 100644 --- a/gcc/config/host-darwin.c +++ b/gcc/config/host-darwin.c @@ -20,62 +20,177 @@ #include "config.h" #include "system.h" #include "coretypes.h" +#include "options.h" #include "diagnostic-core.h" #include "config/host-darwin.h" - -/* Yes, this is really supposed to work. */ -/* This allows for a pagesize of 16384, which we have on Darwin20, but should - continue to work OK for pagesize 4096 which we have on earlier versions. - The size is 1 (binary) Gb. */ -static char pch_address_space[65536*16384] __attribute__((aligned (16384))); - -/* Return the address of the PCH address space, if the PCH will fit in it. */ +#include + +/* For Darwin (macOS only) platforms, without ASLR (PIE) enabled on the + binaries, the following VM addresses are expected to be available. + NOTE, that for aarch64, ASLR is always enabled - but the VM address + mentioned below is available (at least on Darwin20). + + The spaces should all have 512Mb available c.f. PCH files for large + C++ or Objective-C in the range of 150Mb for 64b hosts. + + We also try to steer clear of places already used for sanitizers. */ + +#define PAGE_SZ 4096 +#if defined(__x86_64) && defined(__LP64__) +# define TRY_EMPTY_VM_SPACE 0x180000000000ULL +# define SAFE_ALLOC_SIZE 0x20000000 +#elif defined(__x86_64) +# define TRY_EMPTY_VM_SPACE 0x00006fe00000ULL +# define SAFE_ALLOC_SIZE 0x20000000 +#elif defined(__i386) +# define TRY_EMPTY_VM_SPACE 0x00006fe00000ULL +# define SAFE_ALLOC_SIZE 0x20000000 +#elif defined(__POWERPC__) && defined(__LP64__) +# define TRY_EMPTY_VM_SPACE 0x180000000000ULL +# define SAFE_ALLOC_SIZE 0x20000000 +#elif defined(__POWERPC__) +# define TRY_EMPTY_VM_SPACE 0x00006fe00000ULL +# define SAFE_ALLOC_SIZE 0x20000000 +#elif defined(__aarch64__) +# undef PAGE_SZ +# define PAGE_SZ 16384 +# define TRY_EMPTY_VM_SPACE 0x180000000000ULL +# define SAFE_ALLOC_SIZE 0x20000000 +#else +# error "unknown Darwin target" +#endif + +/* Try to map a known position in the VM. The current PCH implementation + can adjust values at write-time, but not at read-time thus we need to + pick up the same position when reading as we got at write-time. */ void * -darwin_gt_pch_get_address (size_t sz, int fd ATTRIBUTE_UNUSED) +darwin_gt_pch_get_address (size_t sz, int fd) { - if (sz <= sizeof (pch_address_space)) - return pch_address_space; - else - return NULL; + if (sz > SAFE_ALLOC_SIZE) + { + error ("PCH memory request exceeds the available space"); + return NULL; + } + + /* Now try with the constraint that we really want this address... */ + void *addr = mmap ((void *)TRY_EMPTY_VM_SPACE, sz, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_FIXED, fd, 0); + + if (addr != (void *) MAP_FAILED) + munmap (addr, sz); + + /* This ought to be the only alternative to failure, but there are comments + that suggest some versions of mmap can be buggy and return a different + value. */ + if (addr == (void *) TRY_EMPTY_VM_SPACE) + return addr; + + warning (OPT_Winvalid_pch, "PCH memory [fixed at %p] is not available %m", + (void *) TRY_EMPTY_VM_SPACE); + + /* OK try to find a space without the constraint. */ + addr = mmap ((void *) TRY_EMPTY_VM_SPACE, sz, PROT_READ | PROT_WRITE, + MAP_PRIVATE, fd, 0); + + /* If we failed this time, that means there is *no* large enough free + space. */ + if (addr == (void *) MAP_FAILED) + { + error ("no memory is available for PCH : %m"); + return NULL; + } + + /* Unmap the area before returning. */ + munmap (addr, sz); + + /* If we got the exact area we requested, then that's great. */ + if (TRY_EMPTY_VM_SPACE && addr == (void *) TRY_EMPTY_VM_SPACE) + return addr; + + warning (OPT_Winvalid_pch, "PCH memory at %p is not available", + (void *) TRY_EMPTY_VM_SPACE); + + /* Otherwise, we need to try again but put some buffer space first. */ + size_t buffer_size = 32 * 1024 * 1024; + void *buffer = mmap (0, buffer_size, PROT_NONE, + MAP_PRIVATE | MAP_ANON, -1, 0); + addr = mmap ((void *)TRY_EMPTY_VM_SPACE, sz, PROT_READ | PROT_WRITE, + MAP_PRIVATE, fd, 0); + if (buffer != (void *) MAP_FAILED) + munmap (buffer, buffer_size); + + if (addr == (void *) MAP_FAILED) + { + error ("PCH memory not available %m"); + return NULL; + } + + warning (OPT_Winvalid_pch, "PCH memory at %p used instead", addr); + munmap (addr, sz); + return addr; } -/* Check ADDR and SZ for validity, and deallocate (using munmap) that part of - pch_address_space beyond SZ. */ +/* Try to mmap the PCH file at ADDR for SZ bytes at OFF offset in the file. + If we succeed return 1, if we cannot mmap the desired address, then we + fail with -1. */ int darwin_gt_pch_use_address (void *addr, size_t sz, int fd, size_t off) { - const size_t pagesize = getpagesize(); - void *mmap_result; - int ret; + void *mapped_addr; + + /* We're called with size == 0 if we're not planning to load a PCH + file at all. This allows the hook to free any static space that + we might have allocated at link time. */ + if (sz == 0) + return -1; - gcc_assert ((size_t)pch_address_space % pagesize == 0 - && sizeof (pch_address_space) % pagesize == 0); - - ret = (addr == pch_address_space && sz <= sizeof (pch_address_space)); - if (! ret) - sz = 0; + gcc_checking_assert (!(off % PAGE_SZ)); + if (addr != (void *) TRY_EMPTY_VM_SPACE) + warning (OPT_Winvalid_pch, "PCH at %p does not use the default position", + addr); - /* Round the size to a whole page size. Normally this is a no-op. */ - sz = (sz + pagesize - 1) / pagesize * pagesize; + /* Try to map the file with MAP_PRIVATE and FIXED. */ + mapped_addr = mmap (addr, sz, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_FIXED, fd, (off_t) off); - if (munmap (pch_address_space + sz, sizeof (pch_address_space) - sz) != 0) - fatal_error (input_location, - "could not unmap %: %m"); + /* Hopefully, we succeed. */ + if (mapped_addr == addr) + return 1; - if (ret) + warning (OPT_Winvalid_pch, "PCH private mmap of written position (%p)" + " failed [errno %d] %m", addr, errno); + + if (mapped_addr != (void *) MAP_FAILED) + munmap (mapped_addr, sz); + + /* Try to make an anonymous private mmap at the desired location. */ + mapped_addr = mmap (addr, sz, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANON | MAP_FIXED, -1, (off_t)0); + + if (mapped_addr != addr) { - mmap_result = mmap (addr, sz, - PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_FIXED, - fd, off); + warning (OPT_Winvalid_pch, "PCH anon mmap at written position (%p)" + " failed [errno %d] %m", addr, errno); + if (mapped_addr != (void *) MAP_FAILED) + munmap (mapped_addr, sz); + return -1; + } + + if (lseek (fd, off, SEEK_SET) == (off_t) -1) + return -1; - /* The file might not be mmap-able. */ - ret = mmap_result != (void *) MAP_FAILED; + while (sz) + { + ssize_t nbytes; - /* Sanity check for broken MAP_FIXED. */ - gcc_assert (!ret || mmap_result == addr); + nbytes = read (fd, addr, MIN (sz, (size_t) -1 >> 1)); + if (nbytes <= 0) + return -1; + addr = (char *) addr + nbytes; + sz -= nbytes; } - return ret; + return 1; }