From patchwork Wed Nov 17 21:03:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Iain Sandoe X-Patchwork-Id: 47854 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id B840D3858409 for ; Wed, 17 Nov 2021 21:06:53 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org B840D3858409 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1637183213; bh=z1mjVFkg//AsStub2mTNU8PpSXjyGRxmXBn5wXE4Zwg=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=egmVwahwMQH8OrH+WKL0sQr6MsYBbu/RgZcHZWMJxuhDfqmgbYwQOiCg4pJ4DOptZ OpvlVt0F/7afuVgjU337Wxlp728K/a0oauulMwNTPO+IHKP5M5GEfcUYvbf8STmBMh GWeemDfQWyppuk53lkFr3v1YNFakw1TPF+phyhOk= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by sourceware.org (Postfix) with ESMTPS id DBB403858402 for ; Wed, 17 Nov 2021 21:03:26 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org DBB403858402 Received: by mail-wm1-x32e.google.com with SMTP id f7-20020a1c1f07000000b0032ee11917ceso3176110wmf.0 for ; Wed, 17 Nov 2021 13:03:26 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :mime-version:content-transfer-encoding; bh=z1mjVFkg//AsStub2mTNU8PpSXjyGRxmXBn5wXE4Zwg=; b=FBchSgj6HgneHEX3N/fTKkzpEwcwL9XGucemVcMhoaRUPm3QLrHLDBGpeLPiziw4lI ASwwC/xN8cyNquK/Peb/y19BZtxWKZpBSeuILaoBSMTltsOxMU2P9RyB14J/Y4iYwQtJ BOZcJCXmzeu9rXCMYam3yzQqSBMX+TCc07xFJngHcJF9El/bsz4l7rsLMQQOHz+yAYjI qNrYHMnmFxv0h7j/UNDFcMEyeIviiqksUN973rSzAFBMxXxEu1ZYgqC2ac9ytsx8yyEX rfiM3KuNVS5BSMJR00Tor8CH30/Wf+8EKt8s7+g3zYviONUmQqOCrPiebNhzu+UJ/rWp q4XA== X-Gm-Message-State: AOAM531lfaih6hrmpQWwYzsz+i7uouUXCMjtLYmsxIbccMW/v0/8aJQ4 6O3GR98Bi5AjsoXpaATB4rT/aIjmesI= X-Google-Smtp-Source: ABdhPJy7QMXx/vVIcB/R5QZ9o84saWurS/vikK4ysGfIQf1YGSbRlyv2lgKV2QHOfv3jJDWhBmS/Lw== X-Received: by 2002:a1c:9851:: with SMTP id a78mr3218584wme.116.1637183005734; Wed, 17 Nov 2021 13:03:25 -0800 (PST) Received: from localhost.localdomain (host81-138-1-83.in-addr.btopenworld.com. [81.138.1.83]) by smtp.gmail.com with ESMTPSA id m17sm945118wrz.22.2021.11.17.13.03.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 17 Nov 2021 13:03:25 -0800 (PST) X-Google-Original-From: Iain Sandoe To: gcc-patches@gcc.gnu.org Subject: [PATCH 0/4] Darwin: Replace environment runpath with embedded [PR88590]. Date: Wed, 17 Nov 2021 21:03:15 +0000 Message-Id: <20211117210319.92514-1-iain@sandoe.co.uk> X-Mailer: git-send-email 2.24.3 (Apple Git-128) MIME-Version: 1.0 X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Iain Sandoe via Gcc-patches From: Iain Sandoe Reply-To: iain@sandoe.co.uk Cc: Iain Sandoe , joseph@codesourcery.com Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Sender: "Gcc-patches" This is a fairly long explanation of the problems being addressed by the patch set. Most of the changes are Darwin-specific - a change to the libtool component allowing for this @rpath and some minor additions to makefiles where libtool is not in use. At present, this seems pretty specific to the GCC build; since we depend on accessing newly-built components during the bootstrap. There are additional details relevant to each patch in its own commit message. ===== Darwin builds shared libraries with information on the runpath as part of the library name. For example, /installation/path/for/libfoo.dylib. That is regarded as two components; the 'runpath' /installation/path/for/ and the library name libfoo.dylib. This means that (at runtime) two libraries with the same name can be disambiguated by their runpaths, and potentially used by the same exe. ===== Problem #1 That is fine and works well; until we disturb the assumptions by overriding the library runpath with an environment one: DYLD_LIBRARY_PATH. Now the library runpath(s) can be discarded and the libraries are first searched on the basis of that provided by the environment; two libraries with the same name are no longer distinct (if a library with that name is found in the environment path). This causes problems in configuring, building and testing GCC because we set the runpath environment at a very high level so that it applies to stage1+ target configures and stage2+ host configures. This is needed so that executables built during those configures get the newly-built libgcc_s when the target defaults to using a shared libgcc. However, it also means that every tool that is used during the configure has its libgcc_s (or any of the newly-built bootstrapped libs) overriden to use the new one(s) - which might be buggy. In the testsuite it is more serious - since more target libs come into play - especially libstdc++. Several system tools on Darwin use(d) libc++ and that has caused wrong or crashed test output. In principle, LD_LIBRARY_PATH on Linux has the same issue - although perhaps there is less tendency to default to use of shared dependent libs. Ideally, one would have several environment paths, and some way to use the appropriate one at the appropriate time. I experimented with this as a solution to both this and the following problem, but it proved unrealistic - since the process would have to be applied to all relevant OSS projects using auto-tools to be safe - and mostly the uninstalled use of libraries is a GCC build-time issue. ===== Problem #2 A change in security policy for Darwin means that DYLD_LIBRARY_PATH is now removed from the environment for all system tools (e.g. /usr/sh, env etc). This means that for all realistic build steps that use any system utility (like sh) will no longer see the the environment runpath and the only ones available will be those in the libraries. This breaks GCC's configuration since the steps mentioned above are now not seeing the newly-built shared libraries, but actually much olders ones installed on the system. It means that for all Darwin15+ we misconfigure libstdc++. /bin/sh is hardwired into autoconf, one cannot use CONFIG_SHELL to work around this - because /bin/sh is invoked first, and then passes control to CONFIG_SHELL. A second problem is that we cannot bump the SO name for libgcc_s (which I need to do to solve an EH problem) - since the new SO name is not available on the system, and therefore none of the stage1+ target configures will succeed. This is because the eventual install path is correctly encoded into the built library, but it is not present at the install position (and, in general, cannot be installed - since that might not even be a suitable path on the build system). This has also meant that we could not do in-tree testing without first installing the target libraries (which is mostly inconvenient rather than a show-stopper, but still). ===== Tested solution. Darwin has the ability to make the runpaths install-position independent. One sets the library runpath to @rpath/ which essentially means "use the runpath available at the time we want to load this". One can then add (potentially multiple) runpaths to the executable, the built library can be put anywhere convenient - providing we can put that path into the exe. For GCC's build, test and install process this means that we need at each stage to build exes with the runpaths that are relevant to the positions of the dependent libraries. To do this, we add an rpath for each of the startfile paths. While we are building/testing GCC these correspond to (for example gcc/ or /libstdc++/src/.libs etc) and then, after the compiler is installed at its intended install path - these become /compiler/installation/path/lib etc. I have tested this widely on i686, powerpc, x86_64 and aarch64 Darwin over more than a year. So patch 1 : provides a spec that expands to -rpath xxx for each xxx in the startfiles (which means that this works for anything added as -B and thus the testsuite also works without installing the target libs again). The remaining patches implement the Darwin changes to build the target libraries with @rpath runpaths - this is the largest part of the patch in bytes - because the libtool change expands. I would like to get this in early in stage 3 so that it has time to be kicked around before we branch. OK for master? (ideally, we'd also backport to open branches because the breakage is quite significant) thanks Iain Iain Sandoe (4): Driver : Provide a spec to insert rpaths for compiler lib dirs. Darwin : Handle rpaths given on the command line. Darwin : Allow for configuring Darwin to use embedded runpath. Darwin, Ada : Add loader path as a default rpath. configure | 5 + configure.ac | 5 + fixincludes/config.h.in | 204 --------------------------- fixincludes/configure | 2 +- gcc/aclocal.m4 | 50 +++++++ gcc/ada/gcc-interface/Makefile.in | 2 + gcc/config/darwin-driver.c | 18 +++ gcc/config/darwin.h | 11 +- gcc/config/darwin.opt | 4 + gcc/configure | 157 +++++++++++++++++++-- gcc/gcc.c | 18 +++ libatomic/Makefile.am | 6 +- libatomic/Makefile.in | 5 +- libatomic/configure | 82 ++++++++++- libatomic/testsuite/Makefile.in | 1 + libbacktrace/configure | 80 ++++++++++- libcc1/Makefile.am | 9 ++ libcc1/Makefile.in | 12 +- libcc1/configure | 157 +++++++++++++++++++-- libffi/Makefile.am | 7 +- libffi/Makefile.in | 6 +- libffi/configure | 157 +++++++++++++++++++-- libgcc/config.host | 22 ++- libgcc/config/t-darwin-rpath | 5 + libgcc/config/t-slibgcc-darwin | 13 +- libgfortran/Makefile.am | 3 + libgfortran/Makefile.in | 30 ++-- libgfortran/configure | 155 ++++++++++++++++++-- libgfortran/configure.ac | 4 +- libgomp/Makefile.am | 6 +- libgomp/Makefile.in | 3 +- libgomp/configure | 151 ++++++++++++++++++-- libitm/Makefile.am | 5 +- libitm/Makefile.in | 3 +- libitm/configure | 171 ++++++++++++++++++++--- libobjc/configure | 96 +++++++++++-- libobjc/configure.ac | 14 +- liboffloadmic/configure | 199 +++++++++++++++++++++----- liboffloadmic/plugin/Makefile.in | 2 +- liboffloadmic/plugin/aclocal.m4 | 2 +- liboffloadmic/plugin/configure | 199 +++++++++++++++++++++----- libphobos/configure | 151 ++++++++++++++++++-- libphobos/libdruntime/Makefile.am | 5 +- libphobos/libdruntime/Makefile.in | 3 +- libphobos/src/Makefile.am | 5 +- libphobos/src/Makefile.in | 3 +- libquadmath/Makefile.am | 6 +- libquadmath/Makefile.in | 4 +- libquadmath/configure | 225 +++++++++++++++++++++++++++++- libquadmath/configure.ac | 2 + libsanitizer/asan/Makefile.am | 6 +- libsanitizer/asan/Makefile.in | 5 +- libsanitizer/configure | 157 +++++++++++++++++++-- libsanitizer/hwasan/Makefile.am | 6 +- libsanitizer/hwasan/Makefile.in | 5 +- libsanitizer/lsan/Makefile.am | 7 +- libsanitizer/lsan/Makefile.in | 6 +- libsanitizer/tsan/Makefile.am | 6 +- libsanitizer/tsan/Makefile.in | 5 +- libsanitizer/ubsan/Makefile.am | 6 +- libsanitizer/ubsan/Makefile.in | 5 +- libssp/Makefile.am | 5 +- libssp/Makefile.in | 3 +- libssp/configure | 82 ++++++++++- libstdc++-v3/configure | 169 +++++++++++++++++++--- libstdc++-v3/src/Makefile.am | 6 +- libstdc++-v3/src/Makefile.in | 3 +- libtool.m4 | 64 ++++++++- libvtv/configure | 171 ++++++++++++++++++++--- lto-plugin/Makefile.in | 1 - lto-plugin/configure | 80 ++++++++++- zlib/configure | 82 ++++++++++- 72 files changed, 2832 insertions(+), 533 deletions(-) create mode 100644 libgcc/config/t-darwin-rpath