Message ID | 20210609171618.20674-1-vivek@collabora.com |
---|---|
Headers |
Return-Path: <libc-alpha-bounces+patchwork=sourceware.org@sourceware.org> X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 6018039A1806 for <patchwork@sourceware.org>; Wed, 9 Jun 2021 17:19:47 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 6018039A1806 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1623259187; bh=/8Qu5Ht9oi+KM5jRFz6CjN0I2wjh2dqnP9voFEYMCf8=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=olE19/QCdifFH9SDUipln9+fQwAvMbyFQa4w0stLPcm7F5B/fD/H7RJpKv0h2W/A9 Fu0DTWmQooinMjYaZw+egjwJB6Vf+YTICc7K2NsXk/d6aYjbnXjsnM/D6a0WuIfSLX bI2dRWy3L6MBxWbVZlvz+aMmacX/kBT0JPAbQBkY= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by sourceware.org (Postfix) with ESMTPS id BAF05399C032 for <libc-alpha@sourceware.org>; Wed, 9 Jun 2021 17:16:30 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org BAF05399C032 Received: from noise.cbg.collabora.co.uk (unknown [IPv6:2001:4d48:ad5c:ef00:8e70:5aff:fe59:c29c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: vivek) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 50BB41F43568 for <libc-alpha@sourceware.org>; Wed, 9 Jun 2021 18:16:29 +0100 (BST) To: libc-alpha@sourceware.org Subject: [RFC][PATCH v11 0/7] Implementation of RTLD_SHARED for dlmopen Date: Wed, 9 Jun 2021 18:16:11 +0100 Message-Id: <20210609171618.20674-1-vivek@collabora.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-6.5 required=5.0 tests=BAYES_00, KAM_ASCII_DIVIDERS, KAM_DMARC_STATUS, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list <libc-alpha.sourceware.org> List-Unsubscribe: <https://sourceware.org/mailman/options/libc-alpha>, <mailto:libc-alpha-request@sourceware.org?subject=unsubscribe> List-Archive: <https://sourceware.org/pipermail/libc-alpha/> List-Post: <mailto:libc-alpha@sourceware.org> List-Help: <mailto:libc-alpha-request@sourceware.org?subject=help> List-Subscribe: <https://sourceware.org/mailman/listinfo/libc-alpha>, <mailto:libc-alpha-request@sourceware.org?subject=subscribe> From: =?utf-8?q?Vivek_Das=C2=A0Mohapatra_via_Libc-alpha?= <libc-alpha@sourceware.org> Reply-To: =?utf-8?q?Vivek_Das=C2=A0Mohapatra?= <vivek@collabora.com> Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" <libc-alpha-bounces+patchwork=sourceware.org@sourceware.org> |
Series |
Implementation of RTLD_SHARED for dlmopen
|
|
Message
Vivek Dasmohapatra
June 9, 2021, 5:16 p.m. UTC
This is a revision of a previous patchset that I posted here regarding https://sourceware.org/bugzilla/show_bug.cgi?id=22745 Introduction: ======================================================================= As discussed in the URL above dlmopen requires a mechanism for [optionally] sharing some objects between more than one namespace. The following patchset provides an implementation for this: If an object is loaded with the new RTLD_SHARED flag we instead ensure that a "master" copy exists (and is flagged as no-delete) in the main namespace and a thin wrapper or clone is placed in the target namespace. This patch series should address all the comments received on the earlier (v1-v10) series. ======================================================================= Changes from v10: - A segfault in a dlmopen error pathway (which does not seem to have existed when v10 was applied to the then-HEAD commit) has been fixed. - The fallback mechanism for adding DT_GNU_FLAGS_1 sections to the required binaries has had some infrastructure moved to the elf/ directory - The runstatedir setting introduced by autoconf has been omitted from the patchset as it is not relevant to this feature. Not changed: - There is still some diagnostic info in the config.log when the linker layout is acceptable but -z unique is not yet supported. I believe this _is_ useful diagnostic information as a developer might otherwise wonder why the vanilla linker was being rejected when its layout output seemed fine. - libpthread still has the new section applied as I'm not 100% sure it doesn't still need this treatment. I would like to investigate further before removing the new section from libpthread. I have not yet implemented, but plan to address once this series is accepted/acceptable: - Sensible RTLD_GLOBAL semantics for dlmopened DSOs in non-base namespaces - dl_iterate_ns_phdr (cf dl_iterate_phdr but taking a namespace argument) Vivek Das Mohapatra (7): Define a new dynamic section tag - DT_GNU_FLAGS_1 (bug 22745) Abstract loaded-DSO search code into a helper function Use the new DSO finder helper function Add DT_GNU_FLAGS_1/DF_GNU_1_UNIQUE to glibc DSOs (bug 22745) Implement dlmopen RTLD_SHARED flag (bug 22745) Add dlmopen / RTLD_SHARED tests Restore separate libc loading for the TLS/namespace storage test Makeconfig | 3 + Makerules | 18 +- bits/dlfcn.h | 10 + config.make.in | 1 + configure | 42 +- configure.ac | 31 +- elf/Makefile | 104 +++- elf/dl-close.c | 43 +- elf/dl-fini.c | 6 +- elf/dl-init.c | 4 +- elf/dl-load.c | 223 ++++++- elf/dl-object.c | 78 +++ elf/dl-open.c | 116 +++- elf/dl-sym.c | 14 + elf/dynamic-notes.c | 4 + elf/elf.h | 7 +- elf/get-dynamic-info.h | 12 + elf/rtld.c | 2 +- elf/tst-dlmopen-auditmod.c | 23 + elf/tst-dlmopen-common.h | 32 + elf/tst-dlmopen-main.h | 873 +++++++++++++++++++++++++++ elf/tst-dlmopen-modules.h | 21 + elf/tst-dlmopen-rtld-audit-shared1.c | 8 + elf/tst-dlmopen-rtld-audit-shared2.c | 8 + elf/tst-dlmopen-rtld-audit-shared3.c | 7 + elf/tst-dlmopen-rtld-audit-shared4.c | 8 + elf/tst-dlmopen-rtld-audit-shared5.c | 8 + elf/tst-dlmopen-rtld-audit-shared6.c | 8 + elf/tst-dlmopen-rtld-audit-unique1.c | 7 + elf/tst-dlmopen-rtld-audit-unique2.c | 7 + elf/tst-dlmopen-rtld-audit-unique3.c | 7 + elf/tst-dlmopen-rtld-audit-unique4.c | 7 + elf/tst-dlmopen-rtld-audit-unique5.c | 7 + elf/tst-dlmopen-rtld-audit-unique6.c | 7 + elf/tst-dlmopen-rtld-shared1.c | 7 + elf/tst-dlmopen-rtld-shared1.h | 64 ++ elf/tst-dlmopen-rtld-shared2.c | 7 + elf/tst-dlmopen-rtld-shared2.h | 66 ++ elf/tst-dlmopen-rtld-shared3.c | 7 + elf/tst-dlmopen-rtld-shared3.h | 43 ++ elf/tst-dlmopen-rtld-shared4.c | 7 + elf/tst-dlmopen-rtld-shared4.h | 14 + elf/tst-dlmopen-rtld-shared5.c | 7 + elf/tst-dlmopen-rtld-shared5.h | 25 + elf/tst-dlmopen-rtld-shared6.c | 7 + elf/tst-dlmopen-rtld-shared6.h | 36 ++ elf/tst-dlmopen-rtld-unique1.c | 7 + elf/tst-dlmopen-rtld-unique1.h | 86 +++ elf/tst-dlmopen-rtld-unique2.c | 7 + elf/tst-dlmopen-rtld-unique2.h | 25 + elf/tst-dlmopen-rtld-unique3.c | 7 + elf/tst-dlmopen-rtld-unique3.h | 13 + elf/tst-dlmopen-rtld-unique4.c | 7 + elf/tst-dlmopen-rtld-unique4.h | 14 + elf/tst-dlmopen-rtld-unique5.c | 7 + elf/tst-dlmopen-rtld-unique5.h | 58 ++ elf/tst-dlmopen-rtld-unique6.c | 7 + elf/tst-dlmopen-rtld-unique6.h | 51 ++ elf/tst-dlmopen-sharedmod-norm.c | 11 + elf/tst-dlmopen-sharedmod-uniq.c | 11 + elf/tst-dlmopen-std-do-test.c | 12 + elf/tst-tls-ie-dlmopen.c | 4 +- extra-lib.mk | 3 + htl/Makefile | 3 + iconvdata/Makefile | 3 + iconvdata/extra-module.mk | 4 + include/elf.h | 2 + include/link.h | 7 +- nptl/Makefile | 7 +- sysdeps/generic/ldsodefs.h | 9 + sysdeps/mips/bits/dlfcn.h | 10 + 71 files changed, 2349 insertions(+), 72 deletions(-) create mode 100644 elf/dynamic-notes.c create mode 100644 elf/tst-dlmopen-auditmod.c create mode 100644 elf/tst-dlmopen-common.h create mode 100644 elf/tst-dlmopen-main.h create mode 100644 elf/tst-dlmopen-modules.h create mode 100644 elf/tst-dlmopen-rtld-audit-shared1.c create mode 100644 elf/tst-dlmopen-rtld-audit-shared2.c create mode 100644 elf/tst-dlmopen-rtld-audit-shared3.c create mode 100644 elf/tst-dlmopen-rtld-audit-shared4.c create mode 100644 elf/tst-dlmopen-rtld-audit-shared5.c create mode 100644 elf/tst-dlmopen-rtld-audit-shared6.c create mode 100644 elf/tst-dlmopen-rtld-audit-unique1.c create mode 100644 elf/tst-dlmopen-rtld-audit-unique2.c create mode 100644 elf/tst-dlmopen-rtld-audit-unique3.c create mode 100644 elf/tst-dlmopen-rtld-audit-unique4.c create mode 100644 elf/tst-dlmopen-rtld-audit-unique5.c create mode 100644 elf/tst-dlmopen-rtld-audit-unique6.c create mode 100644 elf/tst-dlmopen-rtld-shared1.c create mode 100644 elf/tst-dlmopen-rtld-shared1.h create mode 100644 elf/tst-dlmopen-rtld-shared2.c create mode 100644 elf/tst-dlmopen-rtld-shared2.h create mode 100644 elf/tst-dlmopen-rtld-shared3.c create mode 100644 elf/tst-dlmopen-rtld-shared3.h create mode 100644 elf/tst-dlmopen-rtld-shared4.c create mode 100644 elf/tst-dlmopen-rtld-shared4.h create mode 100644 elf/tst-dlmopen-rtld-shared5.c create mode 100644 elf/tst-dlmopen-rtld-shared5.h create mode 100644 elf/tst-dlmopen-rtld-shared6.c create mode 100644 elf/tst-dlmopen-rtld-shared6.h create mode 100644 elf/tst-dlmopen-rtld-unique1.c create mode 100644 elf/tst-dlmopen-rtld-unique1.h create mode 100644 elf/tst-dlmopen-rtld-unique2.c create mode 100644 elf/tst-dlmopen-rtld-unique2.h create mode 100644 elf/tst-dlmopen-rtld-unique3.c create mode 100644 elf/tst-dlmopen-rtld-unique3.h create mode 100644 elf/tst-dlmopen-rtld-unique4.c create mode 100644 elf/tst-dlmopen-rtld-unique4.h create mode 100644 elf/tst-dlmopen-rtld-unique5.c create mode 100644 elf/tst-dlmopen-rtld-unique5.h create mode 100644 elf/tst-dlmopen-rtld-unique6.c create mode 100644 elf/tst-dlmopen-rtld-unique6.h create mode 100644 elf/tst-dlmopen-sharedmod-norm.c create mode 100644 elf/tst-dlmopen-sharedmod-uniq.c create mode 100644 elf/tst-dlmopen-std-do-test.c
Comments
* Vivek Das Mohapatra via Libc-alpha: > - libpthread still has the new section applied as I'm not 100% sure it doesn't > still need this treatment. I would like to investigate further before > removing the new section from libpthread. libpthread does not need this anymore. We can safely load it as many times as we want. librt still needs it because it's not empty and has global state (for handling the thread notifications). Thanks, Florian
On Wed, 9 Jun 2021, Florian Weimer wrote: >> - libpthread still has the new section applied as I'm not 100% sure it doesn't >> still need this treatment. I would like to investigate further before > > libpthread does not need this anymore. We can safely load it as many > times as we want. Ok, thanks. I'll update the patch series.
I'm testing these patches for our use case in GraalVM. We use dlmopen to load several isolated JVMs within the same process. With the patches I'm hitting the following error: Inconsistency detected by ld.so: dl-version.c: 205: _dl_check_map_versions: Assert ion `needed != NULL' failed! Here's the tail of the LD_DEBUG=all log: 222389: trying file=./nptl/tls/x86_64/libpthread.so.0 222389: trying file=./nptl/tls/libpthread.so.0 222389: trying file=./nptl/x86_64/x86_64/libpthread.so.0 222389: trying file=./nptl/x86_64/libpthread.so.0 222389: trying file=./nptl/x86_64/libpthread.so.0 222389: trying file=./nptl/libpthread.so.0 222389: 222389: checking for version `GLIBC_2.2.5' in file ./dlfcn/libdl.so.2 [0] required by file /home/mukel/Desktop/graal/graal/sdk/mxbuild/linux-amd64/GRAALVM_E SPRESSO_JVM_EE_JAVA11/graalvm-espresso-jvm-ee-java11-21.2.0-dev/lib/libnio.so [1] 222389: checking for version `GLIBC_2.2.5' in file ./nptl/libpthread.so.0 [0] required by file /home/mukel/Desktop/graal/graal/sdk/mxbuild/linux-amd64/GRAAL VM_ESPRESSO_JVM_EE_JAVA11/graalvm-espresso-jvm-ee-java11-21.2.0-dev/lib/libnio.so [1] 222389: checking for version `GLIBC_2.3' in file ./libc.so.6 [0] required by file /home/mukel/Desktop/graal/graal/sdk/mxbuild/linux-amd64/GRAALVM_ESPRESSO_J VM_EE_JAVA11/graalvm-espresso-jvm-ee-java11-21.2.0-dev/lib/libnio.so [1] 222389: checking for version `GLIBC_2.3.2' in file ./libc.so.6 [0] require d by file /home/mukel/Desktop/graal/graal/sdk/mxbuild/linux-amd64/GRAALVM_ESPRESSO _JVM_EE_JAVA11/graalvm-espresso-jvm-ee-java11-21.2.0-dev/lib/libnio.so [1] 222389: checking for version `GLIBC_2.4' in file ./libc.so.6 [0] required by file /home/mukel/Desktop/graal/graal/sdk/mxbuild/linux-amd64/GRAALVM_ESPRESSO_J VM_EE_JAVA11/graalvm-espresso-jvm-ee-java11-21.2.0-dev/lib/libnio.so [1] 222389: checking for version `GLIBC_2.2.5' in file ./libc.so.6 [0] require d by file /home/mukel/Desktop/graal/graal/sdk/mxbuild/linux-amd64/GRAALVM_ESPRESSO _JVM_EE_JAVA11/graalvm-espresso-jvm-ee-java11-21.2.0-dev/lib/libnio.so [1] Inconsistency detected by ld.so: dl-version.c: 205: _dl_check_map_versions: Assert ion `needed != NULL' failed! We use dlmopen a lot, but never seen this error before, hopefully you can pinpoint the issue. The program in question runs fine without the patches. I'm running with ./testrun.sh . I can provide a reproducer, but the setup is quite cumbersome. I can also test any further patches locally if needed. We've been following this feature for a long time (with great excitement) and just wanted to make sure it works (or that at least doesn't break us) before it's merged. Best, Alfonso
On Wed, 16 Jun 2021, Alfonso Alfonso Peterssen wrote: > I'm testing these patches for our use case in GraalVM. We use dlmopen to load several isolated JVMs within the same process. > > With the patches I'm hitting the following error: > Inconsistency detected by ld.so: dl-version.c: 205: _dl_check_map_versions: Assert > ion `needed != NULL' failed! Digging into it. If I can't figure it out I'll ask you for a reproducer.