From patchwork Mon Dec 12 11:46:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Bugaev X-Patchwork-Id: 55355 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id EAE523884CBF for ; Mon, 12 Dec 2022 11:47:25 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org EAE523884CBF DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1670845646; bh=pdm+yRM3sdZRmzV6m35U/NBnq5cuktqxNwRsVrBe51g=; h=To:Cc:Subject:Date:List-Id:List-Unsubscribe:List-Archive: List-Post:List-Help:List-Subscribe:From:Reply-To:From; b=Dy3r/LmyooV4SF9CyQ3BQP06hJr/LPNdxgjzLVntjUUNGMsldvmCtsnVYtxWaa01u WSBgXmyU9FIYhKeuvFiSBn5KnZBTH4SBJiL9v9Jl06tVT/C20yyPzQySKBF4BsdLgX KnK2ogTHZ1UMZ5tvTvyyxgYsTXPulXmPtlg6E/HQ= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com [IPv6:2a00:1450:4864:20::136]) by sourceware.org (Postfix) with ESMTPS id 0F1A13855173 for ; Mon, 12 Dec 2022 11:47:00 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 0F1A13855173 Received: by mail-lf1-x136.google.com with SMTP id c1so18100441lfi.7 for ; Mon, 12 Dec 2022 03:46:59 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=pdm+yRM3sdZRmzV6m35U/NBnq5cuktqxNwRsVrBe51g=; b=WrNxPgtJOrcCbkr9e9V7LxeJX8ZH8s3pzdbzTQA+HLp9SFS3YXGZuDsIngRF21PjRN X7/t1O/DEpEvfeITnc2Cw/qLY9OU8+F4be5LFB0R6t39oq4GvXMvzw/3yL8SV5dP3qXF pptNfeC/kO6kqpcB2EZ1+wEoyknZs1R2GeTLvy0n+obCSEYZos9v1Z0nZOg8RJBytTP1 Y0S+oWMgn2IzSOAaPeALxY8NLJ17o1pPQUnXMnRHntOdZSuo9mFirinhzWFEoc9ZZQhk /7SEDUUt5pn8lN0d0QqETWaeHPsnrlzgdxtW0obhYWOi8D6spBll/WBMcicLyfPjRus9 q6ig== X-Gm-Message-State: ANoB5pkQbz7KzGAVCgj/eP3FCSMmDRa2jXS84+QGZmhQU/BUdmLrWJsb GOkxcDBfKiS7+ZlNN2/Xb6A= X-Google-Smtp-Source: AA0mqf4/sxz62lauWV1GZ4T9T7WHqhArJRjYrsyKyu+MmQbEuw4kXfpG8qdYdydu30n3Wly/3Kwl+g== X-Received: by 2002:a05:6512:1386:b0:4b5:4606:7ad9 with SMTP id p6-20020a056512138600b004b546067ad9mr5230792lfa.39.1670845618524; Mon, 12 Dec 2022 03:46:58 -0800 (PST) Received: from surface-pro-6.. ([194.190.106.50]) by smtp.gmail.com with ESMTPSA id g42-20020a0565123baa00b00497b198987bsm1606514lfv.26.2022.12.12.03.46.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Dec 2022 03:46:57 -0800 (PST) To: bug-hurd@gnu.org, libc-alpha@sourceware.org, samuel.thibault@gnu.org Cc: Sergey Bugaev Subject: [RFC PATCH 0/3] O_TMPFILE and SHM_ANON for the Hurd Date: Mon, 12 Dec 2022 14:46:33 +0300 Message-Id: <20221212114636.74222-1-bugaevc@gmail.com> X-Mailer: git-send-email 2.38.1 MIME-Version: 1.0 X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Sergey Bugaev via Libc-alpha From: Sergey Bugaev Reply-To: Sergey Bugaev Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" This is an attempt to add O_TMPFILE (from Linux) and SHM_ANON (from FreeBSD) to the Hurd. The Hurd already has the very handly dir_mkfile API that lets you create anonymous/unnamed files -- ones that can either be used as just tmp files, given to other processes via fork or IPC, or linked into the filesystem to become regular, named files. This little gem is used by glibc internally to implement atomic creation/ replacement of some files (/etc/hostname and the like, symlinks, etc), as well as for the tmpfile () function (which makes a tmp file under $TMPDIR), but is not otherwise exposed to user programs through a Unix-level API; you have to drop down to Mach/Hurd level if you want to use it. But there is a Unix-level API for this functionality in Linux: the O_TMPFILE flag to open () & co, where you pass the parent directory path w/o a basename to open () along with O_TMPFILE, and that makes an anonymous file "in that directory's filesystem". So I thought it would make sense to expose the Hurd's dir_mkfile by supporting the same O_TMPFILE flag in our open (). Then, Linux also has memfd_create (), which makes a temp file without touching any fs at all. This API is widely used in the Wayland ecosystem as the recommended way to create shared memory. But such an approach would not work as-is on the Hurd: in order for there to be an fd, there has to be a server somewhere, servicing that fd. We can't just make an fd out of "pure memory" -- it may be an fd to /hurd/tmpfs, but that /hurd/tmpfs needs to exist and be accessible. So being usable in an empty chroot is not going to happen anyway, unless we start spawning our own instances of /hurd/tmpfs for each memfd, which sounds like a terrible idea. And so in that light, the FreeBSD alternative to memfd_create () -- namely SHM_ANON -- sounds much more approachable to me, while being, well, a bit less widely supported in the Wayland ecosystem than memfd, but still quite popular. We already implement shm by creating files under /dev/shm/ (which should normally be a tmpfs, but for some reason on my Debian GNU/Hurd box it does not seem to be?), so it seems only natural to use the just-introduced O_TMPFILE to create anonymous shm files there. So that is my motivation for adding O_TMPFILE and SHM_ANON. That being said, maybe we don't actually want these? tmpfile () already gets you most of the way there, and the fact that it creates a file under /tmp/ and not /dev/shm/ is not *that* important. As for the implementation: basically all of the SHM_ANON implementation, except the very definition, ended up in the generic / POSIX version of shm_open () and __shm_get_name (), predicated on defined (SHM_ANON) && defined (O_TMPFILE). This sounds problematic: while there is indeed nothing Hurd-specific about the implementation, and any port that supports O_TMPFILE and wants to support SHM_ANON could use these code paths, what if another port wants to implement SHM_ANON differently? Should I make a separate copy of shm_open.c in sysdeps/mach/hurd instead of modifying the generic version? I've used the following simple program to test shm_open () with SHM_ANON: #include #include #include #include int main() { int fd = shm_open(SHM_ANON, O_RDWR | O_CREAT, 0600); printf("Opened fd %d\n", fd); write(fd, "hello", 5); lseek(fd, SEEK_SET, 0); char buffer[6] = { 0 }; read(fd, buffer, 5); printf("Read %s\n", buffer); } and the following to test O_TMPFILE: #include #include #include #include #include int main() { int fd = open("/tmp", O_TMPFILE | O_RDWR, 0700); if (fd < 0) error(1, errno, "open(O_TMPFILE)"); write(fd, "Hello", 5); int r = fchown(fd, -1, 24); if (r < 0) error(0, errno, "fchown"); r = linkat(fd, "", AT_FDCWD, "/tmp/hello.txt", AT_EMPTY_PATH); if (r < 0) error(1, errno, "linkat"); } After running the latter, /tmp/hello.txt should appear, owned by $(id -u):24, (where 24 is the group 'cdrom' on my system, chosen arbitrarily among those that my user is a member of), containing "hello". Importantly, it does so atomically, i.e. these properties are immediately visible once it appears. Sergey Bugaev (3): hurd: Consolidate file_name_lookup implementation hurd: Implement O_TMPFILE hurd: Implement SHM_ANON hurd/hurdlookup.c | 10 +---- hurd/lookup-at.c | 69 ++++++++++++++++++++++++------- posix/shm-directory.c | 25 +++++++++-- rt/shm_open.c | 5 +++ sysdeps/mach/hurd/bits/fcntl.h | 5 +++ sysdeps/mach/hurd/bits/mman_ext.h | 25 +++++++++++ 6 files changed, 112 insertions(+), 27 deletions(-) create mode 100644 sysdeps/mach/hurd/bits/mman_ext.h