Subject: [PATCH] Linux: Add membarrier system call wrapper
To: libc-alpha@sourceware.org
2017-12-20 Florian Weimer <fweimer@redhat.com>
Linux: Add membarrier system call wrapper.
* manual/memory.texi (Memory): Reference Memory Barriers.
(Memory Barriers): New node.
* sysdeps/unix/sysv/linux/Makefile [misc] (sysdep_routines): Add
membarrier.
[misc] (tests): Add tst-membarrier.
* sysdeps/unix/sysv/linux/Versions (GLIBC_2.17): Export membarrier.
* sysdeps/unix/sysv/linux/bits/mman-shared.h
(MEMBARRIER_CMD_QUERY, MEMBARRIER_CMD_SHARED)
(MEMBARRIER_CMD_PRIVATE_EXPEDITED)
(MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED): Define.
(membarrier): Declare.
* sysdeps/unix/sysv/linux/syscalls.list (membarrier): Add.
* sysdeps/unix/sysv/linux/tst-membarrier.c: New file.
* sysdeps/unix/sysv/linux/**.abilist: Update.
@@ -19,6 +19,7 @@ and allocation of real memory.
* Resizing the Data Segment:: @code{brk}, @code{sbrk}
* Memory Protection:: Controlling access to memory regions.
* Locking Pages:: Preventing page faults
+* Memory Barriers:: System support for concurrent programming.
@end menu
Memory mapped I/O is not discussed in this chapter. @xref{Memory-mapped I/O}.
@@ -3839,3 +3840,99 @@ full, and when it becomes 95% full. Above 95% you get another warning
each time memory usage increases.
@end ignore
+
+@node Memory Barriers
+@section Memory Barriers
+@cindex fences
+@cindex barriers
+@cindex memory barriers
+@cindex concurrency barriers
+
+This section discusses facilities which implement certain forms of
+memory barriers which can be used in the implementation of concurrent
+algorithms. Note that these barriers can be very costly and do not by
+themselves ensure the correct operation of concurrent algorithms.
+
+The barriers discussed here do not imply compiler barriers, which have
+to be supplied separately by the programmer (for example, using
+@samp{atomic_signal_fence (memory_order_acq_rel)} in C11 and later, or
+@samp{std::atomic_signal_fence (std::memory_order_acq_rel)} in C++11 and
+later). Alternatively, you can use the GCC builtin,
+@samp{__atomic_signal_fence (__ATOMIC_ACQ_REL)}. @xref{__atomic
+Builtins, __atomic Builtins, Built-in Functions for Memory Model Aware
+Atomic Operations, gcc, The GNU Compiler Collection}.
+
+@deftypefun int membarrier (int @var{op}, int @var{flags})
+@standards{Linux, sys/mman.h}
+@safety{@prelim{}@mtsafe{}@assafe{}@acsafe{}}
+This function performs operations related to memory barriers, depending
+on the value of @var{op}.
+
+The @var{flags} argument is currently reserved and must be zero.
+
+For the operations which perform a memory barrier, there is a set of
+participating threads. For these threads, the @code{membarrier}
+function call turns an existing compiler barrier (see above) executed by
+these threads into full memory barriers (of the
+@samp{atomic_thread_fence (memory_order_acq_rel)} variety).
+
+The @var{op} argument is one of the following values. Except for
+@code{MEMBARRIER_CMD_QUERY}, these values are powers of two:
+
+@vtable @code
+@item MEMBARRIER_CMD_QUERY
+@standards{Linux, sys/mman.h}
+
+Return a bitmap of the @code{MEMBARRIER_CMD_}* operations supported by
+this system. An application can check whether a particular
+@code{MEMBARRIER_CMD_}* (except @code{MEMBARRIER_CMD_QUERY} itself) is
+supported by calling @samp{membarrier (MEMBARRIER_CMD_QUERY, 0)} and
+checking if the @code{MEMBARRIER_CMD_}* bit is set in the result.
+
+@item MEMBARRIER_CMD_SHARED
+@standards{Linux, sys/mman.h}
+
+This command performs a memory barrier, and the set of participating
+threads consists of all threads in the system. On success, the return
+value is @math{0}.
+
+@item MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED
+@standards{Linux, sys/mman.h}
+
+This command advertises that this process might perform a
+@code{MEMBARRIER_CMD_PRIVATE_EXPEDITED} memory in the future. It does
+not itself perform a barrier. On success, the return value is @math{0}.
+
+@item MEMBARRIER_CMD_PRIVATE_EXPEDITED
+@standards{Linux, sys/mman.h}
+
+This command performs a memory barrier, and the set of participating
+threads consists of all threads belonging to the current process. This
+type of barrier is less costly than the system-wide barrier provided by
+@code{MEMBARRIER_CMD_SHARED}.
+
+This command fails unless it has previously been enabled using the
+@code{MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED} command.
+
+On success, the return value is @math{0}.
+@end vtable
+
+The @code{membarrier} function returns @math{-1} on failure. The
+following @code{errno} values are specific to this function:
+
+@table @code
+@item ENOSYS
+The system does not implement this function.
+
+@item EINVAL
+The @var{op} argument is not valid, or the @var{flags} argument is not
+zero.
+
+@item EPERM
+An attempt was made to perform a @code{MEMBARRIER_CMD_PRIVATE_EXPEDITED}
+memory barrier without first registering the intent using
+@code{MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED}.
+
+@end table
+
+@end deftypefun
@@ -18,7 +18,8 @@ sysdep_routines += clone umount umount2 readahead \
setfsuid setfsgid epoll_pwait signalfd \
eventfd eventfd_read eventfd_write prlimit \
personality epoll_wait tee vmsplice splice \
- open_by_handle_at mlock2 pkey_mprotect pkey_set pkey_get
+ open_by_handle_at mlock2 pkey_mprotect pkey_set pkey_get \
+ membarrier
CFLAGS-gethostid.c = -fexceptions
CFLAGS-tee.c = -fexceptions -fasynchronous-unwind-tables
@@ -44,7 +45,8 @@ sysdep_headers += sys/mount.h sys/acct.h sys/sysctl.h \
tests += tst-clone tst-clone2 tst-clone3 tst-fanotify tst-personality \
tst-quota tst-sync_file_range tst-sysconf-iov_max tst-ttyname \
- test-errno-linux tst-memfd_create tst-mlock2 tst-pkey
+ test-errno-linux tst-memfd_create tst-mlock2 tst-pkey \
+ tst-membarrier \
# Generate the list of SYS_* macros for the system calls (__NR_*
# macros). The file syscall-names.list contains all possible system
@@ -170,6 +170,7 @@ libc {
memfd_create;
mlock2;
pkey_alloc; pkey_free; pkey_set; pkey_get; pkey_mprotect;
+ membarrier;
}
GLIBC_PRIVATE {
# functions used in other libraries
@@ -2106,6 +2106,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -2017,6 +2017,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -107,6 +107,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -39,6 +39,14 @@
# define PKEY_DISABLE_WRITE 0x2
# endif
+/* Command values for membarrier. */
+# ifndef MEMBARRIER_CMD_QUERY
+# define MEMBARRIER_CMD_QUERY 0
+# define MEMBARRIER_CMD_SHARED 1
+# define MEMBARRIER_CMD_PRIVATE_EXPEDITED 8
+# define MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED 16
+# endif
+
__BEGIN_DECLS
/* Create a new memory file descriptor. NAME is a name for debugging.
@@ -71,6 +79,9 @@ int pkey_free (int __key) __THROW;
range. */
int pkey_mprotect (void *__addr, size_t __len, int __prot, int __pkey) __THROW;
+/* Perform a memory barrier on multiple threads. */
+int membarrier (int __op, int __flags) __THROW;
+
__END_DECLS
#endif /* __USE_GNU */
@@ -1871,6 +1871,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -2036,6 +2036,7 @@ GLIBC_2.26 wcstof128_l F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -1900,6 +1900,7 @@ GLIBC_2.26 wcstof128_l F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -108,6 +108,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -1985,6 +1985,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -2106,6 +2106,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -1960,6 +1960,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -1958,6 +1958,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -1956,6 +1956,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -1951,6 +1951,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -2147,6 +2147,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -1989,6 +1989,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -1994,6 +1994,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -2201,6 +2201,7 @@ GLIBC_2.26 wcstof128_l F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -108,6 +108,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -1989,6 +1989,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -1890,6 +1890,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -1875,6 +1875,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -1982,6 +1982,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -1919,6 +1919,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -112,3 +112,4 @@ process_vm_writev EXTRA process_vm_writev i:ipipii process_vm_writev
memfd_create EXTRA memfd_create i:si memfd_create
pkey_alloc EXTRA pkey_alloc i:ii pkey_alloc
pkey_free EXTRA pkey_free i:i pkey_free
+membarrier EXTRA membarrier i:ii membarrier
@@ -2113,6 +2113,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -2113,6 +2113,7 @@ GLIBC_2.26 reallocarray F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
new file mode 100644
@@ -0,0 +1,61 @@
+/* Tests for the membarrier function.
+ Copyright (C) 2017 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <errno.h>
+#include <stdio.h>
+#include <support/check.h>
+#include <sys/mman.h>
+
+static int
+do_test (void)
+{
+ int supported = membarrier (MEMBARRIER_CMD_QUERY, 0);
+ if (supported == -1)
+ {
+ if (errno == ENOSYS)
+ FAIL_UNSUPPORTED ("membarrier system call not iplemented");
+ else
+ FAIL_EXIT1 ("membarrier: %m");
+ }
+
+ if (supported & MEMBARRIER_CMD_SHARED)
+ {
+ puts ("info: MEMBARRIER_CMD_SHARED supported");
+ TEST_COMPARE (membarrier (MEMBARRIER_CMD_SHARED, 0), 0);
+ }
+ else
+ puts ("info: MEMBARRIER_CMD_SHARED not supported");
+
+ /* If the private-expedited barrier is advertised, execute it after
+ registering the intent. */
+ if (supported & MEMBARRIER_CMD_PRIVATE_EXPEDITED)
+ {
+ puts ("info: MEMBARRIER_CMD_PRIVATE_EXPEDITED supported");
+ TEST_COMPARE (supported & MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED,
+ MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED);
+ TEST_COMPARE (membarrier (MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED, 0),
+ 0);
+ TEST_COMPARE (membarrier (MEMBARRIER_CMD_PRIVATE_EXPEDITED, 0), 0);
+ }
+ else
+ puts ("info: MEMBARRIER_CMD_PRIVATE_EXPEDITED not supported");
+
+ return 0;
+}
+
+#include <support/test-driver.c>
@@ -1877,6 +1877,7 @@ GLIBC_2.26 wcstof128_l F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F
@@ -2120,6 +2120,7 @@ GLIBC_2.26 wcstof128_l F
GLIBC_2.27 GLIBC_2.27 A
GLIBC_2.27 glob F
GLIBC_2.27 glob64 F
+GLIBC_2.27 membarrier F
GLIBC_2.27 memfd_create F
GLIBC_2.27 mlock2 F
GLIBC_2.27 pkey_alloc F