[v5,01/22] nptl: Fix Race conditions in pthread cancellation [BZ#12683]

Message ID 20230410195907.4123869-2-adhemerval.zanella@linaro.org
State Superseded
Headers
Series Fix Race conditions in pthread cancellation [BZ#12683] |

Checks

Context Check Description
dj/TryBot-apply_patch success Patch applied to master at the time it was sent

Commit Message

Adhemerval Zanella Netto April 10, 2023, 7:58 p.m. UTC
  The current racy approach is to enable asynchronous cancellation
before making the syscall and restore the previous cancellation
type once the syscall returns, and check if cancellation has happen
during the cancellation entrypoint.

As described in BZ#12683, this approach shows 2 problems:

  1. Cancellation can act after the syscall has returned from the
     kernel, but before userspace saves the return value.  It might
     result in a resource leak if the syscall allocated a resource or a
     side effect (partial read/write), and there is no way to program
     handle it with cancellation handlers.

  2. If a signal is handled while the thread is blocked at a cancellable
     syscall, the entire signal handler runs with asynchronous
     cancellation enabled.  This can lead to issues if the signal
     handler call functions which are async-signal-safe but not
     async-cancel-safe.

For the cancellation to work correctly, there are 5 points at which the
cancellation signal could arrive:

  1. Before the final "testcancel" and before the syscall is made.
  2. Between the "testcancel" and the syscall.
  3. While the syscall is blocked and no side effects have yet taken
     place.
  4. While the syscall is blocked but with some side effects already
     having taken place (e.g. a partial read or write).
  5. After the syscall has returned.

And libc wants to act on cancellation in cases 1, 2, and 3 but not
in cases 4 or 5.  For the 4 and 5 cases, the cancellation will eventually
happen in the next cancellable entrypoint without any further external
event.

The proposed solution for each case is:

  1. Do a conditional branch based on whether the thread has received
     a cancellation request;

  2. It can be caught by the signal handler determining that the saved
     program counter (from the ucontext_t) is in some address range
     beginning just before the "testcancel" and ending with the
     syscall instruction.

  3. In this case, except for certain syscalls that ALWAYS fail with
     EINTR even for non-interrupting signals, the kernel will reset
     the program counter to point at the syscall instruction during
     signal handling, so that the syscall is restarted when the signal
     handler returns.  So, from the signal handler's standpoint, this
     looks the same as case 2, and thus it's taken care of.

  4. For syscalls with side-effects, the kernel cannot restart the
     syscall; when it's interrupted by a signal, the kernel must cause
     the syscall to return with whatever partial result is obtained
     (e.g. partial read or write).

  5. The saved program counter points just after the syscall
     instruction, so the signal handler won't act on cancellation.
     This is similar to 4. since the program counter is past the syscall
     instruction.

So The proposed fixes are:

  1. Remove the enable_asynccancel/disable_asynccancel function usage in
     cancellable syscall definition and instead make them call a common
     symbol that will check if cancellation is enabled (__syscall_cancel
     at nptl/cancellation.c), call the arch-specific cancellable
     entry-point (__syscall_cancel_arch), and cancel the thread when
     required.

  2. Provide an arch-specific generic system call wrapper function
     that contains global markers.  These markers will be used in
     SIGCANCEL signal handler to check if the interruption has been
     called in a valid syscall and if the syscalls has side-effects.

     A reference implementation sysdeps/unix/sysv/linux/syscall_cancel.c
     is provided.  However, the markers may not be set on correct
     expected places depending on how INTERNAL_SYSCALL_NCS is
     implemented by the architecture.  It is expected that all
     architectures add an arch-specific implementation.

  3. Rewrite SIGCANCEL asynchronous handler to check for both canceling
     type and if current IP from signal handler falls between the global
     markers and act accordingly.

  4. Adjust libc code to replace LIBC_CANCEL_ASYNC/LIBC_CANCEL_RESET to
     appropriated cancelable syscalls.

  5. Adjust 'lowlevellock-futex.h' arch-specific implementations to
     provide cancelable futex calls.

This patch adds the proposed changes to NPTL common code and the
following patches add the requires arch-specific bits.  The build for
ia64-linux-gnu, mips-*, and x86_64-* are broken without the
arch-specific patches.
---
 elf/Makefile                             |   5 +-
 nptl/Makefile                            |  11 +-
 nptl/cancellation.c                      | 112 +++++++++---------
 nptl/cleanup_defer.c                     |   5 +-
 nptl/descr-const.sym                     |   6 +
 nptl/descr.h                             |  18 +++
 nptl/libc-cleanup.c                      |   5 +-
 nptl/pthread_cancel.c                    |  78 +++++--------
 nptl/pthread_exit.c                      |   4 +-
 nptl/pthread_setcancelstate.c            |   2 +-
 nptl/pthread_setcanceltype.c             |   2 +-
 nptl/pthread_testcancel.c                |   5 +-
 nptl/tst-cancel31.c                      | 100 ++++++++++++++++
 sysdeps/generic/syscall_types.h          |  25 ++++
 sysdeps/nptl/cancellation-pc-check.h     |  54 +++++++++
 sysdeps/nptl/lowlevellock-futex.h        |  20 +---
 sysdeps/nptl/pthreadP.h                  |  11 +-
 sysdeps/unix/sysdep.h                    | 140 +++++++++++++++++------
 sysdeps/unix/sysv/linux/socketcall.h     |  35 ++++--
 sysdeps/unix/sysv/linux/syscall_cancel.c |  71 ++++++++++++
 sysdeps/unix/sysv/linux/sysdep-cancel.h  |  12 --
 21 files changed, 525 insertions(+), 196 deletions(-)
 create mode 100644 nptl/descr-const.sym
 create mode 100644 nptl/tst-cancel31.c
 create mode 100644 sysdeps/generic/syscall_types.h
 create mode 100644 sysdeps/nptl/cancellation-pc-check.h
 create mode 100644 sysdeps/unix/sysv/linux/syscall_cancel.c
  

Patch

diff --git a/elf/Makefile b/elf/Makefile
index 396ec51424..b684e5d44c 100644
--- a/elf/Makefile
+++ b/elf/Makefile
@@ -1252,11 +1252,8 @@  $(objpfx)dl-allobjs.os: $(all-rtld-routines:%=$(objpfx)%.os)
 # discovery mechanism is not compatible with the libc implementation
 # when compiled for libc.
 rtld-stubbed-symbols = \
-  __GI___pthread_disable_asynccancel \
-  __GI___pthread_enable_asynccancel \
+  __syscall_cancel \
   __libc_assert_fail \
-  __pthread_disable_asynccancel \
-  __pthread_enable_asynccancel \
   calloc \
   free \
   malloc \
diff --git a/nptl/Makefile b/nptl/Makefile
index 8cec6faee3..bce033179e 100644
--- a/nptl/Makefile
+++ b/nptl/Makefile
@@ -204,6 +204,7 @@  routines = \
   sem_timedwait \
   sem_unlink \
   sem_wait \
+  syscall_cancel \
   tpp \
   unwind \
   vars \
@@ -234,7 +235,8 @@  CFLAGS-pthread_setcanceltype.c += -fexceptions -fasynchronous-unwind-tables
 
 # These are internal functions which similar functionality as setcancelstate
 # and setcanceltype.
-CFLAGS-cancellation.c += -fasynchronous-unwind-tables
+CFLAGS-cancellation.c += -fexceptions -fasynchronous-unwind-tables
+CFLAGS-syscall_cancel.c += -fexceptions -fasynchronous-unwind-tables
 
 # Calling pthread_exit() must cause the registered cancel handlers to
 # be executed.  Therefore exceptions have to be thrown through this
@@ -286,7 +288,7 @@  tests = tst-attr2 tst-attr3 tst-default-attr \
 	tst-sem17 \
 	tst-tsd3 tst-tsd4 \
 	tst-cancel4_1 tst-cancel4_2 \
-	tst-cancel7 tst-cancel17 tst-cancel24 \
+	tst-cancel7 tst-cancel17 tst-cancel24 tst-cancel31 \
 	tst-signal3 \
 	tst-exec4 tst-exec5 \
 	tst-stack2 tst-stack3 tst-stack4 \
@@ -339,7 +341,10 @@  xtests += tst-eintr1
 
 test-srcs = tst-oddstacklimit
 
-gen-as-const-headers = unwindbuf.sym
+gen-as-const-headers = \
+  descr-const.sym \
+  unwindbuf.sym \
+  # gen-as-const-headers
 
 gen-py-const-headers := nptl_lock_constants.pysym
 pretty-printers := nptl-printers.py
diff --git a/nptl/cancellation.c b/nptl/cancellation.c
index 765511d66d..eee5b6b758 100644
--- a/nptl/cancellation.c
+++ b/nptl/cancellation.c
@@ -18,74 +18,78 @@ 
 #include <setjmp.h>
 #include <stdlib.h>
 #include "pthreadP.h"
-#include <futex-internal.h>
 
-
-/* The next two functions are similar to pthread_setcanceltype() but
-   more specialized for the use in the cancelable functions like write().
-   They do not need to check parameters etc.  These functions must be
-   AS-safe, with the exception of the actual cancellation, because they
-   are called by wrappers around AS-safe functions like write().*/
-int
-__pthread_enable_asynccancel (void)
+/* Called by the INTERNAL_SYSCALL_CANCEL macro, check for cancellation and
+   returns the syscall value or its negative error code.  */
+long int
+__internal_syscall_cancel (__syscall_arg_t a1, __syscall_arg_t a2,
+			   __syscall_arg_t a3, __syscall_arg_t a4,
+			   __syscall_arg_t a5, __syscall_arg_t a6,
+			   __syscall_arg_t nr)
 {
-  struct pthread *self = THREAD_SELF;
-  int oldval = atomic_load_relaxed (&self->cancelhandling);
+  long int result;
+  struct pthread *pd = THREAD_SELF;
 
-  while (1)
+  /* If cancellation is not enabled, call the syscall directly and also
+     for thread terminatation to avoid call __syscall_do_cancel while
+     executing cleanup handlers.  */
+  int ch = atomic_load_relaxed (&pd->cancelhandling);
+  if (SINGLE_THREAD_P || !cancel_enabled (ch) || cancel_exiting (ch))
     {
-      int newval = oldval | CANCELTYPE_BITMASK;
-
-      if (newval == oldval)
-	break;
+      result = INTERNAL_SYSCALL_NCS_CALL (nr, a1, a2, a3, a4, a5, a6);
+      if (INTERNAL_SYSCALL_ERROR_P (result))
+	return -INTERNAL_SYSCALL_ERRNO (result);
+      return result;
+    }
 
-      if (atomic_compare_exchange_weak_acquire (&self->cancelhandling,
-						&oldval, newval))
-	{
-	  if (cancel_enabled_and_canceled_and_async (newval))
-	    {
-	      self->result = PTHREAD_CANCELED;
-	      __do_cancel ();
-	    }
+  /* Call the arch-specific entry points that contains the globals markers
+     to be checked by SIGCANCEL handler.  */
+  result = __syscall_cancel_arch (&pd->cancelhandling, nr, a1, a2, a3, a4, a5,
+			          a6);
 
-	  break;
-	}
-    }
+  /* If the cancellable syscall was interrupted by SIGCANCEL and it has not
+     side-effect, cancel the thread if cancellation is enabled.  */
+  ch = atomic_load_relaxed (&pd->cancelhandling);
+  if (result == -EINTR && cancel_enabled_and_canceled (ch))
+    __syscall_do_cancel ();
 
-  return oldval;
+  return result;
 }
-libc_hidden_def (__pthread_enable_asynccancel)
 
-/* See the comment for __pthread_enable_asynccancel regarding
-   the AS-safety of this function.  */
-void
-__pthread_disable_asynccancel (int oldtype)
+/* Called by the SYSCALL_CANCEL macro, check for cancellation and return the
+   syscall expected success value (usually 0) or, in case of failure, -1 and
+   sets errno to syscall return value.  */
+long int
+__syscall_cancel (__syscall_arg_t a1, __syscall_arg_t a2,
+		  __syscall_arg_t a3, __syscall_arg_t a4,
+		  __syscall_arg_t a5, __syscall_arg_t a6,
+		  __syscall_arg_t nr)
 {
-  /* If asynchronous cancellation was enabled before we do not have
-     anything to do.  */
-  if (oldtype & CANCELTYPE_BITMASK)
-    return;
+  int r = __internal_syscall_cancel (a1, a2, a3, a4, a5, a6, nr);
+  return __glibc_unlikely (INTERNAL_SYSCALL_ERROR_P (r))
+	 ? SYSCALL_ERROR_LABEL (INTERNAL_SYSCALL_ERRNO (r))
+	 : r;
+}
 
+/* Called by __syscall_cancel_arch or function above start the thread
+   cancellation.  */
+_Noreturn void
+__syscall_do_cancel (void)
+{
   struct pthread *self = THREAD_SELF;
-  int newval;
+
+  /* Disable thread cancellation to avoid cancellable entrypoints to call
+     __syscall_do_cancel recursively.  */
   int oldval = atomic_load_relaxed (&self->cancelhandling);
-  do
+  while (1)
     {
-      newval = oldval & ~CANCELTYPE_BITMASK;
+      int newval = oldval | CANCELSTATE_BITMASK;
+      if (oldval == newval)
+	break;
+      if (atomic_compare_exchange_weak_acquire (&self->cancelhandling,
+						&oldval, newval))
+	break;
     }
-  while (!atomic_compare_exchange_weak_acquire (&self->cancelhandling,
-						&oldval, newval));
 
-  /* We cannot return when we are being canceled.  Upon return the
-     thread might be things which would have to be undone.  The
-     following loop should loop until the cancellation signal is
-     delivered.  */
-  while (__glibc_unlikely ((newval & (CANCELING_BITMASK | CANCELED_BITMASK))
-			   == CANCELING_BITMASK))
-    {
-      futex_wait_simple ((unsigned int *) &self->cancelhandling, newval,
-			 FUTEX_PRIVATE);
-      newval = atomic_load_relaxed (&self->cancelhandling);
-    }
+  __do_cancel (PTHREAD_CANCELED);
 }
-libc_hidden_def (__pthread_disable_asynccancel)
diff --git a/nptl/cleanup_defer.c b/nptl/cleanup_defer.c
index eef87f9a9c..d04227722b 100644
--- a/nptl/cleanup_defer.c
+++ b/nptl/cleanup_defer.c
@@ -82,10 +82,7 @@  ___pthread_unregister_cancel_restore (__pthread_unwind_buf_t *buf)
 						    &cancelhandling, newval));
 
       if (cancel_enabled_and_canceled (cancelhandling))
-	{
-	  self->result = PTHREAD_CANCELED;
-	  __do_cancel ();
-	}
+	__do_cancel (PTHREAD_CANCELED);
     }
 }
 versioned_symbol (libc, ___pthread_unregister_cancel_restore,
diff --git a/nptl/descr-const.sym b/nptl/descr-const.sym
new file mode 100644
index 0000000000..8608248354
--- /dev/null
+++ b/nptl/descr-const.sym
@@ -0,0 +1,6 @@ 
+#include <tls.h>
+
+-- Not strictly offsets, these values are using thread cancellation by arch
+-- specific cancel entrypoint.
+TCB_CANCELED_BIT	 CANCELED_BIT
+TCB_CANCELED_BITMASK	 CANCELED_BITMASK
diff --git a/nptl/descr.h b/nptl/descr.h
index f8b5ac7c22..142470f3f3 100644
--- a/nptl/descr.h
+++ b/nptl/descr.h
@@ -415,6 +415,24 @@  struct pthread
   (sizeof (struct pthread) - offsetof (struct pthread, end_padding))
 } __attribute ((aligned (TCB_ALIGNMENT)));
 
+static inline bool
+cancel_enabled (int value)
+{
+  return (value & CANCELSTATE_BITMASK) == 0;
+}
+
+static inline bool
+cancel_async_enabled (int value)
+{
+  return (value & CANCELTYPE_BITMASK) != 0;
+}
+
+static inline bool
+cancel_exiting (int value)
+{
+  return (value & EXITING_BITMASK) != 0;
+}
+
 static inline bool
 cancel_enabled_and_canceled (int value)
 {
diff --git a/nptl/libc-cleanup.c b/nptl/libc-cleanup.c
index 4c7bcda302..252006060a 100644
--- a/nptl/libc-cleanup.c
+++ b/nptl/libc-cleanup.c
@@ -69,10 +69,7 @@  __libc_cleanup_pop_restore (struct _pthread_cleanup_buffer *buffer)
 						    &cancelhandling, newval));
 
       if (cancel_enabled_and_canceled (cancelhandling))
-	{
-	  self->result = PTHREAD_CANCELED;
-	  __do_cancel ();
-	}
+	__do_cancel (PTHREAD_CANCELED);
     }
 }
 libc_hidden_def (__libc_cleanup_pop_restore)
diff --git a/nptl/pthread_cancel.c b/nptl/pthread_cancel.c
index 87c9ef69ad..fc5ca8b3d4 100644
--- a/nptl/pthread_cancel.c
+++ b/nptl/pthread_cancel.c
@@ -23,6 +23,7 @@ 
 #include <sysdep.h>
 #include <unistd.h>
 #include <unwind-link.h>
+#include <cancellation-pc-check.h>
 #include <stdio.h>
 #include <gnu/lib-names.h>
 #include <sys/single_threaded.h>
@@ -40,31 +41,16 @@  sigcancel_handler (int sig, siginfo_t *si, void *ctx)
       || si->si_code != SI_TKILL)
     return;
 
+  /* Check if asynchronous cancellation mode is set or if interrupted
+     instruction pointer falls within the cancellable syscall bridge.  For
+     interruptable syscalls with external side-effects (i.e. partial reads),
+     the kernel  will set the IP to after __syscall_cancel_arch_end, thus
+     disabling the cancellation and allowing the process to handle such
+     conditions.  */
   struct pthread *self = THREAD_SELF;
-
   int oldval = atomic_load_relaxed (&self->cancelhandling);
-  while (1)
-    {
-      /* We are canceled now.  When canceled by another thread this flag
-	 is already set but if the signal is directly send (internally or
-	 from another process) is has to be done here.  */
-      int newval = oldval | CANCELING_BITMASK | CANCELED_BITMASK;
-
-      if (oldval == newval || (oldval & EXITING_BITMASK) != 0)
-	/* Already canceled or exiting.  */
-	break;
-
-      if (atomic_compare_exchange_weak_acquire (&self->cancelhandling,
-						&oldval, newval))
-	{
-	  self->result = PTHREAD_CANCELED;
-
-	  /* Make sure asynchronous cancellation is still enabled.  */
-	  if ((oldval & CANCELTYPE_BITMASK) != 0)
-	    /* Run the registered destructors and terminate the thread.  */
-	    __do_cancel ();
-	}
-    }
+  if (cancel_async_enabled (oldval) || cancellation_pc_check (ctx))
+    __syscall_do_cancel ();
 }
 
 int
@@ -106,15 +92,13 @@  __pthread_cancel (pthread_t th)
   /* Some syscalls are never restarted after being interrupted by a signal
      handler, regardless of the use of SA_RESTART (they always fail with
      EINTR).  So pthread_cancel cannot send SIGCANCEL unless the cancellation
-     is enabled and set as asynchronous (in this case the cancellation will
-     be acted in the cancellation handler instead by the syscall wrapper).
-     Otherwise the target thread is set as 'cancelling' (CANCELING_BITMASK)
+     is enabled.
+     In this case the target thread is set as 'cancelled' (CANCELED_BITMASK)
      by atomically setting 'cancelhandling' and the cancelation will be acted
      upon on next cancellation entrypoing in the target thread.
 
-     It also requires to atomically check if cancellation is enabled and
-     asynchronous, so both cancellation state and type are tracked on
-     'cancelhandling'.  */
+     It also requires to atomically check if cancellation is enabled, so the
+     state are also tracked on 'cancelhandling'.  */
 
   int result = 0;
   int oldval = atomic_load_relaxed (&pd->cancelhandling);
@@ -122,19 +106,17 @@  __pthread_cancel (pthread_t th)
   do
     {
     again:
-      newval = oldval | CANCELING_BITMASK | CANCELED_BITMASK;
+      newval = oldval | CANCELED_BITMASK;
       if (oldval == newval)
 	break;
 
-      /* If the cancellation is handled asynchronously just send a
-	 signal.  We avoid this if possible since it's more
-	 expensive.  */
-      if (cancel_enabled_and_canceled_and_async (newval))
+      /* Only send the SIGANCEL signal is cancellation is enabled, since some
+	 syscalls are never restarted even with SA_RESTART.  The signal
+	 will act iff async cancellation is enabled.  */
+      if (cancel_enabled (newval))
 	{
-	  /* Mark the cancellation as "in progress".  */
-	  int newval2 = oldval | CANCELING_BITMASK;
 	  if (!atomic_compare_exchange_weak_acquire (&pd->cancelhandling,
-						     &oldval, newval2))
+						     &oldval, newval))
 	    goto again;
 
 	  if (pd == THREAD_SELF)
@@ -143,9 +125,8 @@  __pthread_cancel (pthread_t th)
 	       pthread_create, so the signal handler may not have been
 	       set up for a self-cancel.  */
 	    {
-	      pd->result = PTHREAD_CANCELED;
-	      if ((newval & CANCELTYPE_BITMASK) != 0)
-		__do_cancel ();
+	      if (cancel_async_enabled (newval))
+		__do_cancel (PTHREAD_CANCELED);
 	    }
 	  else
 	    /* The cancellation handler will take care of marking the
@@ -154,19 +135,18 @@  __pthread_cancel (pthread_t th)
 
 	  break;
 	}
-
-	/* A single-threaded process should be able to kill itself, since
-	   there is nothing in the POSIX specification that says that it
-	   cannot.  So we set multiple_threads to true so that cancellation
-	   points get executed.  */
-	THREAD_SETMEM (THREAD_SELF, header.multiple_threads, 1);
-#ifndef TLS_MULTIPLE_THREADS_IN_TCB
-	__libc_single_threaded_internal = 0;
-#endif
     }
   while (!atomic_compare_exchange_weak_acquire (&pd->cancelhandling, &oldval,
 						newval));
 
+  /* A single-threaded process should be able to kill itself, since there is
+     nothing in the POSIX specification that says that it cannot.  So we set
+     multiple_threads to true so that cancellation points get executed.  */
+  THREAD_SETMEM (THREAD_SELF, header.multiple_threads, 1);
+#ifndef TLS_MULTIPLE_THREADS_IN_TCB
+  __libc_single_threaded_internal = 0;
+#endif
+
   return result;
 }
 versioned_symbol (libc, __pthread_cancel, pthread_cancel, GLIBC_2_34);
diff --git a/nptl/pthread_exit.c b/nptl/pthread_exit.c
index 9f48dcc5d0..125f44b78a 100644
--- a/nptl/pthread_exit.c
+++ b/nptl/pthread_exit.c
@@ -31,9 +31,7 @@  __pthread_exit (void *value)
                     " must be installed for pthread_exit to work\n");
   }
 
-  THREAD_SETMEM (THREAD_SELF, result, value);
-
-  __do_cancel ();
+  __do_cancel (value);
 }
 libc_hidden_def (__pthread_exit)
 weak_alias (__pthread_exit, pthread_exit)
diff --git a/nptl/pthread_setcancelstate.c b/nptl/pthread_setcancelstate.c
index 7f81d812dd..ffb482a83d 100644
--- a/nptl/pthread_setcancelstate.c
+++ b/nptl/pthread_setcancelstate.c
@@ -48,7 +48,7 @@  __pthread_setcancelstate (int state, int *oldstate)
 						&oldval, newval))
 	{
 	  if (cancel_enabled_and_canceled_and_async (newval))
-	    __do_cancel ();
+	    __do_cancel (PTHREAD_CANCELED);
 
 	  break;
 	}
diff --git a/nptl/pthread_setcanceltype.c b/nptl/pthread_setcanceltype.c
index 7dfeee4364..9fe7c0029b 100644
--- a/nptl/pthread_setcanceltype.c
+++ b/nptl/pthread_setcanceltype.c
@@ -48,7 +48,7 @@  __pthread_setcanceltype (int type, int *oldtype)
 	  if (cancel_enabled_and_canceled_and_async (newval))
 	    {
 	      THREAD_SETMEM (self, result, PTHREAD_CANCELED);
-	      __do_cancel ();
+	      __do_cancel (PTHREAD_CANCELED);
 	    }
 
 	  break;
diff --git a/nptl/pthread_testcancel.c b/nptl/pthread_testcancel.c
index 38b5a2d4bc..b574c0f001 100644
--- a/nptl/pthread_testcancel.c
+++ b/nptl/pthread_testcancel.c
@@ -25,10 +25,7 @@  ___pthread_testcancel (void)
   struct pthread *self = THREAD_SELF;
   int cancelhandling = atomic_load_relaxed (&self->cancelhandling);
   if (cancel_enabled_and_canceled (cancelhandling))
-    {
-      self->result = PTHREAD_CANCELED;
-      __do_cancel ();
-    }
+    __do_cancel (PTHREAD_CANCELED);
 }
 versioned_symbol (libc, ___pthread_testcancel, pthread_testcancel, GLIBC_2_34);
 libc_hidden_ver (___pthread_testcancel, __pthread_testcancel)
diff --git a/nptl/tst-cancel31.c b/nptl/tst-cancel31.c
new file mode 100644
index 0000000000..4e93cc5ae1
--- /dev/null
+++ b/nptl/tst-cancel31.c
@@ -0,0 +1,100 @@ 
+/* Check side-effect act for cancellable syscalls (BZ #12683).
+   Copyright (C) 2023 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+/* This testcase checks if there is resource leakage if the syscall has
+   returned from kernelspace, but before userspace saves the return
+   value.  The 'leaker' thread should be able to close the file descriptor
+   if the resource is already allocated, meaning that if the cancellation
+   signal arrives *after* the open syscal return from kernel, the
+   side-effect should be visible to application.  */
+
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <stdlib.h>
+
+#include <support/xunistd.h>
+#include <support/xthread.h>
+#include <support/check.h>
+#include <support/temp_file.h>
+#include <support/support.h>
+#include <support/descriptors.h>
+
+static void *
+writeopener (void *arg)
+{
+  int fd;
+  for (;;)
+    {
+      fd = open (arg, O_WRONLY);
+      xclose (fd);
+    }
+  return NULL;
+}
+
+static void *
+leaker (void *arg)
+{
+  int fd = open (arg, O_RDONLY);
+  TEST_VERIFY_EXIT (fd > 0);
+  pthread_setcancelstate (PTHREAD_CANCEL_DISABLE, 0);
+  xclose (fd);
+  return NULL;
+}
+
+static int
+do_test (void)
+{
+  enum {
+    iter_count = 1000
+  };
+
+  char *dir = support_create_temp_directory ("tst-cancel28");
+  char *name = xasprintf ("%s/fifo", dir);
+  TEST_COMPARE (mkfifo (name, 0600), 0);
+  add_temp_file (name);
+
+  struct support_descriptors *descrs = support_descriptors_list ();
+
+  srand (1);
+
+  xpthread_create (NULL, writeopener, name);
+  for (int i = 0; i < iter_count; i++)
+    {
+      pthread_t td = xpthread_create (NULL, leaker, name);
+      struct timespec ts =
+	{ .tv_nsec = rand () % 100000, .tv_sec = 0 };
+      nanosleep (&ts, NULL);
+      /* Ignore pthread_cancel result because it might be the
+	 case when pthread_cancel is called when thread is already
+	 exited.  */
+      pthread_cancel (td);
+      xpthread_join (td);
+    }
+
+  support_descriptors_check (descrs);
+
+  support_descriptors_free (descrs);
+
+  free (name);
+
+  return 0;
+}
+
+#include <support/test-driver.c>
diff --git a/sysdeps/generic/syscall_types.h b/sysdeps/generic/syscall_types.h
new file mode 100644
index 0000000000..2ddeaa2b5f
--- /dev/null
+++ b/sysdeps/generic/syscall_types.h
@@ -0,0 +1,25 @@ 
+/* Types and macros used for syscall issuing.
+   Copyright (C) 2023 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#ifndef _SYSCALL_TYPES_H
+#define _SYSCALL_TYPES_H
+
+typedef long int __syscall_arg_t;
+#define __SSC(__x) ((__syscall_arg_t) (__x))
+
+#endif
diff --git a/sysdeps/nptl/cancellation-pc-check.h b/sysdeps/nptl/cancellation-pc-check.h
new file mode 100644
index 0000000000..cb38ad6819
--- /dev/null
+++ b/sysdeps/nptl/cancellation-pc-check.h
@@ -0,0 +1,54 @@ 
+/* Architecture specific code for pthread cancellation handling.
+   Copyright (C) 2023 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#ifndef _NPTL_CANCELLATION_PC_CHECK
+#define _NPTL_CANCELLATION_PC_CHECK
+
+#include <sigcontextinfo.h>
+
+/* For syscalls with side-effects (e.g read that might return partial read),
+   the kernel cannot restart the syscall when interrupted by a signal, it must
+   return from the call with whatever partial result.  In this case, the saved
+   program counter is set just after the syscall instruction, so the SIGCANCEL
+   handler should not act on cancellation.
+
+   The __syscall_cancel_arch function, used for all cancellable syscalls,
+   contains two extra markers, __syscall_cancel_arch_start and
+   __syscall_cancel_arch_end.  The former points to just before the initial
+   conditional branch that checks if the thread has received a cancellation
+   request, while former points to the instruction after the one responsible
+   to issue the syscall.
+
+   The function check if the program counter (PC) from ucontext_t CTX is
+   within the start and then end boundary from the __syscall_cancel_arch
+   bridge.  Return TRUE if the PC is within the boundary, meaning the
+   syscall does not have any side effects; or FALSE otherwise.  */
+
+static __always_inline bool
+cancellation_pc_check (void *ctx)
+{
+  /* Both are defined in syscall_cancel.S.  */
+  extern const char __syscall_cancel_arch_start[1];
+  extern const char __syscall_cancel_arch_end[1];
+
+  uintptr_t pc = sigcontext_get_pc (ctx);
+  return pc >= (uintptr_t) __syscall_cancel_arch_start
+	 && pc < (uintptr_t) __syscall_cancel_arch_end;
+}
+
+#endif
diff --git a/sysdeps/nptl/lowlevellock-futex.h b/sysdeps/nptl/lowlevellock-futex.h
index 0392b5c04f..bd57913b6f 100644
--- a/sysdeps/nptl/lowlevellock-futex.h
+++ b/sysdeps/nptl/lowlevellock-futex.h
@@ -21,7 +21,6 @@ 
 
 #ifndef __ASSEMBLER__
 # include <sysdep.h>
-# include <sysdep-cancel.h>
 # include <kernel-features.h>
 #endif
 
@@ -120,21 +119,10 @@ 
 		     nr_wake, nr_move, mutex, val)
 
 /* Like lll_futex_wait, but acting as a cancellable entrypoint.  */
-# define lll_futex_wait_cancel(futexp, val, private) \
-  ({                                                                   \
-    int __oldtype = LIBC_CANCEL_ASYNC ();			       \
-    long int __err = lll_futex_wait (futexp, val, LLL_SHARED);	       \
-    LIBC_CANCEL_RESET (__oldtype);				       \
-    __err;							       \
-  })
-
-/* Like lll_futex_timed_wait, but acting as a cancellable entrypoint.  */
-# define lll_futex_timed_wait_cancel(futexp, val, timeout, private) \
-  ({									   \
-    int __oldtype = LIBC_CANCEL_ASYNC ();			       	   \
-    long int __err = lll_futex_timed_wait (futexp, val, timeout, private); \
-    LIBC_CANCEL_RESET (__oldtype);					   \
-    __err;								   \
+# define lll_futex_wait_cancel(futexp, val, private)			\
+  ({									\
+     int __op = __lll_private_flag (FUTEX_WAIT, private);		\
+     INTERNAL_SYSCALL_CANCEL (futex, futexp, __op, val, NULL);		\
   })
 
 #endif  /* !__ASSEMBLER__  */
diff --git a/sysdeps/nptl/pthreadP.h b/sysdeps/nptl/pthreadP.h
index 54f9198681..15a7a063e5 100644
--- a/sysdeps/nptl/pthreadP.h
+++ b/sysdeps/nptl/pthreadP.h
@@ -261,10 +261,12 @@  libc_hidden_proto (__pthread_unregister_cancel)
 /* Called when a thread reacts on a cancellation request.  */
 static inline void
 __attribute ((noreturn, always_inline))
-__do_cancel (void)
+__do_cancel (void *result)
 {
   struct pthread *self = THREAD_SELF;
 
+  self->result = result;
+
   /* Make sure we get no more cancellations.  */
   atomic_fetch_or_relaxed (&self->cancelhandling, EXITING_BITMASK);
 
@@ -272,6 +274,13 @@  __do_cancel (void)
 		    THREAD_GETMEM (self, cleanup_jmp_buf));
 }
 
+extern long int __syscall_cancel_arch (volatile int *, __syscall_arg_t nr,
+     __syscall_arg_t arg1, __syscall_arg_t arg2, __syscall_arg_t arg3,
+     __syscall_arg_t arg4, __syscall_arg_t arg5, __syscall_arg_t arg6)
+  attribute_hidden;
+
+extern _Noreturn void __syscall_do_cancel (void) attribute_hidden;
+
 
 /* Internal prototypes.  */
 
diff --git a/sysdeps/unix/sysdep.h b/sysdeps/unix/sysdep.h
index 1ba4de99db..32bc85592e 100644
--- a/sysdeps/unix/sysdep.h
+++ b/sysdeps/unix/sysdep.h
@@ -24,6 +24,9 @@ 
 #define	SYSCALL__(name, args)	PSEUDO (__##name, name, args)
 #define	SYSCALL(name, args)	PSEUDO (name, name, args)
 
+#ifndef __ASSEMBLER__
+# include <errno.h>
+
 #define __SYSCALL_CONCAT_X(a,b)     a##b
 #define __SYSCALL_CONCAT(a,b)       __SYSCALL_CONCAT_X (a, b)
 
@@ -108,42 +111,115 @@ 
 #define INLINE_SYSCALL_CALL(...) \
   __INLINE_SYSCALL_DISP (__INLINE_SYSCALL, __VA_ARGS__)
 
+#define __INTERNAL_SYSCALL_NCS0(name) \
+  INTERNAL_SYSCALL_NCS (name, 0)
+#define __INTERNAL_SYSCALL_NCS1(name, a1) \
+  INTERNAL_SYSCALL_NCS (name, 1, a1)
+#define __INTERNAL_SYSCALL_NCS2(name, a1, a2) \
+  INTERNAL_SYSCALL_NCS (name, 2, a1, a2)
+#define __INTERNAL_SYSCALL_NCS3(name, a1, a2, a3) \
+  INTERNAL_SYSCALL_NCS (name, 3, a1, a2, a3)
+#define __INTERNAL_SYSCALL_NCS4(name, a1, a2, a3, a4) \
+  INTERNAL_SYSCALL_NCS (name, 4, a1, a2, a3, a4)
+#define __INTERNAL_SYSCALL_NCS5(name, a1, a2, a3, a4, a5) \
+  INTERNAL_SYSCALL_NCS (name, 5, a1, a2, a3, a4, a5)
+#define __INTERNAL_SYSCALL_NCS6(name, a1, a2, a3, a4, a5, a6) \
+  INTERNAL_SYSCALL_NCS (name, 6, a1, a2, a3, a4, a5, a6)
+#define __INTERNAL_SYSCALL_NCS7(name, a1, a2, a3, a4, a5, a6, a7) \
+  INTERNAL_SYSCALL_NCS (name, 7, a1, a2, a3, a4, a5, a6, a7)
+
+/* Issue a syscall defined by syscall number plus any other argument required.
+   It is similar to INTERNAL_SYSCALL_NCS macro, but without the need to pass
+   the expected argument number as third parameter.  */
+#define INTERNAL_SYSCALL_NCS_CALL(...) \
+  __INTERNAL_SYSCALL_DISP (__INTERNAL_SYSCALL_NCS, __VA_ARGS__)
+
+/* Cancellation macros.  */
+#include <syscall_types.h>
+
+long int __internal_syscall_cancel (__syscall_arg_t a1, __syscall_arg_t a2,
+				    __syscall_arg_t a3, __syscall_arg_t a4,
+				    __syscall_arg_t a5, __syscall_arg_t a6,
+				    __syscall_arg_t nr) attribute_hidden;
+
+long int __syscall_cancel (__syscall_arg_t nr, __syscall_arg_t arg1,
+			   __syscall_arg_t arg2, __syscall_arg_t arg3,
+			   __syscall_arg_t arg4, __syscall_arg_t arg5,
+			   __syscall_arg_t arg6) attribute_hidden;
+
+#define __SYSCALL_CANCEL0(name)						\
+  __syscall_cancel (0, 0, 0, 0, 0, 0, __NR_##name)
+#define __SYSCALL_CANCEL1(name, a1)					\
+  __syscall_cancel (__SSC (a1), 0, 0, 0, 0, 0, __NR_##name)
+#define __SYSCALL_CANCEL2(name, a1, a2) \
+  __syscall_cancel (__SSC (a1), __SSC (a2), 0, 0, 0, 0, __NR_##name)
+#define __SYSCALL_CANCEL3(name, a1, a2, a3) \
+  __syscall_cancel (__SSC (a1), __SSC (a2), __SSC (a3), 0, 0, 0,	\
+		    __NR_##name)
+#define __SYSCALL_CANCEL4(name, a1, a2, a3, a4) \
+  __syscall_cancel (__SSC (a1), __SSC (a2), __SSC (a3),			\
+		    __SSC(a4), 0, 0, __NR_##name)
+#define __SYSCALL_CANCEL5(name, a1, a2, a3, a4, a5) \
+  __syscall_cancel (__SSC (a1), __SSC (a2), __SSC (a3), __SSC(a4),	\
+		    __SSC (a5), 0, __NR_##name)
+#define __SYSCALL_CANCEL6(name, a1, a2, a3, a4, a5, a6) \
+  __syscall_cancel (__SSC (a1), __SSC (a2), __SSC (a3), __SSC (a4),	\
+		    __SSC (a5), __SSC (a6), __NR_##name)
+
+#define __SYSCALL_CANCEL_NARGS_X(a,b,c,d,e,f,g,h,n,...) n
+#define __SYSCALL_CANCEL_NARGS(...) \
+  __SYSCALL_CANCEL_NARGS_X (__VA_ARGS__,7,6,5,4,3,2,1,0,)
+#define __SYSCALL_CANCEL_CONCAT_X(a,b)     a##b
+#define __SYSCALL_CANCEL_CONCAT(a,b)       __SYSCALL_CANCEL_CONCAT_X (a, b)
+#define __SYSCALL_CANCEL_DISP(b,...) \
+  __SYSCALL_CANCEL_CONCAT (b,__SYSCALL_CANCEL_NARGS(__VA_ARGS__))(__VA_ARGS__)
+
+/* Issue a cancellable syscall defined first argument plus any other argument
+   required.  If and error occurs its value, the macro returns -1 and sets
+   errno accordingly.  */
+#define __SYSCALL_CANCEL_CALL(...) \
+  __SYSCALL_CANCEL_DISP (__SYSCALL_CANCEL, __VA_ARGS__)
+
+#define __INTERNAL_SYSCALL_CANCEL0(name)				\
+  __internal_syscall_cancel (0, 0, 0, 0, 0, 0, __NR_##name)
+#define __INTERNAL_SYSCALL_CANCEL1(name, a1)				\
+  __internal_syscall_cancel (__SSC (a1), 0, 0, 0, 0, 0, __NR_##name)
+#define __INTERNAL_SYSCALL_CANCEL2(name, a1, a2)			\
+  __internal_syscall_cancel (__SSC (a1), __SSC (a2), 0, 0, 0, 0,	\
+			     __NR_##name)
+#define __INTERNAL_SYSCALL_CANCEL3(name, a1, a2, a3)			\
+  __internal_syscall_cancel (__SSC (a1), __SSC (a2), __SSC (a3), 0,	\
+			     0, 0, __NR_##name)
+#define __INTERNAL_SYSCALL_CANCEL4(name, a1, a2, a3, a4)		\
+  __internal_syscall_cancel (__SSC (a1), __SSC (a2), __SSC (a3),	\
+			     __SSC(a4), 0, 0, __NR_##name)
+#define __INTERNAL_SYSCALL_CANCEL5(name, a1, a2, a3, a4, a5)		\
+  __internal_syscall_cancel (__SSC (a1), __SSC (a2), __SSC (a3),	\
+			     __SSC(a4), __SSC (a5), 0, __NR_##name)
+#define __INTERNAL_SYSCALL_CANCEL6(name, a1, a2, a3, a4, a5, a6)	\
+  __internal_syscall_cancel (__SSC (a1), __SSC (a2), __SSC (a3),	\
+			     __SSC (a4), __SSC (a5), __SSC (a6),	\
+			     __NR_##name)
+
+/* Issue a cancellable syscall defined by syscall number NAME plus any other
+   argument required.  If an error occurs its value is returned as an negative
+   number unmodified and errno is not set.  */
+#define __INTERNAL_SYSCALL_CANCEL_CALL(...) \
+  __SYSCALL_CANCEL_DISP (__INTERNAL_SYSCALL_CANCEL, __VA_ARGS__)
+
 #if IS_IN (rtld)
-/* All cancellation points are compiled out in the dynamic loader.  */
-# define NO_SYSCALL_CANCEL_CHECKING 1
+/* The loader does not need to handle thread cancellation, use direct
+   syscall instead.  */
+# define INTERNAL_SYSCALL_CANCEL(...) INTERNAL_SYSCALL_CALL(__VA_ARGS__)
+# define SYSCALL_CANCEL(...)          INLINE_SYSCALL_CALL (__VA_ARGS__)
 #else
-# define NO_SYSCALL_CANCEL_CHECKING SINGLE_THREAD_P
+# define INTERNAL_SYSCALL_CANCEL(...) \
+  __INTERNAL_SYSCALL_CANCEL_CALL (__VA_ARGS__)
+# define SYSCALL_CANCEL(...) \
+  __SYSCALL_CANCEL_CALL (__VA_ARGS__)
 #endif
 
-#define SYSCALL_CANCEL(...) \
-  ({									     \
-    long int sc_ret;							     \
-    if (NO_SYSCALL_CANCEL_CHECKING)					     \
-      sc_ret = INLINE_SYSCALL_CALL (__VA_ARGS__); 			     \
-    else								     \
-      {									     \
-	int sc_cancel_oldtype = LIBC_CANCEL_ASYNC ();			     \
-	sc_ret = INLINE_SYSCALL_CALL (__VA_ARGS__);			     \
-        LIBC_CANCEL_RESET (sc_cancel_oldtype);				     \
-      }									     \
-    sc_ret;								     \
-  })
-
-/* Issue a syscall defined by syscall number plus any other argument
-   required.  Any error will be returned unmodified (including errno).  */
-#define INTERNAL_SYSCALL_CANCEL(...) \
-  ({									     \
-    long int sc_ret;							     \
-    if (NO_SYSCALL_CANCEL_CHECKING) 					     \
-      sc_ret = INTERNAL_SYSCALL_CALL (__VA_ARGS__); 			     \
-    else								     \
-      {									     \
-	int sc_cancel_oldtype = LIBC_CANCEL_ASYNC ();			     \
-	sc_ret = INTERNAL_SYSCALL_CALL (__VA_ARGS__);			     \
-        LIBC_CANCEL_RESET (sc_cancel_oldtype);				     \
-      }									     \
-    sc_ret;								     \
-  })
+#endif /* __ASSEMBLER__  */
 
 /* Machine-dependent sysdep.h files are expected to define the macro
    PSEUDO (function_name, syscall_name) to emit assembly code to define the
diff --git a/sysdeps/unix/sysv/linux/socketcall.h b/sysdeps/unix/sysv/linux/socketcall.h
index d1a173277e..19a6c17a86 100644
--- a/sysdeps/unix/sysv/linux/socketcall.h
+++ b/sysdeps/unix/sysv/linux/socketcall.h
@@ -88,14 +88,33 @@ 
     sc_ret;								\
   })
 
-
-#define SOCKETCALL_CANCEL(name, args...)				\
-  ({									\
-    int oldtype = LIBC_CANCEL_ASYNC ();					\
-    long int sc_ret = __SOCKETCALL (SOCKOP_##name, args);		\
-    LIBC_CANCEL_RESET (oldtype);					\
-    sc_ret;								\
-  })
+#define __SOCKETCALL_CANCEL1(__name, __a1) \
+  SYSCALL_CANCEL (socketcall, __name, \
+     ((long int [1]) { (long int) __a1 }))
+#define __SOCKETCALL_CANCEL2(__name, __a1, __a2) \
+  SYSCALL_CANCEL (socketcall, __name, \
+     ((long int [2]) { (long int) __a1, (long int) __a2 }))
+#define __SOCKETCALL_CANCEL3(__name, __a1, __a2, __a3) \
+  SYSCALL_CANCEL (socketcall, __name, \
+     ((long int [3]) { (long int) __a1, (long int) __a2, (long int) __a3 }))
+#define __SOCKETCALL_CANCEL4(__name, __a1, __a2, __a3, __a4) \
+  SYSCALL_CANCEL (socketcall, __name, \
+     ((long int [4]) { (long int) __a1, (long int) __a2, (long int) __a3, \
+                       (long int) __a4 }))
+#define __SOCKETCALL_CANCEL5(__name, __a1, __a2, __a3, __a4, __a5) \
+  SYSCALL_CANCEL (socketcall, __name, \
+     ((long int [5]) { (long int) __a1, (long int) __a2, (long int) __a3, \
+                       (long int) __a4, (long int) __a5 }))
+#define __SOCKETCALL_CANCEL6(__name, __a1, __a2, __a3, __a4, __a5, __a6) \
+  SYSCALL_CANCEL (socketcall, __name, \
+     ((long int [6]) { (long int) __a1, (long int) __a2, (long int) __a3, \
+                       (long int) __a4, (long int) __a5, (long int) __a6 }))
+
+#define __SOCKETCALL_CANCEL(...) __SOCKETCALL_DISP (__SOCKETCALL_CANCEL,\
+						    __VA_ARGS__)
+
+#define SOCKETCALL_CANCEL(name, args...) \
+   __SOCKETCALL_CANCEL (SOCKOP_##name, args)
 
 
 #endif /* sys/socketcall.h */
diff --git a/sysdeps/unix/sysv/linux/syscall_cancel.c b/sysdeps/unix/sysv/linux/syscall_cancel.c
new file mode 100644
index 0000000000..260680c99f
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/syscall_cancel.c
@@ -0,0 +1,71 @@ 
+/* Pthread cancellation syscall bridge.  Default Linux version.
+   Copyright (C) 2023 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <sysdep.h>
+#include <pthreadP.h>
+
+#warning "This implementation should be use just as reference or for bootstrapping"
+
+/* This is the generic version of the cancellable syscall code which
+   adds the label guards (__syscall_cancel_arch_{start,end}) used on SIGCANCEL
+   handler to check if the cancelled syscall have side-effects that need to be
+   returned to the caller.
+
+   This implementation should be used as a reference one to document the
+   implementation constraints:
+
+     1. The __syscall_cancel_arch_start should point just before the test
+        that thread is already cancelled,
+     2.	The __syscall_cancel_arch_end should point to the immediate next
+        instruction after the syscall one.
+     3. It should return the syscall value or a negative result if is has
+        failed, similar to INTERNAL_SYSCALL_CALL.
+
+   The __syscall_cancel_arch_end one is because the kernel will signal
+   interrupted syscall with side effects by setting the signal frame program
+   counter (on the ucontext_t third argument from SA_SIGINFO signal handler)
+   right after the syscall instruction.
+
+   For some architecture, the INTERNAL_SYSCALL_NCS macro use more instructions
+   to get the error condition from kernel (as for powerpc and sparc that
+   checks for the conditional register), or uses an out of the line helper
+   (ARM thumb), or uses a kernel helper gate (i686 or ia64).  In this case
+   the architecture should either adjust the macro or provide a custom
+   __syscall_cancel_arch implementation.   */
+
+long int
+__syscall_cancel_arch (volatile int *ch, __syscall_arg_t nr,
+		       __syscall_arg_t a1, __syscall_arg_t a2,
+		       __syscall_arg_t a3, __syscall_arg_t a4,
+		       __syscall_arg_t a5, __syscall_arg_t a6)
+{
+#define ADD_LABEL(__label)		\
+  asm volatile (			\
+    ".global " __label "\t\n"		\
+    __label ":\n");
+
+  ADD_LABEL ("__syscall_cancel_arch_start");
+  if (__glibc_unlikely (*ch & CANCELED_BITMASK))
+    __syscall_do_cancel();
+
+  long int result = INTERNAL_SYSCALL_NCS_CALL (nr, a1, a2, a3, a4, a5, a6);
+  ADD_LABEL ("__syscall_cancel_arch_end");
+  if (__glibc_unlikely (INTERNAL_SYSCALL_ERROR_P (result)))
+    return -INTERNAL_SYSCALL_ERRNO (result);
+  return result;
+}
diff --git a/sysdeps/unix/sysv/linux/sysdep-cancel.h b/sysdeps/unix/sysv/linux/sysdep-cancel.h
index 102682c5ee..1b686d53a9 100644
--- a/sysdeps/unix/sysv/linux/sysdep-cancel.h
+++ b/sysdeps/unix/sysv/linux/sysdep-cancel.h
@@ -21,17 +21,5 @@ 
 #define _SYSDEP_CANCEL_H
 
 #include <sysdep.h>
-#include <tls.h>
-#include <errno.h>
-
-/* Set cancellation mode to asynchronous.  */
-extern int __pthread_enable_asynccancel (void);
-libc_hidden_proto (__pthread_enable_asynccancel)
-#define LIBC_CANCEL_ASYNC() __pthread_enable_asynccancel ()
-
-/* Reset to previous cancellation mode.  */
-extern void __pthread_disable_asynccancel (int oldtype);
-libc_hidden_proto (__pthread_disable_asynccancel)
-#define LIBC_CANCEL_RESET(oldtype) __pthread_disable_asynccancel (oldtype)
 
 #endif