[v2,06/19] nptl: Replace struct thread cancelhandling field

Message ID 20210823195047.543237-7-adhemerval.zanella@linaro.org
State Superseded
Headers
Series Fix various NPTL synchronization issues |

Checks

Context Check Description
dj/TryBot-apply_patch success Patch applied to master at the time it was sent

Commit Message

Adhemerval Zanella Aug. 23, 2021, 7:50 p.m. UTC
  Now that both thread state and setxid is being tracked by two different
fields ('joinstate' and 'setxid_flag'), there is no need to keep track
of exiting (EXITING_BIT) and terminated (TERMINATED_BIT) state in
'cancelhandling'.

The 'cancelhandling' field is renamed to 'cancelstatus' and it only
signals whether the thread has been cancelled (set by pthread_cancel()).
It allows simplify the atomic operation required to avoid CAS
operations.

There is no need to check the thread state on
__pthread_enable_asynccancel() nor on pthread_testcancel () anymore,
since cancellation is explicit disabled when thread start the exit code
on __do_cancel().

On __nptl_free_tcp(), the 'joinstate' now defines whether it is the
creating thread or the created thread that calls it.  So there is
no concurrent call within the function and thus no need to set the
TERMINATED_BIT.

For SIGCANCEL handler, sigcancel_handler(), 'joinstate' is used instead
(pthread_cancel() might still be called concurrenty in assynchronous
mode).

Checked on x86_64-linux-gnu and aarch64-linux-gnu.
---
 nptl/Versions                       |  1 +
 nptl/allocatestack.c                |  2 +-
 nptl/cancellation.c                 | 10 ++--------
 nptl/descr.h                        | 13 ++-----------
 nptl/nptl_free_tcb.c                | 22 ++++++----------------
 nptl/pthread_cancel.c               | 10 ++++------
 nptl/pthread_create.c               | 10 ++--------
 nptl/pthread_join_common.c          | 10 +++-------
 nptl/pthread_testcancel.c           | 10 ++--------
 nptl_db/structs.def                 |  2 +-
 nptl_db/td_thr_get_info.c           | 16 +++++++---------
 nptl_db/td_thr_getfpregs.c          |  9 +++++----
 nptl_db/td_thr_getgregs.c           |  9 +++++----
 nptl_db/td_thr_setfpregs.c          |  9 +++++----
 nptl_db/td_thr_setgregs.c           |  9 +++++----
 sysdeps/hppa/nptl/tcb-offsets.sym   |  1 -
 sysdeps/i386/nptl/tcb-offsets.sym   |  1 -
 sysdeps/nptl/pthreadP.h             |  3 ---
 sysdeps/sh/nptl/tcb-offsets.sym     |  1 -
 sysdeps/x86_64/nptl/tcb-offsets.sym |  4 ----
 20 files changed, 51 insertions(+), 101 deletions(-)
  

Comments

Florian Weimer Aug. 26, 2021, 2:34 p.m. UTC | #1
* Adhemerval Zanella:

> Now that both thread state and setxid is being tracked by two different
> fields ('joinstate' and 'setxid_flag'), there is no need to keep track
> of exiting (EXITING_BIT) and terminated (TERMINATED_BIT) state in
> 'cancelhandling'.
>
> The 'cancelhandling' field is renamed to 'cancelstatus' and it only
> signals whether the thread has been cancelled (set by pthread_cancel()).
> It allows simplify the atomic operation required to avoid CAS
> operations.

“cancelstatus” is a bad name because “cancel state” is POSIX terminology
with different meaning.  Please choose a different name.

I think pending_cancel or cancel_requested would work.

> There is no need to check the thread state on
> __pthread_enable_asynccancel() nor on pthread_testcancel () anymore,
> since cancellation is explicit disabled when thread start the exit code
> on __do_cancel().

“explict[ly]”

>
> On __nptl_free_tcp(), the 'joinstate' now defines whether it is the

“__nptl_free_tcb”.

> creating thread or the created thread that calls it.  So there is
> no concurrent call within the function and thus no need to set the
> TERMINATED_BIT.
>
> For SIGCANCEL handler, sigcancel_handler(), 'joinstate' is used instead
> (pthread_cancel() might still be called concurrenty in assynchronous
> mode).

“as[]ynchronous”

> diff --git a/nptl/nptl_free_tcb.c b/nptl/nptl_free_tcb.c
> index cbf3580f59..15e1a18562 100644
> --- a/nptl/nptl_free_tcb.c
> +++ b/nptl/nptl_free_tcb.c
> @@ -24,22 +24,12 @@
>  void
>  __nptl_free_tcb (struct pthread *pd)
>  {
> -  /* The thread is exiting now.  */
> -  if (atomic_bit_test_set (&pd->cancelhandling, TERMINATED_BIT) == 0)
> -    {
> -      /* Free TPP data.  */
> -      if (pd->tpp != NULL)
> -        {
> -          struct priority_protection_data *tpp = pd->tpp;
> +  free (pd->tpp);
> +  pd->tpp = NULL;

For detached threads, this is called on the thread after signals have
been blocked.  I don't think this is entirely correct.  And things
become rather interesting if malloc needs pd->tpp.

It's not clear to me whether this could happen before.

> +  /* Queue the stack memory block for reuse and exit the process.  The kernel
> +     will signal via writing to the address returned by QUEUE-STACK when the
> +     stack is available.  */
> +  __nptl_deallocate_stack (pd);
>  }

That old comment seems to be out-of-date.

>  libc_hidden_def (__nptl_free_tcb)
> diff --git a/nptl/pthread_cancel.c b/nptl/pthread_cancel.c
> index 67e00ef007..aed6c1ea47 100644
> --- a/nptl/pthread_cancel.c
> +++ b/nptl/pthread_cancel.c

> @@ -93,8 +90,9 @@ __pthread_cancel (pthread_t th)
>    }
>  #endif
>  
> -  int oldch = atomic_fetch_or_acquire (&pd->cancelhandling, CANCELED_BITMASK);
> -  if ((oldch & CANCELED_BITMASK) != 0)
> +  /* If already cancelled just return (cancellation will be acted upon in next
> +     cancellation entrypoint).  */
> +  if (atomic_exchange_acquire (&pd->cancelstatus, 1) == 1)
>      return 0;

This should probably use relaxed MO.

The libthreaddb changes look okay to me.

Thanks,
Florian
  
Adhemerval Zanella Aug. 26, 2021, 4:48 p.m. UTC | #2
On 26/08/2021 11:34, Florian Weimer wrote:
> * Adhemerval Zanella:
> 
>> Now that both thread state and setxid is being tracked by two different
>> fields ('joinstate' and 'setxid_flag'), there is no need to keep track
>> of exiting (EXITING_BIT) and terminated (TERMINATED_BIT) state in
>> 'cancelhandling'.
>>
>> The 'cancelhandling' field is renamed to 'cancelstatus' and it only
>> signals whether the thread has been cancelled (set by pthread_cancel()).
>> It allows simplify the atomic operation required to avoid CAS
>> operations.
> 
> “cancelstatus” is a bad name because “cancel state” is POSIX terminology
> with different meaning.  Please choose a different name.
> 
> I think pending_cancel or cancel_requested would work.

The cancel_requested sounds ok.

> 
>> There is no need to check the thread state on
>> __pthread_enable_asynccancel() nor on pthread_testcancel () anymore,
>> since cancellation is explicit disabled when thread start the exit code
>> on __do_cancel().
> 
> “explict[ly]”

Ack.

> 
>>
>> On __nptl_free_tcp(), the 'joinstate' now defines whether it is the
> 
> “__nptl_free_tcb”.

Ack.

> 
>> creating thread or the created thread that calls it.  So there is
>> no concurrent call within the function and thus no need to set the
>> TERMINATED_BIT.
>>
>> For SIGCANCEL handler, sigcancel_handler(), 'joinstate' is used instead
>> (pthread_cancel() might still be called concurrenty in assynchronous
>> mode).
> 
> “as[]ynchronous”
> 

Ack.

>> diff --git a/nptl/nptl_free_tcb.c b/nptl/nptl_free_tcb.c
>> index cbf3580f59..15e1a18562 100644
>> --- a/nptl/nptl_free_tcb.c
>> +++ b/nptl/nptl_free_tcb.c
>> @@ -24,22 +24,12 @@
>>  void
>>  __nptl_free_tcb (struct pthread *pd)
>>  {
>> -  /* The thread is exiting now.  */
>> -  if (atomic_bit_test_set (&pd->cancelhandling, TERMINATED_BIT) == 0)
>> -    {
>> -      /* Free TPP data.  */
>> -      if (pd->tpp != NULL)
>> -        {
>> -          struct priority_protection_data *tpp = pd->tpp;
>> +  free (pd->tpp);
>> +  pd->tpp = NULL;
> 
> For detached threads, this is called on the thread after signals have
> been blocked.  I don't think this is entirely correct.  And things
> become rather interesting if malloc needs pd->tpp.

Why do you think this is not correct?  This is accessed through
mutex code, so I don't think this would matter on the detached
exit phase.

And what do you mean by malloc needing?

> 
> It's not clear to me whether this could happen before.
> 
>> +  /* Queue the stack memory block for reuse and exit the process.  The kernel
>> +     will signal via writing to the address returned by QUEUE-STACK when the
>> +     stack is available.  */
>> +  __nptl_deallocate_stack (pd);
>>  }
> 
> That old comment seems to be out-of-date.

It is still correct I think, __nptl_stack_in_use() now checks the joinstate
which is still advertise by the kernel.

> 
>>  libc_hidden_def (__nptl_free_tcb)
>> diff --git a/nptl/pthread_cancel.c b/nptl/pthread_cancel.c
>> index 67e00ef007..aed6c1ea47 100644
>> --- a/nptl/pthread_cancel.c
>> +++ b/nptl/pthread_cancel.c
> 
>> @@ -93,8 +90,9 @@ __pthread_cancel (pthread_t th)
>>    }
>>  #endif
>>  
>> -  int oldch = atomic_fetch_or_acquire (&pd->cancelhandling, CANCELED_BITMASK);
>> -  if ((oldch & CANCELED_BITMASK) != 0)
>> +  /* If already cancelled just return (cancellation will be acted upon in next
>> +     cancellation entrypoint).  */
>> +  if (atomic_exchange_acquire (&pd->cancelstatus, 1) == 1)
>>      return 0;
> 
> This should probably use relaxed MO.

Ack.

> 
> The libthreaddb changes look okay to me.
> 
> Thanks,
> Florian
>
  
Florian Weimer Aug. 30, 2021, 10:36 a.m. UTC | #3
* Adhemerval Zanella:

>>> diff --git a/nptl/nptl_free_tcb.c b/nptl/nptl_free_tcb.c
>>> index cbf3580f59..15e1a18562 100644
>>> --- a/nptl/nptl_free_tcb.c
>>> +++ b/nptl/nptl_free_tcb.c
>>> @@ -24,22 +24,12 @@
>>>  void
>>>  __nptl_free_tcb (struct pthread *pd)
>>>  {
>>> -  /* The thread is exiting now.  */
>>> -  if (atomic_bit_test_set (&pd->cancelhandling, TERMINATED_BIT) == 0)
>>> -    {
>>> -      /* Free TPP data.  */
>>> -      if (pd->tpp != NULL)
>>> -        {
>>> -          struct priority_protection_data *tpp = pd->tpp;
>>> +  free (pd->tpp);
>>> +  pd->tpp = NULL;
>> 
>> For detached threads, this is called on the thread after signals have
>> been blocked.  I don't think this is entirely correct.  And things
>> become rather interesting if malloc needs pd->tpp.
>
> Why do you think this is not correct?  This is accessed through
> mutex code, so I don't think this would matter on the detached
> exit phase.
>
> And what do you mean by malloc needing?

free is essentially user code, and we should call user code with all
signals disabled.

malloc locking could need the TPP facility.

>> It's not clear to me whether this could happen before.
>> 
>>> +  /* Queue the stack memory block for reuse and exit the process.  The kernel
>>> +     will signal via writing to the address returned by QUEUE-STACK when the
>>> +     stack is available.  */
>>> +  __nptl_deallocate_stack (pd);
>>>  }
>> 
>> That old comment seems to be out-of-date.
>
> It is still correct I think, __nptl_stack_in_use() now checks the joinstate
> which is still advertise by the kernel.

I meant: there is no QUEUE-STACK.

Thanks,
Florian
  

Patch

diff --git a/nptl/Versions b/nptl/Versions
index 3221de89d1..3b23c6e248 100644
--- a/nptl/Versions
+++ b/nptl/Versions
@@ -413,6 +413,7 @@  libc {
     _thread_db_pthread_eventbuf;
     _thread_db_pthread_eventbuf_eventmask;
     _thread_db_pthread_eventbuf_eventmask_event_bits;
+    _thread_db_pthread_joinstate;
     _thread_db_pthread_key_data_data;
     _thread_db_pthread_key_data_level2_data;
     _thread_db_pthread_key_data_seq;
diff --git a/nptl/allocatestack.c b/nptl/allocatestack.c
index ccb1aa21c1..a2a1fa7b81 100644
--- a/nptl/allocatestack.c
+++ b/nptl/allocatestack.c
@@ -119,7 +119,7 @@  get_cached_stack (size_t *sizep, void **memp)
   *memp = result->stackblock;
 
   /* Cancellation handling is back to the default.  */
-  result->cancelhandling = 0;
+  result->cancelstatus = 0;
   result->cancelstate = PTHREAD_CANCEL_ENABLE;
   result->canceltype = PTHREAD_CANCEL_DEFERRED;
   result->cleanup = NULL;
diff --git a/nptl/cancellation.c b/nptl/cancellation.c
index 6478c029de..a244b3eeea 100644
--- a/nptl/cancellation.c
+++ b/nptl/cancellation.c
@@ -35,15 +35,9 @@  __pthread_enable_asynccancel (void)
   int oldval = THREAD_GETMEM (self, canceltype);
   THREAD_SETMEM (self, canceltype, PTHREAD_CANCEL_ASYNCHRONOUS);
 
-  int ch = THREAD_GETMEM (self, cancelhandling);
-
   if (self->cancelstate == PTHREAD_CANCEL_ENABLE
-      && (ch & CANCELED_BITMASK)
-      && !(ch & EXITING_BITMASK)
-      && !(ch & TERMINATED_BITMASK))
-    {
-      __do_cancel ();
-    }
+      && self->cancelstatus == 1)
+    __do_cancel ();
 
   return oldval;
 }
diff --git a/nptl/descr.h b/nptl/descr.h
index 563b152bff..1bfa2b9b52 100644
--- a/nptl/descr.h
+++ b/nptl/descr.h
@@ -286,17 +286,8 @@  struct pthread
   struct pthread_unwind_buf *cleanup_jmp_buf;
 #define HAVE_CLEANUP_JMP_BUF
 
-  /* Flags determining processing of cancellation.  */
-  int cancelhandling;
-  /* Bit set if canceled.  */
-#define CANCELED_BIT		3
-#define CANCELED_BITMASK	(0x01 << CANCELED_BIT)
-  /* Bit set if thread is exiting.  */
-#define EXITING_BIT		4
-#define EXITING_BITMASK		(0x01 << EXITING_BIT)
-  /* Bit set if thread terminated and TCB is freed.  */
-#define TERMINATED_BIT		5
-#define TERMINATED_BITMASK	(0x01 << TERMINATED_BIT)
+  /* Flag to determine whether the thread is signaled to be cancelled.  */
+  int cancelstatus;
 
   /* Flags.  Including those copied from the thread attribute.  */
   int flags;
diff --git a/nptl/nptl_free_tcb.c b/nptl/nptl_free_tcb.c
index cbf3580f59..15e1a18562 100644
--- a/nptl/nptl_free_tcb.c
+++ b/nptl/nptl_free_tcb.c
@@ -24,22 +24,12 @@ 
 void
 __nptl_free_tcb (struct pthread *pd)
 {
-  /* The thread is exiting now.  */
-  if (atomic_bit_test_set (&pd->cancelhandling, TERMINATED_BIT) == 0)
-    {
-      /* Free TPP data.  */
-      if (pd->tpp != NULL)
-        {
-          struct priority_protection_data *tpp = pd->tpp;
+  free (pd->tpp);
+  pd->tpp = NULL;
 
-          pd->tpp = NULL;
-          free (tpp);
-        }
-
-      /* Queue the stack memory block for reuse and exit the process.  The
-         kernel will signal via writing to the address returned by
-         QUEUE-STACK when the stack is available.  */
-      __nptl_deallocate_stack (pd);
-    }
+  /* Queue the stack memory block for reuse and exit the process.  The kernel
+     will signal via writing to the address returned by QUEUE-STACK when the
+     stack is available.  */
+  __nptl_deallocate_stack (pd);
 }
 libc_hidden_def (__nptl_free_tcb)
diff --git a/nptl/pthread_cancel.c b/nptl/pthread_cancel.c
index 67e00ef007..aed6c1ea47 100644
--- a/nptl/pthread_cancel.c
+++ b/nptl/pthread_cancel.c
@@ -43,11 +43,8 @@  sigcancel_handler (int sig, siginfo_t *si, void *ctx)
 
   struct pthread *self = THREAD_SELF;
 
-  int ch = atomic_load_relaxed (&self->cancelhandling);
-  /* Cancelation not enabled, not cancelled, or already exitting.  */
   if (self->cancelstate == PTHREAD_CANCEL_DISABLE
-      || (ch & CANCELED_BITMASK) == 0
-      || (ch & EXITING_BITMASK) != 0)
+      || atomic_load_relaxed (&self->joinstate) == THREAD_STATE_EXITING)
     return;
 
   /* Set the return value.  */
@@ -93,8 +90,9 @@  __pthread_cancel (pthread_t th)
   }
 #endif
 
-  int oldch = atomic_fetch_or_acquire (&pd->cancelhandling, CANCELED_BITMASK);
-  if ((oldch & CANCELED_BITMASK) != 0)
+  /* If already cancelled just return (cancellation will be acted upon in next
+     cancellation entrypoint).  */
+  if (atomic_exchange_acquire (&pd->cancelstatus, 1) == 1)
     return 0;
 
   if (pd == THREAD_SELF)
diff --git a/nptl/pthread_create.c b/nptl/pthread_create.c
index c00d6ae00c..b354f62995 100644
--- a/nptl/pthread_create.c
+++ b/nptl/pthread_create.c
@@ -479,12 +479,6 @@  start_thread (void *arg)
 	}
     }
 
-  /* The thread is exiting now.  Don't set this bit until after we've hit
-     the event-reporting breakpoint, so that td_thr_get_info on us while at
-     the breakpoint reports TD_THR_RUN state rather than TD_THR_ZOMBIE.  */
-  atomic_bit_set (&pd->cancelhandling, EXITING_BIT);
-
-
   /* CONCURRENCY NOTES:
 
      Concurrent pthread_detach() will either set state to
@@ -548,8 +542,8 @@  start_thread (void *arg)
       do
 	/* XXX This differs from the typical futex_wait_simple pattern in that
 	   the futex_wait condition (setxid_futex) is different from the
-	   condition used in the surrounding loop (cancelhandling).  We need
-	   to check and document why this is correct.  */
+	   condition used in the surrounding loop.  We need to check and
+	   document why this is correct.  */
 	futex_wait_simple (&pd->setxid_futex, 0, FUTEX_PRIVATE);
       while (atomic_load_relaxed (&pd->setxid_flag) == 1);
 
diff --git a/nptl/pthread_join_common.c b/nptl/pthread_join_common.c
index 428ed35101..132a40a56e 100644
--- a/nptl/pthread_join_common.c
+++ b/nptl/pthread_join_common.c
@@ -33,14 +33,10 @@  check_for_deadlock (int state, struct pthread *pd)
 {
   struct pthread *self = THREAD_SELF;
   return ((pd == self
-	   || (atomic_load_acquire (&self->joinstate) == THREAD_STATE_DETACHED
-	       && (pd->cancelhandling
-		   & (CANCELED_BITMASK | EXITING_BITMASK
-		      | TERMINATED_BITMASK)) == 0))
+	   || (atomic_load_acquire (&self->joinstate)
+	       == THREAD_STATE_DETACHED))
 	   && !(self->cancelstate == PTHREAD_CANCEL_ENABLE
-		&& (pd->cancelhandling & (CANCELED_BITMASK | EXITING_BITMASK
-					  | TERMINATED_BITMASK))
-		== CANCELED_BITMASK));
+		&& atomic_load_relaxed (&self->cancelstatus) == 1));
 }
 
 int
diff --git a/nptl/pthread_testcancel.c b/nptl/pthread_testcancel.c
index f728a35524..4623f9e7ae 100644
--- a/nptl/pthread_testcancel.c
+++ b/nptl/pthread_testcancel.c
@@ -24,14 +24,8 @@  void
 ___pthread_testcancel (void)
 {
   struct pthread *self = THREAD_SELF;
-  int cancelhandling = THREAD_GETMEM (self, cancelhandling);
-  if (self->cancelstate == PTHREAD_CANCEL_ENABLE
-      && (cancelhandling & CANCELED_BITMASK)
-      && !(cancelhandling & EXITING_BITMASK)
-      && !(cancelhandling & TERMINATED_BITMASK))
-    {
-      __do_cancel ();
-    }
+  if (self->cancelstate == PTHREAD_CANCEL_ENABLE && self->cancelstatus == 1)
+    __do_cancel ();
 }
 versioned_symbol (libc, ___pthread_testcancel, pthread_testcancel, GLIBC_2_34);
 libc_hidden_ver (___pthread_testcancel, __pthread_testcancel)
diff --git a/nptl_db/structs.def b/nptl_db/structs.def
index 248ecf4335..65a068e9d1 100644
--- a/nptl_db/structs.def
+++ b/nptl_db/structs.def
@@ -53,7 +53,7 @@  DB_STRUCT_FIELD (pthread, list)
 DB_STRUCT_FIELD (pthread, report_events)
 DB_STRUCT_FIELD (pthread, tid)
 DB_STRUCT_FIELD (pthread, start_routine)
-DB_STRUCT_FIELD (pthread, cancelhandling)
+DB_STRUCT_FIELD (pthread, joinstate)
 DB_STRUCT_FIELD (pthread, schedpolicy)
 DB_STRUCT_FIELD (pthread, schedparam_sched_priority)
 DB_STRUCT_FIELD (pthread, specific)
diff --git a/nptl_db/td_thr_get_info.c b/nptl_db/td_thr_get_info.c
index 01af021d2a..82f1f2667c 100644
--- a/nptl_db/td_thr_get_info.c
+++ b/nptl_db/td_thr_get_info.c
@@ -28,7 +28,7 @@  td_thr_get_info (const td_thrhandle_t *th, td_thrinfo_t *infop)
 {
   td_err_e err;
   void *copy;
-  psaddr_t tls, schedpolicy, schedprio, cancelhandling, tid, report_events;
+  psaddr_t tls, schedpolicy, schedprio, joinstate, tid, report_events;
 
   LOG ("td_thr_get_info");
 
@@ -37,7 +37,7 @@  td_thr_get_info (const td_thrhandle_t *th, td_thrinfo_t *infop)
       /* Special case for the main thread before initialization.  */
       copy = NULL;
       tls = 0;
-      cancelhandling = 0;
+      joinstate = 0;
       schedpolicy = SCHED_OTHER;
       schedprio = 0;
       tid = 0;
@@ -76,8 +76,8 @@  td_thr_get_info (const td_thrhandle_t *th, td_thrinfo_t *infop)
       err = DB_GET_FIELD_LOCAL (tid, th->th_ta_p, copy, pthread, tid, 0);
       if (err != TD_OK)
 	return err;
-      err = DB_GET_FIELD_LOCAL (cancelhandling, th->th_ta_p, copy, pthread,
-				cancelhandling, 0);
+      err = DB_GET_FIELD_LOCAL (joinstate, th->th_ta_p, copy, pthread,
+				joinstate, 0);
       if (err != TD_OK)
 	return err;
       err = DB_GET_FIELD_LOCAL (report_events, th->th_ta_p, copy, pthread,
@@ -96,13 +96,11 @@  td_thr_get_info (const td_thrhandle_t *th, td_thrinfo_t *infop)
 		   ? 0 : (uintptr_t) schedprio);
   infop->ti_type = TD_THR_USER;
 
-  if ((((int) (uintptr_t) cancelhandling) & EXITING_BITMASK) == 0)
-    /* XXX For now there is no way to get more information.  */
+  int js = (int) (uintptr_t) joinstate;
+  if (js == THREAD_STATE_JOINABLE || js == THREAD_STATE_DETACHED)
     infop->ti_state = TD_THR_ACTIVE;
-  else if ((((int) (uintptr_t) cancelhandling) & TERMINATED_BITMASK) == 0)
-    infop->ti_state = TD_THR_ZOMBIE;
   else
-    infop->ti_state = TD_THR_UNKNOWN;
+    infop->ti_state = TD_THR_ZOMBIE;
 
   /* Initialization which are the same in both cases.  */
   infop->ti_ta_p = th->th_ta_p;
diff --git a/nptl_db/td_thr_getfpregs.c b/nptl_db/td_thr_getfpregs.c
index 3d08aa3f60..23a8a215c2 100644
--- a/nptl_db/td_thr_getfpregs.c
+++ b/nptl_db/td_thr_getfpregs.c
@@ -23,7 +23,7 @@ 
 td_err_e
 td_thr_getfpregs (const td_thrhandle_t *th, prfpregset_t *regset)
 {
-  psaddr_t cancelhandling, tid;
+  psaddr_t joinstate, tid;
   td_err_e err;
 
   LOG ("td_thr_getfpregs");
@@ -34,13 +34,14 @@  td_thr_getfpregs (const td_thrhandle_t *th, prfpregset_t *regset)
 			  regset) != PS_OK ? TD_ERR : TD_OK;
 
   /* We have to get the state and the PID for this thread.  */
-  err = DB_GET_FIELD (cancelhandling, th->th_ta_p, th->th_unique, pthread,
-		      cancelhandling, 0);
+  err = DB_GET_FIELD (joinstate, th->th_ta_p, th->th_unique, pthread,
+		      joinstate, 0);
   if (err != TD_OK)
     return err;
 
   /* If the thread already terminated we return all zeroes.  */
-  if (((int) (uintptr_t) cancelhandling) & TERMINATED_BITMASK)
+  int js = (int) (uintptr_t) joinstate;
+  if (js == THREAD_STATE_EXITING || js == THREAD_STATE_EXITED)
     memset (regset, '\0', sizeof (*regset));
   /* Otherwise get the register content through the callback.  */
   else
diff --git a/nptl_db/td_thr_getgregs.c b/nptl_db/td_thr_getgregs.c
index 8f9fab096f..b92402670f 100644
--- a/nptl_db/td_thr_getgregs.c
+++ b/nptl_db/td_thr_getgregs.c
@@ -23,7 +23,7 @@ 
 td_err_e
 td_thr_getgregs (const td_thrhandle_t *th, prgregset_t regset)
 {
-  psaddr_t cancelhandling, tid;
+  psaddr_t joinstate, tid;
   td_err_e err;
 
   LOG ("td_thr_getgregs");
@@ -34,13 +34,14 @@  td_thr_getgregs (const td_thrhandle_t *th, prgregset_t regset)
 			regset) != PS_OK ? TD_ERR : TD_OK;
 
   /* We have to get the state and the PID for this thread.  */
-  err = DB_GET_FIELD (cancelhandling, th->th_ta_p, th->th_unique, pthread,
-		      cancelhandling, 0);
+  err = DB_GET_FIELD (joinstate, th->th_ta_p, th->th_unique, pthread,
+		      joinstate, 0);
   if (err != TD_OK)
     return err;
 
   /* If the thread already terminated we return all zeroes.  */
-  if (((int) (uintptr_t) cancelhandling) & TERMINATED_BITMASK)
+  int js = (int) (uintptr_t) joinstate;
+  if (js == THREAD_STATE_EXITING || js == THREAD_STATE_EXITED)
     memset (regset, '\0', sizeof (*regset));
   /* Otherwise get the register content through the callback.  */
   else
diff --git a/nptl_db/td_thr_setfpregs.c b/nptl_db/td_thr_setfpregs.c
index bddb0359a8..73ab675ce6 100644
--- a/nptl_db/td_thr_setfpregs.c
+++ b/nptl_db/td_thr_setfpregs.c
@@ -23,7 +23,7 @@ 
 td_err_e
 td_thr_setfpregs (const td_thrhandle_t *th, const prfpregset_t *fpregs)
 {
-  psaddr_t cancelhandling, tid;
+  psaddr_t joinstate, tid;
   td_err_e err;
 
   LOG ("td_thr_setfpregs");
@@ -34,13 +34,14 @@  td_thr_setfpregs (const td_thrhandle_t *th, const prfpregset_t *fpregs)
 			  fpregs) != PS_OK ? TD_ERR : TD_OK;
 
   /* We have to get the state and the PID for this thread.  */
-  err = DB_GET_FIELD (cancelhandling, th->th_ta_p, th->th_unique, pthread,
-		      cancelhandling, 0);
+  err = DB_GET_FIELD (joinstate, th->th_ta_p, th->th_unique, pthread,
+		      joinstate, 0);
   if (err != TD_OK)
     return err;
 
   /* Only set the registers if the thread hasn't yet terminated.  */
-  if ((((int) (uintptr_t) cancelhandling) & TERMINATED_BITMASK) == 0)
+  int js = (int) (uintptr_t) joinstate;
+  if (js != THREAD_STATE_EXITING || js != THREAD_STATE_EXITED)
     {
       err = DB_GET_FIELD (tid, th->th_ta_p, th->th_unique, pthread, tid, 0);
       if (err != TD_OK)
diff --git a/nptl_db/td_thr_setgregs.c b/nptl_db/td_thr_setgregs.c
index 2a76a10754..186df94cbc 100644
--- a/nptl_db/td_thr_setgregs.c
+++ b/nptl_db/td_thr_setgregs.c
@@ -23,7 +23,7 @@ 
 td_err_e
 td_thr_setgregs (const td_thrhandle_t *th, prgregset_t gregs)
 {
-  psaddr_t cancelhandling, tid;
+  psaddr_t joinstate, tid;
   td_err_e err;
 
   LOG ("td_thr_setgregs");
@@ -34,13 +34,14 @@  td_thr_setgregs (const td_thrhandle_t *th, prgregset_t gregs)
 			gregs) != PS_OK ? TD_ERR : TD_OK;
 
   /* We have to get the state and the PID for this thread.  */
-  err = DB_GET_FIELD (cancelhandling, th->th_ta_p, th->th_unique, pthread,
-		      cancelhandling, 0);
+  err = DB_GET_FIELD (joinstate, th->th_ta_p, th->th_unique, pthread,
+		      joinstate, 0);
   if (err != TD_OK)
     return err;
 
   /* Only set the registers if the thread hasn't yet terminated.  */
-  if ((((int) (uintptr_t) cancelhandling) & TERMINATED_BITMASK) == 0)
+  int js = (int) (uintptr_t) joinstate;
+  if (js != THREAD_STATE_EXITING || js != THREAD_STATE_EXITED)
     {
       err = DB_GET_FIELD (tid, th->th_ta_p, th->th_unique, pthread, tid, 0);
       if (err != TD_OK)
diff --git a/sysdeps/hppa/nptl/tcb-offsets.sym b/sysdeps/hppa/nptl/tcb-offsets.sym
index 6e852f35b1..8937f1ec21 100644
--- a/sysdeps/hppa/nptl/tcb-offsets.sym
+++ b/sysdeps/hppa/nptl/tcb-offsets.sym
@@ -3,7 +3,6 @@ 
 
 RESULT			offsetof (struct pthread, result)
 TID			offsetof (struct pthread, tid)
-CANCELHANDLING		offsetof (struct pthread, cancelhandling)
 CLEANUP_JMP_BUF		offsetof (struct pthread, cleanup_jmp_buf)
 MULTIPLE_THREADS_OFFSET	offsetof (struct pthread, header.multiple_threads)
 TLS_PRE_TCB_SIZE	sizeof (struct pthread)
diff --git a/sysdeps/i386/nptl/tcb-offsets.sym b/sysdeps/i386/nptl/tcb-offsets.sym
index 2ec9e787c1..85a27dc29f 100644
--- a/sysdeps/i386/nptl/tcb-offsets.sym
+++ b/sysdeps/i386/nptl/tcb-offsets.sym
@@ -4,7 +4,6 @@ 
 
 RESULT			offsetof (struct pthread, result)
 TID			offsetof (struct pthread, tid)
-CANCELHANDLING		offsetof (struct pthread, cancelhandling)
 CLEANUP_JMP_BUF		offsetof (struct pthread, cleanup_jmp_buf)
 MULTIPLE_THREADS_OFFSET	offsetof (tcbhead_t, multiple_threads)
 SYSINFO_OFFSET		offsetof (tcbhead_t, sysinfo)
diff --git a/sysdeps/nptl/pthreadP.h b/sysdeps/nptl/pthreadP.h
index e89c1b92ae..d366d691cd 100644
--- a/sysdeps/nptl/pthreadP.h
+++ b/sysdeps/nptl/pthreadP.h
@@ -272,9 +272,6 @@  __exit_thread (void *value)
 {
   struct pthread *self = THREAD_SELF;
 
-  /* Make sure we get no more cancellations.  */
-  THREAD_ATOMIC_BIT_SET (self, cancelhandling, EXITING_BIT);
-
   THREAD_SETMEM (self, result, value);
 
   /* It is required by POSIX XSH 2.9.5 Thread Cancellation under the heading
diff --git a/sysdeps/sh/nptl/tcb-offsets.sym b/sysdeps/sh/nptl/tcb-offsets.sym
index 234207779d..60c9e40b72 100644
--- a/sysdeps/sh/nptl/tcb-offsets.sym
+++ b/sysdeps/sh/nptl/tcb-offsets.sym
@@ -4,7 +4,6 @@ 
 
 RESULT			offsetof (struct pthread, result)
 TID			offsetof (struct pthread, tid)
-CANCELHANDLING		offsetof (struct pthread, cancelhandling)
 CLEANUP_JMP_BUF		offsetof (struct pthread, cleanup_jmp_buf)
 MULTIPLE_THREADS_OFFSET	offsetof (struct pthread, header.multiple_threads)
 TLS_PRE_TCB_SIZE	sizeof (struct pthread)
diff --git a/sysdeps/x86_64/nptl/tcb-offsets.sym b/sysdeps/x86_64/nptl/tcb-offsets.sym
index 2bbd563a6c..6cc845f7ed 100644
--- a/sysdeps/x86_64/nptl/tcb-offsets.sym
+++ b/sysdeps/x86_64/nptl/tcb-offsets.sym
@@ -4,7 +4,6 @@ 
 
 RESULT			offsetof (struct pthread, result)
 TID			offsetof (struct pthread, tid)
-CANCELHANDLING		offsetof (struct pthread, cancelhandling)
 CLEANUP_JMP_BUF		offsetof (struct pthread, cleanup_jmp_buf)
 CLEANUP			offsetof (struct pthread, cleanup)
 CLEANUP_PREV		offsetof (struct _pthread_cleanup_buffer, __prev)
@@ -13,6 +12,3 @@  MULTIPLE_THREADS_OFFSET	offsetof (tcbhead_t, multiple_threads)
 POINTER_GUARD		offsetof (tcbhead_t, pointer_guard)
 FEATURE_1_OFFSET	offsetof (tcbhead_t, feature_1)
 SSP_BASE_OFFSET		offsetof (tcbhead_t, ssp_base)
-
--- Not strictly offsets, but these values are also used in the TCB.
-TCB_CANCELED_BITMASK	 CANCELED_BITMASK