fix static TLS consumption by TLS descriptors

  AArch64 presently defaults to use TLS descriptors, as opposed to
traditional dynamic TLS relocations, and has hit a bug in the copy-pasta
used in elf_machine_rel{,a} for all TLS descriptor implementations. When
a program calls dlopen() with R_AARCH64_TLSDESC relocs, we're consuming
static TLS until it's exhausted (presumably as an optimisation.)
However, this means that once our static TLS space is consumed,
attempting to dlopen() any shared object which contain an
R_AARCH64_TPREL reloc will fail, since we've exhausted all the space.

Instead, always use dynamic TLS for TLS descriptors, which lets us load
arbitrarily many proper dynamic TLS using shared objects, and allow the
static TLS fallback only in the non-SHARED case.

This issue is reproducible on x86_64, by specifying -ftls-dialect=gnu2
when building with gcc. I've written a test for this:
https://github.com/jkkm/test-tls_desc.git
Tweak DIALECT to be appropriate, and run ./test-tls_desc.sh, ./main
and it should fail with:
% ./test-tls_desc.sh; ./main
./tmp16.so: cannot allocate memory in static TLS block
./test-tls_desc.sh: line 18: 10141 Aborted                 (core dumped) ./main

2014-05-14  Kyle McMartin  <kyle@redhat.com>

        * sysdeps/aarch64/dl-machine.h (elf_machine_rela): always
          allocate dynamic TLS space for TLS descriptors.
        * sysdeps/x86_64/dl-machine.h (elf_machine_rela): ditto.
        * sysdeps/arm/dl-machine.h (elf_machine_rel): ditto.
        * sysdeps/i386/dl-machine.h (elf_machine_rel): ditto.
  

On 14 May 2014 21:32, Kyle McMartin <kmcmarti@redhat.com> wrote:
> AArch64 presently defaults to use TLS descriptors, as opposed to
> traditional dynamic TLS relocations, and has hit a bug in the copy-pasta
> used in elf_machine_rel{,a} for all TLS descriptor implementations. When
> a program calls dlopen() with R_AARCH64_TLSDESC relocs, we're consuming
> static TLS until it's exhausted (presumably as an optimisation.)
> However, this means that once our static TLS space is consumed,
> attempting to dlopen() any shared object which contain an
> R_AARCH64_TPREL reloc will fail, since we've exhausted all the space.
>
> Instead, always use dynamic TLS for TLS descriptors, which lets us load
> arbitrarily many proper dynamic TLS using shared objects, and allow the
> static TLS fallback only in the non-SHARED case.
>
> This issue is reproducible on x86_64, by specifying -ftls-dialect=gnu2
> when building with gcc. I've written a test for this:
> https://github.com/jkkm/test-tls_desc.git
> Tweak DIALECT to be appropriate, and run ./test-tls_desc.sh, ./main
> and it should fail with:
> % ./test-tls_desc.sh; ./main
> ./tmp16.so: cannot allocate memory in static TLS block
> ./test-tls_desc.sh: line 18: 10141 Aborted                 (core dumped) ./main
>
> 2014-05-14  Kyle McMartin  <kyle@redhat.com>
>
>         * sysdeps/aarch64/dl-machine.h (elf_machine_rela): always
>           allocate dynamic TLS space for TLS descriptors.
>         * sysdeps/x86_64/dl-machine.h (elf_machine_rela): ditto.
>         * sysdeps/arm/dl-machine.h (elf_machine_rel): ditto.
>         * sysdeps/i386/dl-machine.h (elf_machine_rel): ditto.

ChangeLog entries should be sentences, so start with a capital letter.

> --- a/sysdeps/aarch64/dl-machine.h
> +++ b/sysdeps/aarch64/dl-machine.h
> @@ -295,13 +295,12 @@ elf_machine_rela (struct link_map *map, const ElfW(Rela) *reloc,
>  # ifndef SHARED
>                 CHECK_STATIC_TLS (map, sym_map);
>  # else
> -               if (!TRY_STATIC_TLS (map, sym_map))
>                   {
>                     td->arg = _dl_make_tlsdesc_dynamic
>                       (sym_map, sym->st_value + reloc->r_addend);
>                     td->entry = _dl_tlsdesc_dynamic;
>                   }
> -               else
> +               if (0)
>  # endif
>  #endif
>                   {
> diff --git a/sysdeps/arm/dl-machine.h b/sysdeps/arm/dl-machine.h
> index 899b256..dcb2127 100644
> --- a/sysdeps/arm/dl-machine.h
> +++ b/sysdeps/arm/dl-machine.h
> @@ -458,13 +458,12 @@ elf_machine_rel (struct link_map *map, const Elf32_Rel *reloc,
>  #  ifndef SHARED
>                 CHECK_STATIC_TLS (map, sym_map);
>  #  else
> -               if (!TRY_STATIC_TLS (map, sym_map))
>                   {
>                     td->argument.pointer
>                       = _dl_make_tlsdesc_dynamic (sym_map, value);
>                     td->entry = _dl_tlsdesc_dynamic;
>                   }
> -               else
> +               if (0)
>  #  endif
>  # endif
>                 {
> diff --git a/sysdeps/i386/dl-machine.h b/sysdeps/i386/dl-machine.h
> index 368bee2..64cf74c 100644
> --- a/sysdeps/i386/dl-machine.h
> +++ b/sysdeps/i386/dl-machine.h
> @@ -394,13 +394,12 @@ elf_machine_rel (struct link_map *map, const Elf32_Rel *reloc,
>  #  ifndef SHARED
>                 CHECK_STATIC_TLS (map, sym_map);
>  #  else
> -               if (!TRY_STATIC_TLS (map, sym_map))
>                   {
>                     td->arg = _dl_make_tlsdesc_dynamic
>                       (sym_map, sym->st_value + (ElfW(Word))td->arg);
>                     td->entry = _dl_tlsdesc_dynamic;
>                   }
> -               else
> +               if (0)
>  #  endif
>  # endif
>                   {
> diff --git a/sysdeps/x86_64/dl-machine.h b/sysdeps/x86_64/dl-machine.h
> index 8df04a9..67e98d1 100644
> --- a/sysdeps/x86_64/dl-machine.h
> +++ b/sysdeps/x86_64/dl-machine.h
> @@ -359,13 +359,12 @@ elf_machine_rela (struct link_map *map, const ElfW(Rela) *reloc,
>  #   ifndef SHARED
>                 CHECK_STATIC_TLS (map, sym_map);
>  #   else
> -               if (!TRY_STATIC_TLS (map, sym_map))
>                   {
>                     td->arg = _dl_make_tlsdesc_dynamic
>                       (sym_map, sym->st_value + reloc->r_addend);
>                     td->entry = _dl_tlsdesc_dynamic;
>                   }
> -               else
> +               if (0)
>  #   endif
>  #  endif
>                   {
  

On Wed, May 14, 2014 at 1:33 PM Kyle McMartin <kmcmarti@redhat.com> wrote:
>
> AArch64 presently defaults to use TLS descriptors, as opposed to
> traditional dynamic TLS relocations, and has hit a bug in the copy-pasta
> used in elf_machine_rel{,a} for all TLS descriptor implementations. When
> a program calls dlopen() with R_AARCH64_TLSDESC relocs, we're consuming
> static TLS until it's exhausted (presumably as an optimisation.)
> However, this means that once our static TLS space is consumed,
> attempting to dlopen() any shared object which contain an
> R_AARCH64_TPREL reloc will fail, since we've exhausted all the space.
>
> Instead, always use dynamic TLS for TLS descriptors, which lets us load
> arbitrarily many proper dynamic TLS using shared objects, and allow the
> static TLS fallback only in the non-SHARED case.
>
> This issue is reproducible on x86_64, by specifying -ftls-dialect=gnu2
> when building with gcc. I've written a test for this:
> https://github.com/jkkm/test-tls_desc.git
> Tweak DIALECT to be appropriate, and run ./test-tls_desc.sh, ./main
> and it should fail with:
> % ./test-tls_desc.sh; ./main
> ./tmp16.so: cannot allocate memory in static TLS block
> ./test-tls_desc.sh: line 18: 10141 Aborted                 (core dumped) ./main

This or a variant of this patch never went in it seems and VPP folks
had run into this similar thing:
https://jira.fd.io/browse/VPP-1284

Thanks,
Andrew Pinski

>
> 2014-05-14  Kyle McMartin  <kyle@redhat.com>
>
>         * sysdeps/aarch64/dl-machine.h (elf_machine_rela): always
>           allocate dynamic TLS space for TLS descriptors.
>         * sysdeps/x86_64/dl-machine.h (elf_machine_rela): ditto.
>         * sysdeps/arm/dl-machine.h (elf_machine_rel): ditto.
>         * sysdeps/i386/dl-machine.h (elf_machine_rel): ditto.
>
> --- a/sysdeps/aarch64/dl-machine.h
> +++ b/sysdeps/aarch64/dl-machine.h
> @@ -295,13 +295,12 @@ elf_machine_rela (struct link_map *map, const ElfW(Rela) *reloc,
>  # ifndef SHARED
>                 CHECK_STATIC_TLS (map, sym_map);
>  # else
> -               if (!TRY_STATIC_TLS (map, sym_map))
>                   {
>                     td->arg = _dl_make_tlsdesc_dynamic
>                       (sym_map, sym->st_value + reloc->r_addend);
>                     td->entry = _dl_tlsdesc_dynamic;
>                   }
> -               else
> +               if (0)
>  # endif
>  #endif
>                   {
> diff --git a/sysdeps/arm/dl-machine.h b/sysdeps/arm/dl-machine.h
> index 899b256..dcb2127 100644
> --- a/sysdeps/arm/dl-machine.h
> +++ b/sysdeps/arm/dl-machine.h
> @@ -458,13 +458,12 @@ elf_machine_rel (struct link_map *map, const Elf32_Rel *reloc,
>  #  ifndef SHARED
>                 CHECK_STATIC_TLS (map, sym_map);
>  #  else
> -               if (!TRY_STATIC_TLS (map, sym_map))
>                   {
>                     td->argument.pointer
>                       = _dl_make_tlsdesc_dynamic (sym_map, value);
>                     td->entry = _dl_tlsdesc_dynamic;
>                   }
> -               else
> +               if (0)
>  #  endif
>  # endif
>                 {
> diff --git a/sysdeps/i386/dl-machine.h b/sysdeps/i386/dl-machine.h
> index 368bee2..64cf74c 100644
> --- a/sysdeps/i386/dl-machine.h
> +++ b/sysdeps/i386/dl-machine.h
> @@ -394,13 +394,12 @@ elf_machine_rel (struct link_map *map, const Elf32_Rel *reloc,
>  #  ifndef SHARED
>                 CHECK_STATIC_TLS (map, sym_map);
>  #  else
> -               if (!TRY_STATIC_TLS (map, sym_map))
>                   {
>                     td->arg = _dl_make_tlsdesc_dynamic
>                       (sym_map, sym->st_value + (ElfW(Word))td->arg);
>                     td->entry = _dl_tlsdesc_dynamic;
>                   }
> -               else
> +               if (0)
>  #  endif
>  # endif
>                   {
> diff --git a/sysdeps/x86_64/dl-machine.h b/sysdeps/x86_64/dl-machine.h
> index 8df04a9..67e98d1 100644
> --- a/sysdeps/x86_64/dl-machine.h
> +++ b/sysdeps/x86_64/dl-machine.h
> @@ -359,13 +359,12 @@ elf_machine_rela (struct link_map *map, const ElfW(Rela) *reloc,
>  #   ifndef SHARED
>                 CHECK_STATIC_TLS (map, sym_map);
>  #   else
> -               if (!TRY_STATIC_TLS (map, sym_map))
>                   {
>                     td->arg = _dl_make_tlsdesc_dynamic
>                       (sym_map, sym->st_value + reloc->r_addend);
>                     td->entry = _dl_tlsdesc_dynamic;
>                   }
> -               else
> +               if (0)
>  #   endif
>  #  endif
>                   {
  

* Andrew Pinski via Libc-alpha:

> On Wed, May 14, 2014 at 1:33 PM Kyle McMartin <kmcmarti@redhat.com> wrote:
>>
>> AArch64 presently defaults to use TLS descriptors, as opposed to
>> traditional dynamic TLS relocations, and has hit a bug in the copy-pasta
>> used in elf_machine_rel{,a} for all TLS descriptor implementations. When
>> a program calls dlopen() with R_AARCH64_TLSDESC relocs, we're consuming
>> static TLS until it's exhausted (presumably as an optimisation.)
>> However, this means that once our static TLS space is consumed,
>> attempting to dlopen() any shared object which contain an
>> R_AARCH64_TPREL reloc will fail, since we've exhausted all the space.
>>
>> Instead, always use dynamic TLS for TLS descriptors, which lets us load
>> arbitrarily many proper dynamic TLS using shared objects, and allow the
>> static TLS fallback only in the non-SHARED case.
>>
>> This issue is reproducible on x86_64, by specifying -ftls-dialect=gnu2
>> when building with gcc. I've written a test for this:
>> https://github.com/jkkm/test-tls_desc.git
>> Tweak DIALECT to be appropriate, and run ./test-tls_desc.sh, ./main
>> and it should fail with:
>> % ./test-tls_desc.sh; ./main
>> ./tmp16.so: cannot allocate memory in static TLS block
>> ./test-tls_desc.sh: line 18: 10141 Aborted                 (core dumped) ./main
>
> This or a variant of this patch never went in it seems and VPP folks
> had run into this similar thing:

> https://jira.fd.io/browse/VPP-1284

There isn't really enough information in the JIRA ticket.  It could also
be the race fixed in:

commit b7cf203b5c17dd6d9878537d41e0c7cc3d270a67
Author: Szabolcs Nagy <szabolcs.nagy@arm.com>
Date:   Wed Sep 27 16:55:14 2017 +0100

    aarch64: Disable lazy symbol binding of TLSDESC

The allocation issue should be fixed in:

commit ffb17e7ba3a5ba9632cee97330b325072fbe41dd
Author: Szabolcs Nagy <szabolcs.nagy@arm.com>
Date:   Wed Jun 10 13:40:40 2020 +0100

    rtld: Avoid using up static TLS surplus for optimizations [BZ #25051]

Thanks,
Florian