[2/2] libc-abis: Define ABSOLUTE ABI [BZ #19818][BZ #23307]

  Define a new ABSOLUTE ABI for static linker's use with EI_ABIVERSION 
where correct absolute (SHN_ABS) symbol run-time load semantics is 
required.  This way it can be ensured at static link time that a program 
or DSO will not suffer from previous semantics where absolute symbols 
were relocated by the base address, or symbols whose `st_value' is zero 
silently ignored leading to a confusing "undefined symbol" error message 
at load time, and instead "ELF file ABI version invalid" is printed with 
old dynamic loaders, making it clear that there is an ABI version 
incompatibility.

	[BZ #19818]
	[BZ #23307]
	* libc-abis (ABSOLUTE): New ABI.
	* sysdeps/unix/sysv/linux/mips/libc-abis (ABSOLUTE): New ABI.
---
Hi,

 Arguably the "ELF file ABI version invalid" message could be improved 
too, e.g. I think "Unsupported ELF file ABI version" or even "Unsupported 
ELF file ABI version, please upgrade `ld.so'" would make it clearer what 
is going on.  But that's a matter for a separate change.

 OK to apply?

  Maciej
---
 libc-abis                              |    2 ++
 sysdeps/unix/sysv/linux/mips/libc-abis |    2 ++
 2 files changed, 4 insertions(+)

glibc-abi-absolute.diff
  

* Maciej W. Rozycki:

> Define a new ABSOLUTE ABI for static linker's use with EI_ABIVERSION 
> where correct absolute (SHN_ABS) symbol run-time load semantics is 
> required.  This way it can be ensured at static link time that a program 
> or DSO will not suffer from previous semantics where absolute symbols 
> were relocated by the base address, or symbols whose `st_value' is zero 
> silently ignored leading to a confusing "undefined symbol" error message 
> at load time, and instead "ELF file ABI version invalid" is printed with 
> old dynamic loaders, making it clear that there is an ABI version 
> incompatibility.

Is this really necessary?  It essentially precludes backporting the
fixes.
  

Hi Florian,

> > Define a new ABSOLUTE ABI for static linker's use with EI_ABIVERSION 
> > where correct absolute (SHN_ABS) symbol run-time load semantics is 
> > required.  This way it can be ensured at static link time that a program 
> > or DSO will not suffer from previous semantics where absolute symbols 
> > were relocated by the base address, or symbols whose `st_value' is zero 
> > silently ignored leading to a confusing "undefined symbol" error message 
> > at load time, and instead "ELF file ABI version invalid" is printed with 
> > old dynamic loaders, making it clear that there is an ABI version 
> > incompatibility.
> 
> Is this really necessary?  It essentially precludes backporting the
> fixes.

 Thank you for your input.  Of course bumping up the ABI version is not 
itself required for the change to work, and all existing working binaries 
will continue to.

 Technically it does not preclude backporting either, although of course 
it is limited by the previous ABI bump, which is (in the reverse order by 
version/date):

* glibc 2.21 for MIPS targets, by commit 12e6ee86c4f2 ("Add support for 
  MIPS O32 FPXX and .MIPS.abiflags"),

* glibc 2.12 for PowerPC and SPARC targets, by commit 12e6ee86c4f2 ("A few 
  more archs have IFUNC support."),

* glibc 2.12 for the remaining targets, by commit 92ad15a8f1d3 ("Implement 
  handling of libc ABI in ELF header."), when the feature was introduced.

I don't believe we backport *that* far in upstream branches, however I'll 
be happy to be corrected.

 Otherwise whether to backport or not to would be a matter of policy.  And 
policies are there to make life easier and not more difficult.  So what 
does our current actually policy say?  Well, at:
<https://sourceware.org/glibc/wiki/Release/#General_policy> I read this:

"Patch backports to stable branches are discussed on libc-stable, and any 
patch on master that doesn't change ABI or API is immediately suitable for 
backporting to a stable branch."

To me it means that a patch which does change an ABI or an API, while not 
immediately, can still be suitable for backporting, however consensus has 
to be reached.

 So what are the pros and the cons of having an ABI defined for absolute 
symbol handling?

 The pros I can identify are:

1. It matches reality.

   Previously absolute symbols were effectively completely not supported.  
   Not just failing to work in some cases or consistently working 
   incorrectly, while still being recognised.  No, they were not 
   recognised at all and treated as ordinary symbols that express memory 
   addresses and, if applicable, are relocated by the base address at load 
   time.

   Yes, from the ELF gABI's point of view this is a conformance bug fix, 
   however from GNU C library's point of view this is a new feature.

2. It is reported by running `libc.so', e.g. with the patch applied on a 
   MIPS system you get information like this:

GNU C Library (GNU libc) development release version 2.27.9000.
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 8.0.1 20180410 (experimental).
libc ABIs: MIPS_PLT UNIQUE MIPS_O32_FP64 ABSOLUTE
For bug reporting instructions, please see:
<http://www.gnu.org/software/libc/bugs.html>.

   where you can readily determine that the feature is supported, which is 
   suitable for inclusion in release notes of binary software packages 
   that have been compiled to require that feature for correct operation.

3. It replaces an obscure "undefined symbol" error message with an 
   informative "ELF file ABI version invalid" one for absolute symbols
   whose value is zero.

   Such symbols will be produced by the static linker in MIPS ELF binaries 
   for undefined weak symbols whose scope is local (and therefore have to 
   be wired to zero) and which are referred to via the GOT once the 
   upcoming fix for binutils PR ld/21375 has been applied.

   The reason for that solution is that the MIPS psABI makes the GOT 
   implicitly relocated and consequently undefined weak symbol references 
   cannot use a local GOT entry, because all such entries are relocated by 
   the base address at load time.  Consequently such references have to be
   made to a global GOT entry associated with a global absolute symbol 
   whose value is zero and remains such at load time.

   The usual approach with targets that have the GOT explicitly relocated 
   (all the non-MIPS targets I believe) is to have no dynamic R_*_RELATIVE 
   relocation attached to local GOT entries associated with undefined weak 
   symbols whose scope is local, which prevents them from being relocated 
   at load time.

   NB the fix for binutils PR ld/21375 will strive to avoid using that 
   global absolute symbol where possible by code relaxation applied to the 
   usual instructions used for GOT references, but GOT relocations are 
   generic and if used with an instruction the linker cannot interpret it 
   will have to stay there and refer to a global GOT entry associated with 
   such a symbol.

   (Granted, the message we print for an attempt to load an ELF file that 
   requests an ABI version higher than the dynamic loader supports is also  
   but obscure, however at least it is indicative and we can improve it, 
   whereas the message printed for a symbol lookup that has failed because 
   a symbol definition has been ignored is completely useless to whoever 
   has encountered it.)

4. It prevents software relying on non-zero absolute symbols from 
   malfunctioning, perhaps in a way that may be difficult to identify.

   I have no plans at the moment to arrange for the static linker to 
   recognise the general case of dynamic absolute symbols being produced 
   and to adjust the libc ABI requested accordingly.  Using dynamic 
   symbols in such a way would I think be a means to determine the 
   presence of features or to examine configuration settings among a set
   of DSOs whose configuration may vary between builds.

5. This is similar in spirit a change to the dynamic loader to the earlier 
   addition of the STB_GNU_UNIQUE ELF gABI extenstion.

   Likewise we could have no ABI version defined for STB_GNU_UNIQUE and 
   rely on an obscure error path or the lack of, wherever an old dynamic 
   loader handles modules using the feature.

 The cons I can identify are:

1. It may too unimportant a feature to deserve an ABI bump.

   The only user of the feature at the moment is the upcoming fix for 
   binutils PR ld/21375, which addresses an obscure corner case of the 
   MIPS psABI.  So its use it quite limited.

 Overall I think the maximum libc ABI version supported is not a property 
of the ABI that should prevent the change introducing it from being 
backported:

1. Bumping it up does not affect, in any way, binary software 
   whose ABI version requested is lower than or equal to the current 
   maximum ABI version.

2. It does not export itself any extra feature available to binary 
   software; that is already done by the absolute symbol handling change 
   itself.  It only announces the presence of the change, which would 
   otherwise be hidden.

3. It does affect newly-built software which will request that ABI version
   from the dynamic loader.  However that software, in the absence of a 
   way to request a new ABI version, would not work anyway with an old 
   dynamic loader.

 Therefore I believe there should be no policy issue with backporting the 
change.  Although I think that consensus is still required on whether the 
ABI update is important enough to deserve a new version.

 Comments, thoughts, questions?

  Maciej
  

On 06/28/2018 12:07 AM, Maciej W. Rozycki wrote:
>> Is this really necessary?  It essentially precludes backporting the
>> fixes.
>   Thank you for your input.  Of course bumping up the ABI version is not
> itself required for the change to work, and all existing working binaries
> will continue to.
> 
>   Technically it does not preclude backporting either, although of course
> it is limited by the previous ABI bump, which is (in the reverse order by
> version/date):

Maybe I misunderstood what you were trying to do here.

Will the EI_ABIVERSION in created binaries increase if the programmer 
simply upgrades binutils?  Or will this happen only when particular 
features are used?

Thanks,
Florian