[v3,2/8] elf: Add a tunable to control use of tagged memory

  Add a new glibc tunable: mtag.enable, bound to the environment variable
_MTAG_ENABLE.  This is a decimal constant in the range 0-255 but used
as a bit-field.

Bit 0 enables use of tagged memory in the malloc family of functions.
Bit 1 enables precise faulting of tag failure on platforms where this
can be controlled.
Other bits are currently unused, but if set will cause memory tag
checking for the current process to be enabled in the kernel.
---
 elf/dl-tunables.list |  9 +++++++++
 manual/tunables.texi | 31 +++++++++++++++++++++++++++++++
 2 files changed, 40 insertions(+)
  

On 11/23/20 9:12 PM, Richard Earnshaw via Libc-alpha wrote:
> 
> Add a new glibc tunable: mtag.enable, bound to the environment variable
> _MTAG_ENABLE.  This is a decimal constant in the range 0-255 but used
> as a bit-field.
> 
> Bit 0 enables use of tagged memory in the malloc family of functions.
> Bit 1 enables precise faulting of tag failure on platforms where this
> can be controlled.
> Other bits are currently unused, but if set will cause memory tag
> checking for the current process to be enabled in the kernel.
> ---
>   elf/dl-tunables.list |  9 +++++++++
>   manual/tunables.texi | 31 +++++++++++++++++++++++++++++++
>   2 files changed, 40 insertions(+)
> 
> diff --git a/elf/dl-tunables.list b/elf/dl-tunables.list
> index e1d8225128..652cadc334 100644
> --- a/elf/dl-tunables.list
> +++ b/elf/dl-tunables.list
> @@ -141,4 +141,13 @@ glibc {
>        default: 512
>      }
>    }
> +
> +memtag {
> +    enable {
> +      type: INT_32
> +      minval: 0
> +      maxval: 255
> +      env_alias: _MTAG_ENABLE
> +    }
> +  }
>  }
> diff --git a/manual/tunables.texi b/manual/tunables.texi
> index d72d7a5ec0..6ab432a73f 100644
> --- a/manual/tunables.texi
> +++ b/manual/tunables.texi
> @@ -36,6 +36,8 @@ their own namespace.
>  * POSIX Thread Tunables:: Tunables in the POSIX thread subsystem
>  * Hardware Capability Tunables::  Tunables that modify the hardware
>  				  capabilities seen by @theglibc{}
> +* Memory Tagging Tunables::  Tunables that control the use of hardware
> +			     memory tagging
>  @end menu
>  
>  @node Tunable names
> @@ -484,3 +486,32 @@ instead.
>  
>  This tunable is specific to i386 and x86-64.
>  @end deftp
> +
> +@node Memory Tagging Tunables
> +@section Memory Tagging Tunables
> +@cindex memory tagging tunables
> +
> +@deftp {Tunable namespace} glibc.memtag
> +If the hardware supports memory tagging, these tunables can be used to
> +control the way @theglibc{} uses this feature.  Currently, only AArch64
> +supports this feature.
> +@end deftp
> +
> +@deftp Tunable glibc.memtag.enable
> +This tunable takes a value between 0 and 255 and acts as a bitmask
> +that enables various capabilities.
> +
> +Bit 0 (the least significant bit) causes the malloc subsystem to allocate
> +tagged memory, with each allocation being assigned a random tag.
> +
> +Bit 1 enables precise faulting mode for tag violations on systems that
> +support deferred tag violation reporting.  This may cause programs
> +to run more slowly.
> +
> +Other bits are currently reserved.
> +
> +@Theglibc{} startup code will automatically enable memory tagging
> +support in the kernel if this tunable has any non-zero value.
> +
> +The default value is @samp{0}, which disables all memory tagging.
> +@end deftp

This is OK.

Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
  

On Mon, Nov 23, 2020 at 7:44 AM Richard Earnshaw via Libc-alpha
<libc-alpha@sourceware.org> wrote:
>
>
> Add a new glibc tunable: mtag.enable, bound to the environment variable
> _MTAG_ENABLE.  This is a decimal constant in the range 0-255 but used

Do we really need LD_MTAG_ENABLE?

> as a bit-field.
>
> Bit 0 enables use of tagged memory in the malloc family of functions.
> Bit 1 enables precise faulting of tag failure on platforms where this
> can be controlled.
> Other bits are currently unused, but if set will cause memory tag
> checking for the current process to be enabled in the kernel.
> ---
>  elf/dl-tunables.list |  9 +++++++++
>  manual/tunables.texi | 31 +++++++++++++++++++++++++++++++
>  2 files changed, 40 insertions(+)
>
  

On 11/25/20 10:05 PM, H.J. Lu via Libc-alpha wrote:
> On Mon, Nov 23, 2020 at 7:44 AM Richard Earnshaw via Libc-alpha
> <libc-alpha@sourceware.org> wrote:
>>
>>
>> Add a new glibc tunable: mtag.enable, bound to the environment variable
>> _MTAG_ENABLE.  This is a decimal constant in the range 0-255 but used
> 
> Do we really need LD_MTAG_ENABLE?
> 

Ahh, thanks for pointing that out, I missed that.  The env aliases are 
really just for compatibility and newer tunables should not add them.

Siddhesh
  

On 25/11/2020 16:53, Siddhesh Poyarekar wrote:
> On 11/25/20 10:05 PM, H.J. Lu via Libc-alpha wrote:
>> On Mon, Nov 23, 2020 at 7:44 AM Richard Earnshaw via Libc-alpha
>> <libc-alpha@sourceware.org> wrote:
>>>
>>>
>>> Add a new glibc tunable: mtag.enable, bound to the environment variable
>>> _MTAG_ENABLE.  This is a decimal constant in the range 0-255 but used
>>
>> Do we really need LD_MTAG_ENABLE?
>>
> 
> Ahh, thanks for pointing that out, I missed that.  The env aliases are
> really just for compatibility and newer tunables should not add them.
> 
> Siddhesh

Well,
1) We already have MALLOC_CHECK_; this is just similar but with a wider
scope because ...
2) Turning on MTE enables the feature beyond just GLIBC, since it
activates the prctl call in the kernel to permit calls to mmap to
request tagged memory.  So I think this is beyond the level of a real
tunable (though having a tunable does make things easier in the internal
logic of glibc).

R.
  

On 11/25/20 10:28 PM, Richard Earnshaw wrote:
> Well,
> 1) We already have MALLOC_CHECK_; this is just similar but with a wider
> scope because ...

glibc.malloc.check has MALLOC_CHECK_ as a compatibility env alias only 
because the latter predates tunables.

Siddhesh
  

On 25/11/2020 17:12, Siddhesh Poyarekar wrote:
> On 11/25/20 10:28 PM, Richard Earnshaw wrote:
>> Well,
>> 1) We already have MALLOC_CHECK_; this is just similar but with a wider
>> scope because ...
> 
> glibc.malloc.check has MALLOC_CHECK_ as a compatibility env alias only
> because the latter predates tunables.
> 
> Siddhesh

But that doesn't really address the issue that the scope of this feature
extends beyond just tuning.

R.
  

On 11/25/20 10:54 PM, Richard Earnshaw wrote:
> But that doesn't really address the issue that the scope of this feature
> extends beyond just tuning.

I think that's just a naming issue; it's going to be hard to avoid 
having tunables affect whole programs; I'm surprised that it's taken so 
long to discover a scenario where tunables end up modifying whole 
program behaviour rather than just the library.

Maybe it deserves a top namespace for itself, e.g. proc.memtag so that 
it's clearer that it affects the entire process and not just glibc.

Siddhesh
  

On Wed, Nov 25, 2020 at 9:49 AM Siddhesh Poyarekar <siddhesh@gotplt.org> wrote:
>
> On 11/25/20 10:54 PM, Richard Earnshaw wrote:
> > But that doesn't really address the issue that the scope of this feature
> > extends beyond just tuning.
>
> I think that's just a naming issue; it's going to be hard to avoid
> having tunables affect whole programs; I'm surprised that it's taken so
> long to discover a scenario where tunables end up modifying whole
> program behaviour rather than just the library.

glibc.cpu.x86_shstk and glibc.cpu.x86_ibt also affect the whole program.

> Maybe it deserves a top namespace for itself, e.g. proc.memtag so that
> it's clearer that it affects the entire process and not just glibc.
>
> Siddhesh
  

On 11/26/20 12:36 AM, H.J. Lu wrote:
>> I think that's just a naming issue; it's going to be hard to avoid
>> having tunables affect whole programs; I'm surprised that it's taken so
>> long to discover a scenario where tunables end up modifying whole
>> program behaviour rather than just the library.
> 
> glibc.cpu.x86_shstk and glibc.cpu.x86_ibt also affect the whole program.

Richard, since we do have precedent with CET, I'd suggest keeping the 
memory tagging tunable in the glibc namespace.  It ought to be 
glibc.mem.tagging instead of glibc.memtag.enable; glibc.mem then becomes 
home for future memory related tunables that are not limited to malloc.

Another note on this, please put the tunable into 
sysdeps/aarch64/dl-tunables.list since this is aarch64-specific at the 
moment.  We can always bring the tunable into elf/dl-tunables.list if or 
when we either implement memory tagging in software or for another 
architecture.

Siddhesh

PS: Conversations around naming are always the longest and perhaps most 
entertaining too.
  

On 26/11/2020 00:47, Siddhesh Poyarekar wrote:
> On 11/26/20 12:36 AM, H.J. Lu wrote:
>>> I think that's just a naming issue; it's going to be hard to avoid
>>> having tunables affect whole programs; I'm surprised that it's taken so
>>> long to discover a scenario where tunables end up modifying whole
>>> program behaviour rather than just the library.
>>
>> glibc.cpu.x86_shstk and glibc.cpu.x86_ibt also affect the whole program.
> 
> Richard, since we do have precedent with CET, I'd suggest keeping the
> memory tagging tunable in the glibc namespace.  It ought to be
> glibc.mem.tagging instead of glibc.memtag.enable; glibc.mem then becomes
> home for future memory related tunables that are not limited to malloc.
> 
> Another note on this, please put the tunable into
> sysdeps/aarch64/dl-tunables.list since this is aarch64-specific at the
> moment.  We can always bring the tunable into elf/dl-tunables.list if or
> when we either implement memory tagging in software or for another
> architecture.
> 

Sure, I can do that if you really think it's the right thing (I presume
this has already been done for other tunables on other architectures, so
that there isn't a lot of additional plumbing needed).  But is it?  It
seems odd to me that the generic malloc code would read a tunable that
only existed in a particular sysdep configuration.  There has to exist
some mechanism for the machine independent code to know that the tagging
behaviour is needed.

R.

> Siddhesh
> 
> PS: Conversations around naming are always the longest and perhaps most
> entertaining too.

For the record, I prefer green ;)
  

On 11/26/20 7:45 PM, Richard Earnshaw wrote:
> Sure, I can do that if you really think it's the right thing (I presume
> this has already been done for other tunables on other architectures, so

There's sysdeps/aarch64/dl-tunables.list too, so there's no additional 
plumbing needed...

> that there isn't a lot of additional plumbing needed).  But is it?  It
> seems odd to me that the generic malloc code would read a tunable that
> only existed in a particular sysdep configuration.  There has to exist
> some mechanism for the machine independent code to know that the tagging
> behaviour is needed.

... but I see your point.  How about if we look at the patchset as 
follows, which should make it more clearer.  It doesn't really change 
your patchset in any major way (other than fixing failures and review 
comments), it's only to make the story behind it and hence the design 
decisions more deliberate.

The first part of the patchset (1-3) enables infrastructure to enable 
memory tagging in glibc.  At the project level, this involves adding 
tagging calls (can't call them hooks because that name's taken and also 
invoke nightmares for some) in malloc to allow tagging malloc'd objects. 
  The tagging calls are nops in the default case but support could be 
added either at the architecture level or in the form of a software 
implementation.

The library could add more tag calls in other parts of the library to 
colour them library-internal (e.g. dynamic linker data, glibc internal 
data) but that's for later.

This basically means that memory tagging becomes a library-wide concept 
and hence the glibc.mem.tagging tunable and configury should be 
implemented project-wide, i.e. the way you've done it with your v3 
patchset with just the tunable naming changed.

The second part (6-8, assuming 4 and 5 get subsumed into 3) of the 
patchset implements aarch64 architecture support for memory tagging. 
This involves enabling tagging for the entire process using prctl at 
startup and tagging malloc'd objects.  It is unavoidable that tunables 
will eventually have processwide impact and not just in the library; 
there's precedent for that in x86 CET.

What do you think?

On a slightly different but related point, you may want to think about 
inheritance of the glibc.mem.tagging tunable when you work on v4, i.e.:

1. Should child processes inherit them?  If you're modeling it along the 
lines of MALLOC_CHECK_ (i.e. diagnostics only) then you'd want to keep 
it as default, i.e. no inheritance.  However if you model it as a 
hardening feature, you may want to set security_level to IGNORE so that 
children inherit tagging and forking doesn't become a way to escape 
tagging protections.

2. Should setxid children inherit enabled memory tagging? Again if 
you're modeling it as a hardening feature, then maybe you want to set 
security_level to NONE so that it is inherited by setxid children.  I 
think it will be the first tunable to cross that boundary if you decide 
to take that route!

Siddhesh
  

On 26/11/2020 15:27, Siddhesh Poyarekar wrote:
> On 11/26/20 7:45 PM, Richard Earnshaw wrote:
>> Sure, I can do that if you really think it's the right thing (I presume
>> this has already been done for other tunables on other architectures, so
> 
> There's sysdeps/aarch64/dl-tunables.list too, so there's no additional
> plumbing needed...
> 
>> that there isn't a lot of additional plumbing needed).  But is it?  It
>> seems odd to me that the generic malloc code would read a tunable that
>> only existed in a particular sysdep configuration.  There has to exist
>> some mechanism for the machine independent code to know that the tagging
>> behaviour is needed.
> 
> ... but I see your point.  How about if we look at the patchset as
> follows, which should make it more clearer.  It doesn't really change
> your patchset in any major way (other than fixing failures and review
> comments), it's only to make the story behind it and hence the design
> decisions more deliberate.
> 
> The first part of the patchset (1-3) enables infrastructure to enable
> memory tagging in glibc.  At the project level, this involves adding
> tagging calls (can't call them hooks because that name's taken and also
> invoke nightmares for some) in malloc to allow tagging malloc'd objects.
>  The tagging calls are nops in the default case but support could be
> added either at the architecture level or in the form of a software
> implementation.
> 
> The library could add more tag calls in other parts of the library to
> colour them library-internal (e.g. dynamic linker data, glibc internal
> data) but that's for later.
> 
> This basically means that memory tagging becomes a library-wide concept
> and hence the glibc.mem.tagging tunable and configury should be
> implemented project-wide, i.e. the way you've done it with your v3
> patchset with just the tunable naming changed.
> 
> The second part (6-8, assuming 4 and 5 get subsumed into 3) of the
> patchset implements aarch64 architecture support for memory tagging.
> This involves enabling tagging for the entire process using prctl at
> startup and tagging malloc'd objects.  It is unavoidable that tunables
> will eventually have processwide impact and not just in the library;
> there's precedent for that in x86 CET.
> 
> What do you think?

I think it's exactly the way the patch set was structured, I just wasn't
explicit in saying that :)

> 
> On a slightly different but related point, you may want to think about
> inheritance of the glibc.mem.tagging tunable when you work on v4, i.e.:
> 
> 1. Should child processes inherit them?  If you're modeling it along the
> lines of MALLOC_CHECK_ (i.e. diagnostics only) then you'd want to keep
> it as default, i.e. no inheritance.  However if you model it as a
> hardening feature, you may want to set security_level to IGNORE so that
> children inherit tagging and forking doesn't become a way to escape
> tagging protections.
> 
> 2. Should setxid children inherit enabled memory tagging? Again if
> you're modeling it as a hardening feature, then maybe you want to set
> security_level to NONE so that it is inherited by setxid children.  I
> think it will be the first tunable to cross that boundary if you decide
> to take that route!
> 

A good question.  I'd say at this point it's a bit more of a debugging
feature (at least until things have settled down); but longer term it
may well become a hardening feature as well.  Before we can go down that
route, though we'll need to sort out how to mark binaries that are
genuinely incompatible with MTE.  We already know that python's object
management code violates MTE assumptions, for example; either that will
need to be fixed, or we'll need a route to automatically disable MTE
when running programs like that.

So perhaps for now, we'd want to inherit it through normal fork() type
calls, but perhaps not for setxid at this stage, but we may want to
widen it later.  On the other hand, for a security feature you'd perhaps
want a more robust (harder to turn off) mechanism than just modifying a
user-level environment variable.

R.

> Siddhesh
  

On Thu, Nov 26, 2020 at 7:48 AM Richard Earnshaw
<Richard.Earnshaw@foss.arm.com> wrote:
>
> On 26/11/2020 15:27, Siddhesh Poyarekar wrote:
> > On 11/26/20 7:45 PM, Richard Earnshaw wrote:
> >> Sure, I can do that if you really think it's the right thing (I presume
> >> this has already been done for other tunables on other architectures, so
> >
> > There's sysdeps/aarch64/dl-tunables.list too, so there's no additional
> > plumbing needed...
> >
> >> that there isn't a lot of additional plumbing needed).  But is it?  It
> >> seems odd to me that the generic malloc code would read a tunable that
> >> only existed in a particular sysdep configuration.  There has to exist
> >> some mechanism for the machine independent code to know that the tagging
> >> behaviour is needed.
> >
> > ... but I see your point.  How about if we look at the patchset as
> > follows, which should make it more clearer.  It doesn't really change
> > your patchset in any major way (other than fixing failures and review
> > comments), it's only to make the story behind it and hence the design
> > decisions more deliberate.
> >
> > The first part of the patchset (1-3) enables infrastructure to enable
> > memory tagging in glibc.  At the project level, this involves adding
> > tagging calls (can't call them hooks because that name's taken and also
> > invoke nightmares for some) in malloc to allow tagging malloc'd objects.
> >  The tagging calls are nops in the default case but support could be
> > added either at the architecture level or in the form of a software
> > implementation.
> >
> > The library could add more tag calls in other parts of the library to
> > colour them library-internal (e.g. dynamic linker data, glibc internal
> > data) but that's for later.
> >
> > This basically means that memory tagging becomes a library-wide concept
> > and hence the glibc.mem.tagging tunable and configury should be
> > implemented project-wide, i.e. the way you've done it with your v3
> > patchset with just the tunable naming changed.
> >
> > The second part (6-8, assuming 4 and 5 get subsumed into 3) of the
> > patchset implements aarch64 architecture support for memory tagging.
> > This involves enabling tagging for the entire process using prctl at
> > startup and tagging malloc'd objects.  It is unavoidable that tunables
> > will eventually have processwide impact and not just in the library;
> > there's precedent for that in x86 CET.
> >
> > What do you think?
>
> I think it's exactly the way the patch set was structured, I just wasn't
> explicit in saying that :)
>
> >
> > On a slightly different but related point, you may want to think about
> > inheritance of the glibc.mem.tagging tunable when you work on v4, i.e.:
> >
> > 1. Should child processes inherit them?  If you're modeling it along the
> > lines of MALLOC_CHECK_ (i.e. diagnostics only) then you'd want to keep
> > it as default, i.e. no inheritance.  However if you model it as a
> > hardening feature, you may want to set security_level to IGNORE so that
> > children inherit tagging and forking doesn't become a way to escape
> > tagging protections.
> >
> > 2. Should setxid children inherit enabled memory tagging? Again if
> > you're modeling it as a hardening feature, then maybe you want to set
> > security_level to NONE so that it is inherited by setxid children.  I
> > think it will be the first tunable to cross that boundary if you decide
> > to take that route!
> >
>
> A good question.  I'd say at this point it's a bit more of a debugging
> feature (at least until things have settled down); but longer term it
> may well become a hardening feature as well.  Before we can go down that
> route, though we'll need to sort out how to mark binaries that are
> genuinely incompatible with MTE.  We already know that python's object
> management code violates MTE assumptions, for example; either that will
> need to be fixed, or we'll need a route to automatically disable MTE
> when running programs like that.

I think we need to address binary marking first before adding MTE to
glibc.

> So perhaps for now, we'd want to inherit it through normal fork() type
> calls, but perhaps not for setxid at this stage, but we may want to
> widen it later.  On the other hand, for a security feature you'd perhaps
> want a more robust (harder to turn off) mechanism than just modifying a
> user-level environment variable.
>
> R.
>
> > Siddhesh
>
  

On 11/26/20 9:18 PM, Richard Earnshaw wrote:
> I think it's exactly the way the patch set was structured, I just wasn't
> explicit in saying that :)

Agreed, it only reinforces the patch structure and I guess it was also 
for me to clear my own understanding of the patch structure.

> A good question.  I'd say at this point it's a bit more of a debugging
> feature (at least until things have settled down); but longer term it
> may well become a hardening feature as well.  Before we can go down that

Fair enough.

> route, though we'll need to sort out how to mark binaries that are
> genuinely incompatible with MTE.  We already know that python's object
> management code violates MTE assumptions, for example; either that will
> need to be fixed, or we'll need a route to automatically disable MTE
> when running programs like that
On 11/26/20 9:20 PM, H.J. Lu wrote:
 > I think we need to address binary marking first before adding MTE to
 > glibc.

Could you elaborate on why binary marking should block glibc support in 
MTE?  Do you see it changing the design considerably?  Other than the 
tunables (that are not guaranteed to be stable across releases) and 
configure flags, I could not spot any irreversible ABI/API impact that 
would cause issues later.

> So perhaps for now, we'd want to inherit it through normal fork() type
> calls, but perhaps not for setxid at this stage, but we may want to
> widen it later.  On the other hand, for a security feature you'd perhaps
> want a more robust (harder to turn off) mechanism than just modifying a
> user-level environment variable.

Agreed.  I had proposed systemwide tunables to address systemwide 
configuration issues some years ago.  Given how long it took me to get 
to writing tunables in the first place, I'd say another year or so for 
systemwide tunables ;)

Siddhesh
  

The 11/26/2020 15:48, Richard Earnshaw via Libc-alpha wrote:
> On 26/11/2020 15:27, Siddhesh Poyarekar wrote:
> > On a slightly different but related point, you may want to think about
> > inheritance of the glibc.mem.tagging tunable when you work on v4, i.e.:
> > 
> > 1. Should child processes inherit them?  If you're modeling it along the
> > lines of MALLOC_CHECK_ (i.e. diagnostics only) then you'd want to keep
> > it as default, i.e. no inheritance.  However if you model it as a
> > hardening feature, you may want to set security_level to IGNORE so that
> > children inherit tagging and forking doesn't become a way to escape
> > tagging protections.
> > 
> > 2. Should setxid children inherit enabled memory tagging? Again if
> > you're modeling it as a hardening feature, then maybe you want to set
> > security_level to NONE so that it is inherited by setxid children.  I
> > think it will be the first tunable to cross that boundary if you decide
> > to take that route!
> > 
> 
> A good question.  I'd say at this point it's a bit more of a debugging
> feature (at least until things have settled down); but longer term it
> may well become a hardening feature as well.  Before we can go down that
> route, though we'll need to sort out how to mark binaries that are
> genuinely incompatible with MTE.  We already know that python's object
> management code violates MTE assumptions, for example; either that will
> need to be fixed, or we'll need a route to automatically disable MTE
> when running programs like that.
> 
> So perhaps for now, we'd want to inherit it through normal fork() type
> calls, but perhaps not for setxid at this stage, but we may want to
> widen it later.  On the other hand, for a security feature you'd perhaps
> want a more robust (harder to turn off) mechanism than just modifying a
> user-level environment variable.

fork must inherit it because otherwise existing heap
pointers don't work with free/realloc.

for now i'd ignore the env var for setxid binaries
(though in principle it should be fine: it is supposed
to be a hardenning feature).
  

On Thu, Nov 26, 2020 at 8:04 AM Siddhesh Poyarekar <siddhesh@gotplt.org> wrote:
> On 11/26/20 9:20 PM, H.J. Lu wrote:
>  > I think we need to address binary marking first before adding MTE to
>  > glibc.
>
> Could you elaborate on why binary marking should block glibc support in
> MTE?  Do you see it changing the design considerably?  Other than the
> tunables (that are not guaranteed to be stable across releases) and
> configure flags, I could not spot any irreversible ABI/API impact that
> would cause issues later.
>

The ultimate goal is to have a single glibc binary which runs everywhere.
Glibc needs to handle static executables, dynamic executables as well as
dlopened shared objects.  Initially, no binaries are marked and memory
tag should be disabled by default.  Tunable can be used to enable memory
tag manually at run-time.  We don't know if there are any issues in the current
patches without considering the memory tag marker support.
  

On 26/11/2020 15:50, H.J. Lu wrote:
> On Thu, Nov 26, 2020 at 7:48 AM Richard Earnshaw
> <Richard.Earnshaw@foss.arm.com> wrote:
>>
>> On 26/11/2020 15:27, Siddhesh Poyarekar wrote:
>>> On 11/26/20 7:45 PM, Richard Earnshaw wrote:
>>>> Sure, I can do that if you really think it's the right thing (I presume
>>>> this has already been done for other tunables on other architectures, so
>>>
>>> There's sysdeps/aarch64/dl-tunables.list too, so there's no additional
>>> plumbing needed...
>>>
>>>> that there isn't a lot of additional plumbing needed).  But is it?  It
>>>> seems odd to me that the generic malloc code would read a tunable that
>>>> only existed in a particular sysdep configuration.  There has to exist
>>>> some mechanism for the machine independent code to know that the tagging
>>>> behaviour is needed.
>>>
>>> ... but I see your point.  How about if we look at the patchset as
>>> follows, which should make it more clearer.  It doesn't really change
>>> your patchset in any major way (other than fixing failures and review
>>> comments), it's only to make the story behind it and hence the design
>>> decisions more deliberate.
>>>
>>> The first part of the patchset (1-3) enables infrastructure to enable
>>> memory tagging in glibc.  At the project level, this involves adding
>>> tagging calls (can't call them hooks because that name's taken and also
>>> invoke nightmares for some) in malloc to allow tagging malloc'd objects.
>>>  The tagging calls are nops in the default case but support could be
>>> added either at the architecture level or in the form of a software
>>> implementation.
>>>
>>> The library could add more tag calls in other parts of the library to
>>> colour them library-internal (e.g. dynamic linker data, glibc internal
>>> data) but that's for later.
>>>
>>> This basically means that memory tagging becomes a library-wide concept
>>> and hence the glibc.mem.tagging tunable and configury should be
>>> implemented project-wide, i.e. the way you've done it with your v3
>>> patchset with just the tunable naming changed.
>>>
>>> The second part (6-8, assuming 4 and 5 get subsumed into 3) of the
>>> patchset implements aarch64 architecture support for memory tagging.
>>> This involves enabling tagging for the entire process using prctl at
>>> startup and tagging malloc'd objects.  It is unavoidable that tunables
>>> will eventually have processwide impact and not just in the library;
>>> there's precedent for that in x86 CET.
>>>
>>> What do you think?
>>
>> I think it's exactly the way the patch set was structured, I just wasn't
>> explicit in saying that :)
>>
>>>
>>> On a slightly different but related point, you may want to think about
>>> inheritance of the glibc.mem.tagging tunable when you work on v4, i.e.:
>>>
>>> 1. Should child processes inherit them?  If you're modeling it along the
>>> lines of MALLOC_CHECK_ (i.e. diagnostics only) then you'd want to keep
>>> it as default, i.e. no inheritance.  However if you model it as a
>>> hardening feature, you may want to set security_level to IGNORE so that
>>> children inherit tagging and forking doesn't become a way to escape
>>> tagging protections.
>>>
>>> 2. Should setxid children inherit enabled memory tagging? Again if
>>> you're modeling it as a hardening feature, then maybe you want to set
>>> security_level to NONE so that it is inherited by setxid children.  I
>>> think it will be the first tunable to cross that boundary if you decide
>>> to take that route!
>>>
>>
>> A good question.  I'd say at this point it's a bit more of a debugging
>> feature (at least until things have settled down); but longer term it
>> may well become a hardening feature as well.  Before we can go down that
>> route, though we'll need to sort out how to mark binaries that are
>> genuinely incompatible with MTE.  We already know that python's object
>> management code violates MTE assumptions, for example; either that will
>> need to be fixed, or we'll need a route to automatically disable MTE
>> when running programs like that.
> 
> I think we need to address binary marking first before adding MTE to
> glibc.

Sorry, I disagree.  While this is a debugging tool there's no need for
binary marking.

Once we have a clearer understanding of what's needed there, we can work
out how best to do the marking.

R.

> 
>> So perhaps for now, we'd want to inherit it through normal fork() type
>> calls, but perhaps not for setxid at this stage, but we may want to
>> widen it later.  On the other hand, for a security feature you'd perhaps
>> want a more robust (harder to turn off) mechanism than just modifying a
>> user-level environment variable.
>>
>> R.
>>
>>> Siddhesh
>>
> 
>
  

On Thu, Nov 26, 2020 at 8:28 AM Richard Earnshaw
<Richard.Earnshaw@foss.arm.com> wrote:
>
> On 26/11/2020 15:50, H.J. Lu wrote:
> > On Thu, Nov 26, 2020 at 7:48 AM Richard Earnshaw
> > <Richard.Earnshaw@foss.arm.com> wrote:
> >>
> >> On 26/11/2020 15:27, Siddhesh Poyarekar wrote:
> >>> On 11/26/20 7:45 PM, Richard Earnshaw wrote:
> >>>> Sure, I can do that if you really think it's the right thing (I presume
> >>>> this has already been done for other tunables on other architectures, so
> >>>
> >>> There's sysdeps/aarch64/dl-tunables.list too, so there's no additional
> >>> plumbing needed...
> >>>
> >>>> that there isn't a lot of additional plumbing needed).  But is it?  It
> >>>> seems odd to me that the generic malloc code would read a tunable that
> >>>> only existed in a particular sysdep configuration.  There has to exist
> >>>> some mechanism for the machine independent code to know that the tagging
> >>>> behaviour is needed.
> >>>
> >>> ... but I see your point.  How about if we look at the patchset as
> >>> follows, which should make it more clearer.  It doesn't really change
> >>> your patchset in any major way (other than fixing failures and review
> >>> comments), it's only to make the story behind it and hence the design
> >>> decisions more deliberate.
> >>>
> >>> The first part of the patchset (1-3) enables infrastructure to enable
> >>> memory tagging in glibc.  At the project level, this involves adding
> >>> tagging calls (can't call them hooks because that name's taken and also
> >>> invoke nightmares for some) in malloc to allow tagging malloc'd objects.
> >>>  The tagging calls are nops in the default case but support could be
> >>> added either at the architecture level or in the form of a software
> >>> implementation.
> >>>
> >>> The library could add more tag calls in other parts of the library to
> >>> colour them library-internal (e.g. dynamic linker data, glibc internal
> >>> data) but that's for later.
> >>>
> >>> This basically means that memory tagging becomes a library-wide concept
> >>> and hence the glibc.mem.tagging tunable and configury should be
> >>> implemented project-wide, i.e. the way you've done it with your v3
> >>> patchset with just the tunable naming changed.
> >>>
> >>> The second part (6-8, assuming 4 and 5 get subsumed into 3) of the
> >>> patchset implements aarch64 architecture support for memory tagging.
> >>> This involves enabling tagging for the entire process using prctl at
> >>> startup and tagging malloc'd objects.  It is unavoidable that tunables
> >>> will eventually have processwide impact and not just in the library;
> >>> there's precedent for that in x86 CET.
> >>>
> >>> What do you think?
> >>
> >> I think it's exactly the way the patch set was structured, I just wasn't
> >> explicit in saying that :)
> >>
> >>>
> >>> On a slightly different but related point, you may want to think about
> >>> inheritance of the glibc.mem.tagging tunable when you work on v4, i.e.:
> >>>
> >>> 1. Should child processes inherit them?  If you're modeling it along the
> >>> lines of MALLOC_CHECK_ (i.e. diagnostics only) then you'd want to keep
> >>> it as default, i.e. no inheritance.  However if you model it as a
> >>> hardening feature, you may want to set security_level to IGNORE so that
> >>> children inherit tagging and forking doesn't become a way to escape
> >>> tagging protections.
> >>>
> >>> 2. Should setxid children inherit enabled memory tagging? Again if
> >>> you're modeling it as a hardening feature, then maybe you want to set
> >>> security_level to NONE so that it is inherited by setxid children.  I
> >>> think it will be the first tunable to cross that boundary if you decide
> >>> to take that route!
> >>>
> >>
> >> A good question.  I'd say at this point it's a bit more of a debugging
> >> feature (at least until things have settled down); but longer term it
> >> may well become a hardening feature as well.  Before we can go down that
> >> route, though we'll need to sort out how to mark binaries that are
> >> genuinely incompatible with MTE.  We already know that python's object
> >> management code violates MTE assumptions, for example; either that will
> >> need to be fixed, or we'll need a route to automatically disable MTE
> >> when running programs like that.
> >
> > I think we need to address binary marking first before adding MTE to
> > glibc.
>
> Sorry, I disagree.  While this is a debugging tool there's no need for
> binary marking.
>
> Once we have a clearer understanding of what's needed there, we can work
> out how best to do the marking.
>

If the only goal is to have a malloc with memory tag, you should enable
memory tag in an out-of-tree malloc implementation.  It should be sufficient.
  

On 26/11/2020 16:51, H.J. Lu wrote:
> On Thu, Nov 26, 2020 at 8:28 AM Richard Earnshaw
> <Richard.Earnshaw@foss.arm.com> wrote:
>>
>> On 26/11/2020 15:50, H.J. Lu wrote:
>>> On Thu, Nov 26, 2020 at 7:48 AM Richard Earnshaw
>>> <Richard.Earnshaw@foss.arm.com> wrote:
>>>>
>>>> On 26/11/2020 15:27, Siddhesh Poyarekar wrote:
>>>>> On 11/26/20 7:45 PM, Richard Earnshaw wrote:
>>>>>> Sure, I can do that if you really think it's the right thing (I presume
>>>>>> this has already been done for other tunables on other architectures, so
>>>>>
>>>>> There's sysdeps/aarch64/dl-tunables.list too, so there's no additional
>>>>> plumbing needed...
>>>>>
>>>>>> that there isn't a lot of additional plumbing needed).  But is it?  It
>>>>>> seems odd to me that the generic malloc code would read a tunable that
>>>>>> only existed in a particular sysdep configuration.  There has to exist
>>>>>> some mechanism for the machine independent code to know that the tagging
>>>>>> behaviour is needed.
>>>>>
>>>>> ... but I see your point.  How about if we look at the patchset as
>>>>> follows, which should make it more clearer.  It doesn't really change
>>>>> your patchset in any major way (other than fixing failures and review
>>>>> comments), it's only to make the story behind it and hence the design
>>>>> decisions more deliberate.
>>>>>
>>>>> The first part of the patchset (1-3) enables infrastructure to enable
>>>>> memory tagging in glibc.  At the project level, this involves adding
>>>>> tagging calls (can't call them hooks because that name's taken and also
>>>>> invoke nightmares for some) in malloc to allow tagging malloc'd objects.
>>>>>  The tagging calls are nops in the default case but support could be
>>>>> added either at the architecture level or in the form of a software
>>>>> implementation.
>>>>>
>>>>> The library could add more tag calls in other parts of the library to
>>>>> colour them library-internal (e.g. dynamic linker data, glibc internal
>>>>> data) but that's for later.
>>>>>
>>>>> This basically means that memory tagging becomes a library-wide concept
>>>>> and hence the glibc.mem.tagging tunable and configury should be
>>>>> implemented project-wide, i.e. the way you've done it with your v3
>>>>> patchset with just the tunable naming changed.
>>>>>
>>>>> The second part (6-8, assuming 4 and 5 get subsumed into 3) of the
>>>>> patchset implements aarch64 architecture support for memory tagging.
>>>>> This involves enabling tagging for the entire process using prctl at
>>>>> startup and tagging malloc'd objects.  It is unavoidable that tunables
>>>>> will eventually have processwide impact and not just in the library;
>>>>> there's precedent for that in x86 CET.
>>>>>
>>>>> What do you think?
>>>>
>>>> I think it's exactly the way the patch set was structured, I just wasn't
>>>> explicit in saying that :)
>>>>
>>>>>
>>>>> On a slightly different but related point, you may want to think about
>>>>> inheritance of the glibc.mem.tagging tunable when you work on v4, i.e.:
>>>>>
>>>>> 1. Should child processes inherit them?  If you're modeling it along the
>>>>> lines of MALLOC_CHECK_ (i.e. diagnostics only) then you'd want to keep
>>>>> it as default, i.e. no inheritance.  However if you model it as a
>>>>> hardening feature, you may want to set security_level to IGNORE so that
>>>>> children inherit tagging and forking doesn't become a way to escape
>>>>> tagging protections.
>>>>>
>>>>> 2. Should setxid children inherit enabled memory tagging? Again if
>>>>> you're modeling it as a hardening feature, then maybe you want to set
>>>>> security_level to NONE so that it is inherited by setxid children.  I
>>>>> think it will be the first tunable to cross that boundary if you decide
>>>>> to take that route!
>>>>>
>>>>
>>>> A good question.  I'd say at this point it's a bit more of a debugging
>>>> feature (at least until things have settled down); but longer term it
>>>> may well become a hardening feature as well.  Before we can go down that
>>>> route, though we'll need to sort out how to mark binaries that are
>>>> genuinely incompatible with MTE.  We already know that python's object
>>>> management code violates MTE assumptions, for example; either that will
>>>> need to be fixed, or we'll need a route to automatically disable MTE
>>>> when running programs like that.
>>>
>>> I think we need to address binary marking first before adding MTE to
>>> glibc.
>>
>> Sorry, I disagree.  While this is a debugging tool there's no need for
>> binary marking.
>>
>> Once we have a clearer understanding of what's needed there, we can work
>> out how best to do the marking.
>>
> 
> If the only goal is to have a malloc with memory tag, you should enable
> memory tag in an out-of-tree malloc implementation.  It should be sufficient.
> 

It's not the only goal.  What's more that would require special linking,
rather than just being able to turn on the feature.

Initially, that might be the main usage, but it's not the ultimate goal.

R.
  

On Thu, Nov 26, 2020 at 9:00 AM Richard Earnshaw
<Richard.Earnshaw@foss.arm.com> wrote:
>
> On 26/11/2020 16:51, H.J. Lu wrote:
> > On Thu, Nov 26, 2020 at 8:28 AM Richard Earnshaw
> > <Richard.Earnshaw@foss.arm.com> wrote:
> >>
> >> On 26/11/2020 15:50, H.J. Lu wrote:
> >>> On Thu, Nov 26, 2020 at 7:48 AM Richard Earnshaw
> >>> <Richard.Earnshaw@foss.arm.com> wrote:
> >>>>
> >>>> On 26/11/2020 15:27, Siddhesh Poyarekar wrote:
> >>>>> On 11/26/20 7:45 PM, Richard Earnshaw wrote:
> >>>>>> Sure, I can do that if you really think it's the right thing (I presume
> >>>>>> this has already been done for other tunables on other architectures, so
> >>>>>
> >>>>> There's sysdeps/aarch64/dl-tunables.list too, so there's no additional
> >>>>> plumbing needed...
> >>>>>
> >>>>>> that there isn't a lot of additional plumbing needed).  But is it?  It
> >>>>>> seems odd to me that the generic malloc code would read a tunable that
> >>>>>> only existed in a particular sysdep configuration.  There has to exist
> >>>>>> some mechanism for the machine independent code to know that the tagging
> >>>>>> behaviour is needed.
> >>>>>
> >>>>> ... but I see your point.  How about if we look at the patchset as
> >>>>> follows, which should make it more clearer.  It doesn't really change
> >>>>> your patchset in any major way (other than fixing failures and review
> >>>>> comments), it's only to make the story behind it and hence the design
> >>>>> decisions more deliberate.
> >>>>>
> >>>>> The first part of the patchset (1-3) enables infrastructure to enable
> >>>>> memory tagging in glibc.  At the project level, this involves adding
> >>>>> tagging calls (can't call them hooks because that name's taken and also
> >>>>> invoke nightmares for some) in malloc to allow tagging malloc'd objects.
> >>>>>  The tagging calls are nops in the default case but support could be
> >>>>> added either at the architecture level or in the form of a software
> >>>>> implementation.
> >>>>>
> >>>>> The library could add more tag calls in other parts of the library to
> >>>>> colour them library-internal (e.g. dynamic linker data, glibc internal
> >>>>> data) but that's for later.
> >>>>>
> >>>>> This basically means that memory tagging becomes a library-wide concept
> >>>>> and hence the glibc.mem.tagging tunable and configury should be
> >>>>> implemented project-wide, i.e. the way you've done it with your v3
> >>>>> patchset with just the tunable naming changed.
> >>>>>
> >>>>> The second part (6-8, assuming 4 and 5 get subsumed into 3) of the
> >>>>> patchset implements aarch64 architecture support for memory tagging.
> >>>>> This involves enabling tagging for the entire process using prctl at
> >>>>> startup and tagging malloc'd objects.  It is unavoidable that tunables
> >>>>> will eventually have processwide impact and not just in the library;
> >>>>> there's precedent for that in x86 CET.
> >>>>>
> >>>>> What do you think?
> >>>>
> >>>> I think it's exactly the way the patch set was structured, I just wasn't
> >>>> explicit in saying that :)
> >>>>
> >>>>>
> >>>>> On a slightly different but related point, you may want to think about
> >>>>> inheritance of the glibc.mem.tagging tunable when you work on v4, i.e.:
> >>>>>
> >>>>> 1. Should child processes inherit them?  If you're modeling it along the
> >>>>> lines of MALLOC_CHECK_ (i.e. diagnostics only) then you'd want to keep
> >>>>> it as default, i.e. no inheritance.  However if you model it as a
> >>>>> hardening feature, you may want to set security_level to IGNORE so that
> >>>>> children inherit tagging and forking doesn't become a way to escape
> >>>>> tagging protections.
> >>>>>
> >>>>> 2. Should setxid children inherit enabled memory tagging? Again if
> >>>>> you're modeling it as a hardening feature, then maybe you want to set
> >>>>> security_level to NONE so that it is inherited by setxid children.  I
> >>>>> think it will be the first tunable to cross that boundary if you decide
> >>>>> to take that route!
> >>>>>
> >>>>
> >>>> A good question.  I'd say at this point it's a bit more of a debugging
> >>>> feature (at least until things have settled down); but longer term it
> >>>> may well become a hardening feature as well.  Before we can go down that
> >>>> route, though we'll need to sort out how to mark binaries that are
> >>>> genuinely incompatible with MTE.  We already know that python's object
> >>>> management code violates MTE assumptions, for example; either that will
> >>>> need to be fixed, or we'll need a route to automatically disable MTE
> >>>> when running programs like that.
> >>>
> >>> I think we need to address binary marking first before adding MTE to
> >>> glibc.
> >>
> >> Sorry, I disagree.  While this is a debugging tool there's no need for
> >> binary marking.
> >>
> >> Once we have a clearer understanding of what's needed there, we can work
> >> out how best to do the marking.
> >>
> >
> > If the only goal is to have a malloc with memory tag, you should enable
> > memory tag in an out-of-tree malloc implementation.  It should be sufficient.
> >
>
> It's not the only goal.  What's more that would require special linking,
> rather than just being able to turn on the feature.
>
> Initially, that might be the main usage, but it's not the ultimate goal.
>

What is your ultimate goal? What is your initial goal?
  

On 11/26/20 9:49 PM, H.J. Lu wrote:
> The ultimate goal is to have a single glibc binary which runs everywhere.
> Glibc needs to handle static executables, dynamic executables as well as
> dlopened shared objects.  Initially, no binaries are marked and memory
> tag should be disabled by default.  Tunable can be used to enable memory

The patchset seems to tick all those boxes.

> tag manually at run-time.  We don't know if there are any issues in the current
> patches without considering the memory tag marker support.

Sure, which is why I asked the question so that we can discuss how 
memory tag marker support in the binary would impact this feature. 
Could you please elaborate on that because I still don't see it.  I can 
see how having binaries marked to always run with tagging enabled could 
be a good way to ensure that they're always executed that way, but it's 
something that could get added on top of the current patchset and 
perhaps even backported since it doesn't affect ABI.

Siddhesh
  

On Thu, Nov 26, 2020 at 9:14 AM Siddhesh Poyarekar <siddhesh@gotplt.org> wrote:
>
> On 11/26/20 9:49 PM, H.J. Lu wrote:
> > The ultimate goal is to have a single glibc binary which runs everywhere.
> > Glibc needs to handle static executables, dynamic executables as well as
> > dlopened shared objects.  Initially, no binaries are marked and memory
> > tag should be disabled by default.  Tunable can be used to enable memory
>
> The patchset seems to tick all those boxes.
>
> > tag manually at run-time.  We don't know if there are any issues in the current
> > patches without considering the memory tag marker support.
>
> Sure, which is why I asked the question so that we can discuss how
> memory tag marker support in the binary would impact this feature.
> Could you please elaborate on that because I still don't see it.  I can

The first few questions are

1.  Where should binary markers be checked?
2.  How should binary marker checking work together with tunables?

> see how having binaries marked to always run with tagging enabled could
> be a good way to ensure that they're always executed that way, but it's
> something that could get added on top of the current patchset and
> perhaps even backported since it doesn't affect ABI.
>
> Siddhesh
  

The 11/26/2020 08:51, H.J. Lu via Libc-alpha wrote:
> On Thu, Nov 26, 2020 at 8:28 AM Richard Earnshaw
> <Richard.Earnshaw@foss.arm.com> wrote:
> > On 26/11/2020 15:50, H.J. Lu wrote:
> > > I think we need to address binary marking first before adding MTE to
> > > glibc.
> >
> > Sorry, I disagree.  While this is a debugging tool there's no need for
> > binary marking.
> >
> > Once we have a clearer understanding of what's needed there, we can work
> > out how best to do the marking.
> 
> If the only goal is to have a malloc with memory tag, you should enable
> memory tag in an out-of-tree malloc implementation.  It should be sufficient.

out-of-tree malloc implementation cannot reliably
enable mte: it cannot guarantee that it can initialize
mte before the process becomes multi-threaded or that
it can execute code in all threads before allocated
memory is accessed there, so mte checks can be off
for some threads.

so the mte on/off setting has to be in glibc for now
(out-of-tree implementation is possible, but it won't
be reliable in all cases).
  

On Thu, Nov 26, 2020 at 9:21 AM Szabolcs Nagy <szabolcs.nagy@arm.com> wrote:
>
> The 11/26/2020 08:51, H.J. Lu via Libc-alpha wrote:
> > On Thu, Nov 26, 2020 at 8:28 AM Richard Earnshaw
> > <Richard.Earnshaw@foss.arm.com> wrote:
> > > On 26/11/2020 15:50, H.J. Lu wrote:
> > > > I think we need to address binary marking first before adding MTE to
> > > > glibc.
> > >
> > > Sorry, I disagree.  While this is a debugging tool there's no need for
> > > binary marking.
> > >
> > > Once we have a clearer understanding of what's needed there, we can work
> > > out how best to do the marking.
> >
> > If the only goal is to have a malloc with memory tag, you should enable
> > memory tag in an out-of-tree malloc implementation.  It should be sufficient.
>
> out-of-tree malloc implementation cannot reliably
> enable mte: it cannot guarantee that it can initialize
> mte before the process becomes multi-threaded or that
> it can execute code in all threads before allocated
> memory is accessed there, so mte checks can be off
> for some threads.
>
> so the mte on/off setting has to be in glibc for now
> (out-of-tree implementation is possible, but it won't
> be reliable in all cases).

I don't disagree with it.   If the new libc will be installed as the
system glibc, it needs to support binary marker even if there
are currently no binaries with marker.
  

On 26/11/2020 17:31, H.J. Lu via Libc-alpha wrote:
> On Thu, Nov 26, 2020 at 9:21 AM Szabolcs Nagy <szabolcs.nagy@arm.com> wrote:
>>
>> The 11/26/2020 08:51, H.J. Lu via Libc-alpha wrote:
>>> On Thu, Nov 26, 2020 at 8:28 AM Richard Earnshaw
>>> <Richard.Earnshaw@foss.arm.com> wrote:
>>>> On 26/11/2020 15:50, H.J. Lu wrote:
>>>>> I think we need to address binary marking first before adding MTE to
>>>>> glibc.
>>>>
>>>> Sorry, I disagree.  While this is a debugging tool there's no need for
>>>> binary marking.
>>>>
>>>> Once we have a clearer understanding of what's needed there, we can work
>>>> out how best to do the marking.
>>>
>>> If the only goal is to have a malloc with memory tag, you should enable
>>> memory tag in an out-of-tree malloc implementation.  It should be sufficient.
>>
>> out-of-tree malloc implementation cannot reliably
>> enable mte: it cannot guarantee that it can initialize
>> mte before the process becomes multi-threaded or that
>> it can execute code in all threads before allocated
>> memory is accessed there, so mte checks can be off
>> for some threads.
>>
>> so the mte on/off setting has to be in glibc for now
>> (out-of-tree implementation is possible, but it won't
>> be reliable in all cases).
> 
> I don't disagree with it.   If the new libc will be installed as the
> system glibc, it needs to support binary marker even if there
> are currently no binaries with marker.
> 

Trying to define a binary marker before we know exactly how we want the
binary marker to work is just a good way to define the wrong binary marker.

And for the record, I don't think we know how we'd want that to work at
this point, nor do I think it's necessary for the code proposed to work
as intended.

So I think this is trying to put the cart before the horse.

R.
  

The 11/26/2020 09:31, H.J. Lu wrote:
> I don't disagree with it.   If the new libc will be installed as the
> system glibc, it needs to support binary marker even if there
> are currently no binaries with marker.

i don't yet see how things break if the marking
support comes in a later glibc.

for unmarked binaries the env var behaviour is
fixed and backward compatible.

marked binaries will depend on a new glibc, we
currently don't have a way to do an abi bump
on a binary such that old glibc rejects it
(other than using symbol references or new dyn
relc type), so we may end up with silent breakage
if a new marked binary is used with old glibc.
this is not too bad, but if this is a concern
then e.g. we can do an e_flags check in aarch64
so new flags there are always rejected which will
allow us to bump the mte abi.
  

On Thu, Nov 26, 2020 at 9:56 AM Richard Earnshaw
<Richard.Earnshaw@foss.arm.com> wrote:
>
> On 26/11/2020 17:31, H.J. Lu via Libc-alpha wrote:
> > On Thu, Nov 26, 2020 at 9:21 AM Szabolcs Nagy <szabolcs.nagy@arm.com> wrote:
> >>
> >> The 11/26/2020 08:51, H.J. Lu via Libc-alpha wrote:
> >>> On Thu, Nov 26, 2020 at 8:28 AM Richard Earnshaw
> >>> <Richard.Earnshaw@foss.arm.com> wrote:
> >>>> On 26/11/2020 15:50, H.J. Lu wrote:
> >>>>> I think we need to address binary marking first before adding MTE to
> >>>>> glibc.
> >>>>
> >>>> Sorry, I disagree.  While this is a debugging tool there's no need for
> >>>> binary marking.
> >>>>
> >>>> Once we have a clearer understanding of what's needed there, we can work
> >>>> out how best to do the marking.
> >>>
> >>> If the only goal is to have a malloc with memory tag, you should enable
> >>> memory tag in an out-of-tree malloc implementation.  It should be sufficient.
> >>
> >> out-of-tree malloc implementation cannot reliably
> >> enable mte: it cannot guarantee that it can initialize
> >> mte before the process becomes multi-threaded or that
> >> it can execute code in all threads before allocated
> >> memory is accessed there, so mte checks can be off
> >> for some threads.
> >>
> >> so the mte on/off setting has to be in glibc for now
> >> (out-of-tree implementation is possible, but it won't
> >> be reliable in all cases).
> >
> > I don't disagree with it.   If the new libc will be installed as the
> > system glibc, it needs to support binary marker even if there
> > are currently no binaries with marker.
> >
>
> Trying to define a binary marker before we know exactly how we want the
> binary marker to work is just a good way to define the wrong binary marker.
>
> And for the record, I don't think we know how we'd want that to work at
> this point, nor do I think it's necessary for the code proposed to work
> as intended.

I believe this should be addressed first.

> So I think this is trying to put the cart before the horse.
>
> R.
  

On Thu, Nov 26, 2020 at 10:06 AM Szabolcs Nagy <szabolcs.nagy@arm.com> wrote:
>
> The 11/26/2020 09:31, H.J. Lu wrote:
> > I don't disagree with it.   If the new libc will be installed as the
> > system glibc, it needs to support binary marker even if there
> > are currently no binaries with marker.
>
> i don't yet see how things break if the marking
> support comes in a later glibc.
>

We should anticipate such changes.

> for unmarked binaries the env var behaviour is
> fixed and backward compatible.
>
> marked binaries will depend on a new glibc, we
> currently don't have a way to do an abi bump
> on a binary such that old glibc rejects it
> (other than using symbol references or new dyn
> relc type), so we may end up with silent breakage
> if a new marked binary is used with old glibc.
> this is not too bad, but if this is a concern
> then e.g. we can do an e_flags check in aarch64
> so new flags there are always rejected which will
> allow us to bump the mte abi.

CET enabled binaries don't have such issues.  They are
backward and forward compatible.
  

On Nov 26 2020, Szabolcs Nagy via Libc-alpha wrote:

> marked binaries will depend on a new glibc, we
> currently don't have a way to do an abi bump
> on a binary such that old glibc rejects it

EI_ABIVERSION

Andreas.
  

On 11/26/20 10:49 PM, H.J. Lu wrote:
> The first few questions are

OK this is a good start:

> 1.  Where should binary markers be checked?

At early startup alongside the cpu features resolution.  We enable 
tagging if the CPU supports MTE and the marker is set.

> 2.  How should binary marker checking work together with tunables?

The presence of a binary marker enables tagging and a tunable should not 
be able to disable it.  The exception would be systemwide tunables[1] 
where administrators could set sweeping policies for their systems, 
including disabling tagging systemwide if needed.

If binary marker is not present, tunables behave the way it is proposed 
in the patchset.

Siddhesh

[1] Vapourware alert!
  

On 11/26/20 11:36 PM, Szabolcs Nagy via Libc-alpha wrote:
> The 11/26/2020 09:31, H.J. Lu wrote:
>> I don't disagree with it.   If the new libc will be installed as the
>> system glibc, it needs to support binary marker even if there
>> are currently no binaries with marker.

As long as the marker is defined in a way that doesn't affect ABI, it 
can be backported.

> i don't yet see how things break if the marking
> support comes in a later glibc.
> 
> for unmarked binaries the env var behaviour is
> fixed and backward compatible.
> 
> marked binaries will depend on a new glibc, we
> currently don't have a way to do an abi bump
> on a binary such that old glibc rejects it
> (other than using symbol references or new dyn
> relc type), so we may end up with silent breakage
> if a new marked binary is used with old glibc.

What kind of breakage?  Just that tagging won't work, which is a 
graceful fallback IMO, the same as what CET does.

> this is not too bad, but if this is a concern
> then e.g. we can do an e_flags check in aarch64
> so new flags there are always rejected which will
> allow us to bump the mte abi.

The ABI bump is precisely what you want to avoid if you're looking for 
the binaries to be usable across systems.

Siddhesh
  

The 11/27/2020 08:29, Siddhesh Poyarekar wrote:
> On 11/26/20 11:36 PM, Szabolcs Nagy via Libc-alpha wrote:
> > i don't yet see how things break if the marking
> > support comes in a later glibc.
> > 
> > for unmarked binaries the env var behaviour is
> > fixed and backward compatible.
> > 
> > marked binaries will depend on a new glibc, we
> > currently don't have a way to do an abi bump
> > on a binary such that old glibc rejects it
> > (other than using symbol references or new dyn
> > relc type), so we may end up with silent breakage
> > if a new marked binary is used with old glibc.
> 
> What kind of breakage?  Just that tagging won't work, which is a graceful
> fallback IMO, the same as what CET does.

i think that's an opt-in marking. here we need an opt-out
marking if we want to turn off the effects of the env var.

(it is ok to leave heap tagging off when it should be on,
but the other direction is not backward compatible, which is
why i suggested an ABI bump to reject marked binaries.)

it is also possible to add a new env var together with the
opt-out marking. (old env var keeps ignoring the marking,
new env var handles it. but many env vars can be confusing)

or we can introduce an opt-in marking, but it's not clear
how well the transition works in practice: you have to be
sure binaries are tagging compatible to mark them and after
some long time when enough binaries are marked the marking
becomes useful. the transition with opt-out is clearer i
think (we can mark binaries as we find issues), but if our
aim is "on by default" tagging then opt-in will be needed.

in short this can go in many directions, but if we have
some insurance (e.g. abi bump mechanism) in place then we
can make the decision later.

> > this is not too bad, but if this is a concern
> > then e.g. we can do an e_flags check in aarch64
> > so new flags there are always rejected which will
> > allow us to bump the mte abi.
> 
> The ABI bump is precisely what you want to avoid if you're looking for the
> binaries to be usable across systems.

yes, we want to avoid it, but load time breakage is still
better than runtime breakage.
  

The 11/26/2020 19:25, Andreas Schwab wrote:
> On Nov 26 2020, Szabolcs Nagy via Libc-alpha wrote:
> 
> > marked binaries will depend on a new glibc, we
> > currently don't have a way to do an abi bump
> > on a binary such that old glibc rejects it
> 
> EI_ABIVERSION

that does not work for the exe unless this bug got fixed:
https://sourceware.org/legacy-ml/libc-alpha/2019-06/msg00731.html

we want to abi bump kernel loaded binaries too.
  

On 27/11/2020 02:45, Siddhesh Poyarekar wrote:
> On 11/26/20 10:49 PM, H.J. Lu wrote:
>> The first few questions are
> 
> OK this is a good start:
> 
>> 1.  Where should binary markers be checked?
> 
> At early startup alongside the cpu features resolution.  We enable
> tagging if the CPU supports MTE and the marker is set.

I think that's backwards.  The default should be to assume that a binary
supports tagging and it should only be disabled if the binary explicitly
says that it is incompatible.

Requiring a tag to be set before you enable tagging will
a) Force a complete recompile of all binaries
b) Result in binaries being incorrectly marked as tagging compatible
when they aren't (because they won't really be audited until they start
failing).  They will then have to be rebuilt again without the tag.

Given b) it then seems obvious that the right way to do this is just to
mark binaries that are known not to work; after they've been audited to
make sure that the reason they don't work is legitimate.

> 
>> 2.  How should binary marker checking work together with tunables?
> 
> The presence of a binary marker enables tagging and a tunable should not
> be able to disable it.  The exception would be systemwide tunables[1]
> where administrators could set sweeping policies for their systems,
> including disabling tagging systemwide if needed.
> 
> If binary marker is not present, tunables behave the way it is proposed
> in the patchset.
> 
> Siddhesh
> 
> [1] Vapourware alert!

R.
  

On 27/11/2020 10:40, Richard Earnshaw wrote:
> On 27/11/2020 02:45, Siddhesh Poyarekar wrote:
>> On 11/26/20 10:49 PM, H.J. Lu wrote:
>>> The first few questions are
>>
>> OK this is a good start:
>>
>>> 1.  Where should binary markers be checked?
>>
>> At early startup alongside the cpu features resolution.  We enable
>> tagging if the CPU supports MTE and the marker is set.
> 
> I think that's backwards.  The default should be to assume that a binary
> supports tagging and it should only be disabled if the binary explicitly
> says that it is incompatible.
> 
> Requiring a tag to be set before you enable tagging will
> a) Force a complete recompile of all binaries
> b) Result in binaries being incorrectly marked as tagging compatible
> when they aren't (because they won't really be audited until they start
> failing).  They will then have to be rebuilt again without the tag.
> 
> Given b) it then seems obvious that the right way to do this is just to
> mark binaries that are known not to work; after they've been audited to
> make sure that the reason they don't work is legitimate.
> 
>>
>>> 2.  How should binary marker checking work together with tunables?
>>
>> The presence of a binary marker enables tagging and a tunable should not
>> be able to disable it.  The exception would be systemwide tunables[1]
>> where administrators could set sweeping policies for their systems,
>> including disabling tagging systemwide if needed.
>>
>> If binary marker is not present, tunables behave the way it is proposed
>> in the patchset.
>>
>> Siddhesh
>>
>> [1] Vapourware alert!
> 
> R.
> 

Sorry, I meant to add that if this is to be considered a security
feature it should be an active decision to disable it for a specific
binary, not an active decision to enable it.  What's more, the marker
then can be used to quickly find binaries that are tagging unsafe when
targetting things for audit purposes.

R.
  

* Szabolcs Nagy via Libc-alpha:

> The 11/26/2020 19:25, Andreas Schwab wrote:
>> On Nov 26 2020, Szabolcs Nagy via Libc-alpha wrote:
>> 
>> > marked binaries will depend on a new glibc, we
>> > currently don't have a way to do an abi bump
>> > on a binary such that old glibc rejects it
>> 
>> EI_ABIVERSION
>
> that does not work for the exe unless this bug got fixed:
> https://sourceware.org/legacy-ml/libc-alpha/2019-06/msg00731.html
>
> we want to abi bump kernel loaded binaries too.

I posted a generic ABI proposal:

  Critical program headers and dynamic tags
  <https://groups.google.com/g/generic-abi/c/vdG_G4l3N-Y>

We can add it to the GNU ABI if it doesn't make the generic ABI.

Thanks,
Florian
  

On 11/27/20 4:02 PM, Szabolcs Nagy wrote:
> i think that's an opt-in marking. here we need an opt-out
> marking if we want to turn off the effects of the env var.

Right I assumed it to be an opt-in marker and not opt-out.

> or we can introduce an opt-in marking, but it's not clear
> how well the transition works in practice: you have to be
> sure binaries are tagging compatible to mark them and after
> some long time when enough binaries are marked the marking
> becomes useful. the transition with opt-out is clearer i
> think (we can mark binaries as we find issues), but if our
> aim is "on by default" tagging then opt-in will be needed.

The thing with opt-out though is that it's not really opt-out in the 
real sense.  That is, you need to set the tunable to opt in; and then 
the marker is allowed to override it.  If glibc.mem.tagging is enabled 
by default however, then using the marker to override and opt out makes 
sense.

Or maybe I haven't understood your implementation plan correctly and we 
need to talk more about that as HJ suggests, especially since an ABI 
event seems imminent.

> in short this can go in many directions, but if we have
> some insurance (e.g. abi bump mechanism) in place then we
> can make the decision later.

An opt in marker shouldn't need an ABI bump.

Siddhesh
  

On 11/27/20 4:10 PM, Richard Earnshaw wrote:
> I think that's backwards.  The default should be to assume that a binary
> supports tagging and it should only be disabled if the binary explicitly
> says that it is incompatible.

Sorry, I assumed an opt-in marker.  However opt-in marker being absent 
doesn't really say that the binary does not support tagging, it defers 
the decision to the tunable.

> Requiring a tag to be set before you enable tagging will
> a) Force a complete recompile of all binaries

Binaries shouldn't be needed to be rebuilt to enable tagging; the 
tunable can do that.  If you want distro-wide support for tagging, you 
enable the tunable by default and use the environment variable to opt out.

> b) Result in binaries being incorrectly marked as tagging compatible
> when they aren't (because they won't really be audited until they start
> failing).  They will then have to be rebuilt again without the tag.
> 
> Given b) it then seems obvious that the right way to do this is just to
> mark binaries that are known not to work; after they've been audited to
> make sure that the reason they don't work is legitimate.

With tunable enabled by default, an opt-out marker makes sense because 
then you only rebuild and mark binaries that don't work with tagging and 
have them opt out.

 From a distribution perspective, this could be done in two phases:

1. Phase 1 where you ave the tunable which is disabled by default and no 
marker present.  This makes tagging available to those who want to opt-in

2. Phase 2 is where the tunable is now enabled by default and can be 
overridden with an opt-out marker.  Running this opt-out binary on the 
older version is safe (and shouldn't need an ABI bump) since tunables 
are disabled by default.

Does that make sense?  If not then I'd really like to see a more 
detailed description of how you intend to roll this out.

Siddhesh
  

On 11/27/20 4:19 PM, Richard Earnshaw wrote:
> Sorry, I meant to add that if this is to be considered a security
> feature it should be an active decision to disable it for a specific
> binary, not an active decision to enable it.  What's more, the marker
> then can be used to quickly find binaries that are tagging unsafe when
> targetting things for audit purposes.

I am a bit confused with the contradictory statements about this feature 
and patchset.  Perhaps it is because you want to start out with a debug 
feature and then at some point make it mandatory and we're mixing 
conversations about both.

Thanks,
Siddhesh
  

On 27/11/2020 11:32, Siddhesh Poyarekar wrote:
> On 11/27/20 4:19 PM, Richard Earnshaw wrote:
>> Sorry, I meant to add that if this is to be considered a security
>> feature it should be an active decision to disable it for a specific
>> binary, not an active decision to enable it.  What's more, the marker
>> then can be used to quickly find binaries that are tagging unsafe when
>> targetting things for audit purposes.
> 
> I am a bit confused with the contradictory statements about this feature
> and patchset.  Perhaps it is because you want to start out with a debug
> feature and then at some point make it mandatory and we're mixing
> conversations about both.
> 
> Thanks,
> Siddhesh

Yes.  But that's because if this remains just a debugging feature, then
tagging is completely pointless.

R.
  

On 27/11/2020 11:27, Siddhesh Poyarekar wrote:
> On 11/27/20 4:10 PM, Richard Earnshaw wrote:
>> I think that's backwards.  The default should be to assume that a binary
>> supports tagging and it should only be disabled if the binary explicitly
>> says that it is incompatible.
> 
> Sorry, I assumed an opt-in marker.  However opt-in marker being absent
> doesn't really say that the binary does not support tagging, it defers
> the decision to the tunable.
> 
>> Requiring a tag to be set before you enable tagging will
>> a) Force a complete recompile of all binaries
> 
> Binaries shouldn't be needed to be rebuilt to enable tagging; the
> tunable can do that.  If you want distro-wide support for tagging, you
> enable the tunable by default and use the environment variable to opt out.
> 
>> b) Result in binaries being incorrectly marked as tagging compatible
>> when they aren't (because they won't really be audited until they start
>> failing).  They will then have to be rebuilt again without the tag.
>>
>> Given b) it then seems obvious that the right way to do this is just to
>> mark binaries that are known not to work; after they've been audited to
>> make sure that the reason they don't work is legitimate.
> 
> With tunable enabled by default, an opt-out marker makes sense because
> then you only rebuild and mark binaries that don't work with tagging and
> have them opt out.
> 
> From a distribution perspective, this could be done in two phases:
> 
> 1. Phase 1 where you ave the tunable which is disabled by default and no
> marker present.  This makes tagging available to those who want to opt-in
> 
> 2. Phase 2 is where the tunable is now enabled by default and can be
> overridden with an opt-out marker.  Running this opt-out binary on the
> older version is safe (and shouldn't need an ABI bump) since tunables
> are disabled by default.
> 
> Does that make sense?  If not then I'd really like to see a more
> detailed description of how you intend to roll this out.
> 
> Siddhesh

Yes, you've captured my thoughts and summarised it better that I was
doing myself.  Thank you.

R.
  

On Thu, Nov 26, 2020 at 6:45 PM Siddhesh Poyarekar <siddhesh@gotplt.org> wrote:
>
> On 11/26/20 10:49 PM, H.J. Lu wrote:
> > The first few questions are
>
> OK this is a good start:
>
> > 1.  Where should binary markers be checked?
>
> At early startup alongside the cpu features resolution.  We enable
> tagging if the CPU supports MTE and the marker is set.

We won't know all the issues before we implement it.

> > 2.  How should binary marker checking work together with tunables?
>
> The presence of a binary marker enables tagging and a tunable should not
> be able to disable it.  The exception would be systemwide tunables[1]
> where administrators could set sweeping policies for their systems,
> including disabling tagging systemwide if needed.

The memory tag implementation should be independent of tunables.
Tunables should just turn on and off a few bits in the memory tag
implementation.  Make the memory tag implementation depend on
tunables seems wrong to me.

> If binary marker is not present, tunables behave the way it is proposed
> in the patchset.
>
> Siddhesh
>
> [1] Vapourware alert!
  

On Fri, Nov 27, 2020 at 4:24 AM Richard Earnshaw
<Richard.Earnshaw@foss.arm.com> wrote:
>
> On 27/11/2020 11:27, Siddhesh Poyarekar wrote:
> > On 11/27/20 4:10 PM, Richard Earnshaw wrote:
> >> I think that's backwards.  The default should be to assume that a binary
> >> supports tagging and it should only be disabled if the binary explicitly
> >> says that it is incompatible.
> >
> > Sorry, I assumed an opt-in marker.  However opt-in marker being absent
> > doesn't really say that the binary does not support tagging, it defers
> > the decision to the tunable.
> >
> >> Requiring a tag to be set before you enable tagging will
> >> a) Force a complete recompile of all binaries
> >
> > Binaries shouldn't be needed to be rebuilt to enable tagging; the
> > tunable can do that.  If you want distro-wide support for tagging, you
> > enable the tunable by default and use the environment variable to opt out.
> >
> >> b) Result in binaries being incorrectly marked as tagging compatible
> >> when they aren't (because they won't really be audited until they start
> >> failing).  They will then have to be rebuilt again without the tag.
> >>
> >> Given b) it then seems obvious that the right way to do this is just to
> >> mark binaries that are known not to work; after they've been audited to
> >> make sure that the reason they don't work is legitimate.
> >
> > With tunable enabled by default, an opt-out marker makes sense because
> > then you only rebuild and mark binaries that don't work with tagging and
> > have them opt out.
> >
> > From a distribution perspective, this could be done in two phases:
> >
> > 1. Phase 1 where you ave the tunable which is disabled by default and no
> > marker present.  This makes tagging available to those who want to opt-in
> >
> > 2. Phase 2 is where the tunable is now enabled by default and can be
> > overridden with an opt-out marker.  Running this opt-out binary on the
> > older version is safe (and shouldn't need an ABI bump) since tunables
> > are disabled by default.
> >
> > Does that make sense?  If not then I'd really like to see a more
> > detailed description of how you intend to roll this out.
> >
> > Siddhesh
>
> Yes, you've captured my thoughts and summarised it better that I was
> doing myself.  Thank you.
>

We need a plan for binary marker first.
  

On 27/11/2020 14:52, H.J. Lu via Libc-alpha wrote:
> On Thu, Nov 26, 2020 at 6:45 PM Siddhesh Poyarekar <siddhesh@gotplt.org> wrote:
>>
>> On 11/26/20 10:49 PM, H.J. Lu wrote:
>>> The first few questions are
>>
>> OK this is a good start:
>>
>>> 1.  Where should binary markers be checked?
>>
>> At early startup alongside the cpu features resolution.  We enable
>> tagging if the CPU supports MTE and the marker is set.
> 
> We won't know all the issues before we implement it.
> 
>>> 2.  How should binary marker checking work together with tunables?
>>
>> The presence of a binary marker enables tagging and a tunable should not
>> be able to disable it.  The exception would be systemwide tunables[1]
>> where administrators could set sweeping policies for their systems,
>> including disabling tagging systemwide if needed.
> 
> The memory tag implementation should be independent of tunables.
> Tunables should just turn on and off a few bits in the memory tag
> implementation.  Make the memory tag implementation depend on
> tunables seems wrong to me.

That shouldn't matter.  The tunables are documented as not being stable
and nothing else is exposed to the user; so if we want to change things
later, there's nothing to stop that.

R.

> 
>> If binary marker is not present, tunables behave the way it is proposed
>> in the patchset.
>>
>> Siddhesh
>>
>> [1] Vapourware alert!
> 
> 
>
  

The 11/27/2020 06:54, H.J. Lu via Libc-alpha wrote:
> On Fri, Nov 27, 2020 at 4:24 AM Richard Earnshaw
> <Richard.Earnshaw@foss.arm.com> wrote:
> > On 27/11/2020 11:27, Siddhesh Poyarekar wrote:
> > > From a distribution perspective, this could be done in two phases:
> > >
> > > 1. Phase 1 where you ave the tunable which is disabled by default and no
> > > marker present.  This makes tagging available to those who want to opt-in
> > >
> > > 2. Phase 2 is where the tunable is now enabled by default and can be
> > > overridden with an opt-out marker.  Running this opt-out binary on the
> > > older version is safe (and shouldn't need an ABI bump) since tunables
> > > are disabled by default.
> > >
> > > Does that make sense?  If not then I'd really like to see a more
> > > detailed description of how you intend to roll this out.
> > >
> > > Siddhesh
> >
> > Yes, you've captured my thoughts and summarised it better that I was
> > doing myself.  Thank you.
> >
> 
> We need a plan for binary marker first.

if we enable heap tagging in those two phases then the first
phase has no abi commitment (tunables are not abi stable).

heap tagging will not be very useful in phase 1 but at least
users can start testing it and report back issues. (assuming
they have hardware or emulator.)

i was hoping we can have an abi stable env var ui, but i'm
fine with tunables only since that sounds the safest.
(if we wait for the marking design and binutils etc support
then we have less chance to see deployment problems early.)
  

On Fri, Nov 27, 2020 at 8:08 AM Richard Earnshaw
<Richard.Earnshaw@foss.arm.com> wrote:
>
> On 27/11/2020 14:52, H.J. Lu via Libc-alpha wrote:
> > On Thu, Nov 26, 2020 at 6:45 PM Siddhesh Poyarekar <siddhesh@gotplt.org> wrote:
> >>
> >> On 11/26/20 10:49 PM, H.J. Lu wrote:
> >>> The first few questions are
> >>
> >> OK this is a good start:
> >>
> >>> 1.  Where should binary markers be checked?
> >>
> >> At early startup alongside the cpu features resolution.  We enable
> >> tagging if the CPU supports MTE and the marker is set.
> >
> > We won't know all the issues before we implement it.
> >
> >>> 2.  How should binary marker checking work together with tunables?
> >>
> >> The presence of a binary marker enables tagging and a tunable should not
> >> be able to disable it.  The exception would be systemwide tunables[1]
> >> where administrators could set sweeping policies for their systems,
> >> including disabling tagging systemwide if needed.
> >
> > The memory tag implementation should be independent of tunables.
> > Tunables should just turn on and off a few bits in the memory tag
> > implementation.  Make the memory tag implementation depend on
> > tunables seems wrong to me.
>
> That shouldn't matter.  The tunables are documented as not being stable
> and nothing else is exposed to the user; so if we want to change things
> later, there's nothing to stop that.
>

This is not a good user experience and makes it harder to backport.
  

On Fri, Nov 27, 2020 at 9:02 AM Szabolcs Nagy <szabolcs.nagy@arm.com> wrote:
>
> The 11/27/2020 06:54, H.J. Lu via Libc-alpha wrote:
> > On Fri, Nov 27, 2020 at 4:24 AM Richard Earnshaw
> > <Richard.Earnshaw@foss.arm.com> wrote:
> > > On 27/11/2020 11:27, Siddhesh Poyarekar wrote:
> > > > From a distribution perspective, this could be done in two phases:
> > > >
> > > > 1. Phase 1 where you ave the tunable which is disabled by default and no
> > > > marker present.  This makes tagging available to those who want to opt-in
> > > >
> > > > 2. Phase 2 is where the tunable is now enabled by default and can be
> > > > overridden with an opt-out marker.  Running this opt-out binary on the
> > > > older version is safe (and shouldn't need an ABI bump) since tunables
> > > > are disabled by default.
> > > >
> > > > Does that make sense?  If not then I'd really like to see a more
> > > > detailed description of how you intend to roll this out.
> > > >
> > > > Siddhesh
> > >
> > > Yes, you've captured my thoughts and summarised it better that I was
> > > doing myself.  Thank you.
> > >
> >
> > We need a plan for binary marker first.
>
> if we enable heap tagging in those two phases then the first
> phase has no abi commitment (tunables are not abi stable).
>
> heap tagging will not be very useful in phase 1 but at least
> users can start testing it and report back issues. (assuming
> they have hardware or emulator.)
>
> i was hoping we can have an abi stable env var ui, but i'm
> fine with tunables only since that sounds the safest.
> (if we wait for the marking design and binutils etc support
> then we have less chance to see deployment problems early.)

We should try to get it right if it will be installed as the system glibc.
  

On 11/28/20 12:07 AM, H.J. Lu wrote:
>>> The memory tag implementation should be independent of tunables.
>>> Tunables should just turn on and off a few bits in the memory tag
>>> implementation.  Make the memory tag implementation depend on
>>> tunables seems wrong to me.

That comment seems to suggest that you'd like to see finer grained 
control over memory tagging in tunables.  Could you elaborate on that? 
If you're suggesting splitting up or adding more tunables, could you 
suggest the splits you'd like to see?

>> That shouldn't matter.  The tunables are documented as not being stable
>> and nothing else is exposed to the user; so if we want to change things
>> later, there's nothing to stop that.
>>
> 
> This is not a good user experience and makes it harder to backport.

I think that's only partly true.  Backporting changes to the *meaning* 
of tunables can be hard to backport, but simply flipping the default 
isn't quite in the same category.

Siddhesh