gmon: Fix allocated buffer overflow (bug 2944)

  The `__monstartup()` allocates a buffer used to store all the data
accumulated by the monitor.

The size of this buffer depends on the size of the internal structures
used and the address range for which the monitor is activated, as well
as on the maximum density of call instuctions and/or callable functions
that could be potentially on a segment of executable code.

In particular a hash table of arcs is placed at the end of this buffer.
The size of this hash table is calculated in bytes as
   p->fromssize = p->textsize / HASHFRACTION;

but actually should be
   p->fromssize = ROUNDUP(p->textsize / HASHFRACTION, sizeof(*p->froms));

Another minor error seems a related typo in the calculation of `kcountsize`.

This results in writing beyond the end of the allocated buffer when an
added arc corresponds to a call near from the end of the monitored
address range, since `_mcount()` check the incoming caller address for
monitored range but not the intermediate result hash-like index that
uses to write into the table.

It should be noted that when the results are output to `gmon.out`, the
table is read to the last element calculated from the allocated size in
bytes, so the arcs stored outside the buffer boundary did not fall into
`gprof` for analysis. Thus this "feature" help me to found this bug
during working with https://sourceware.org/bugzilla/show_bug.cgi?id=29438

Just in case, I will explicitly note that the problem breaks the
`make test t=gmon/tst-gmon-dso` added for Bug 29438.
There, the arc of the `f3()` call disappears from the output, since in
the DSO case, the call to `f3` is located close to the end of the
monitored range.

Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
---
 gmon/gmon.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
  

On 2/4/23 06:41, Леонид Юрьев (Leonid Yuriev) wrote:
> The `__monstartup()` allocates a buffer used to store all the data
> accumulated by the monitor.

I haven't reviewed this yet, but the bug number is "29444" and is incorrectly
written in the Subject line.

Thanks for posting this. We talked briefly about this patch during the Monday
patch review and we're trying to find a developer to review it.
 
> The size of this buffer depends on the size of the internal structures
> used and the address range for which the monitor is activated, as well
> as on the maximum density of call instuctions and/or callable functions
> that could be potentially on a segment of executable code.
> 
> In particular a hash table of arcs is placed at the end of this buffer.
> The size of this hash table is calculated in bytes as
>    p->fromssize = p->textsize / HASHFRACTION;
> 
> but actually should be
>    p->fromssize = ROUNDUP(p->textsize / HASHFRACTION, sizeof(*p->froms));
> 
> Another minor error seems a related typo in the calculation of `kcountsize`.
> 
> This results in writing beyond the end of the allocated buffer when an
> added arc corresponds to a call near from the end of the monitored
> address range, since `_mcount()` check the incoming caller address for
> monitored range but not the intermediate result hash-like index that
> uses to write into the table.
> 
> It should be noted that when the results are output to `gmon.out`, the
> table is read to the last element calculated from the allocated size in
> bytes, so the arcs stored outside the buffer boundary did not fall into
> `gprof` for analysis. Thus this "feature" help me to found this bug
> during working with https://sourceware.org/bugzilla/show_bug.cgi?id=29438
> 
> Just in case, I will explicitly note that the problem breaks the
> `make test t=gmon/tst-gmon-dso` added for Bug 29438.
> There, the arc of the `f3()` call disappears from the output, since in
> the DSO case, the call to `f3` is located close to the end of the
> monitored range.
> 
> Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>
> ---
>  gmon/gmon.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/gmon/gmon.c b/gmon/gmon.c
> index dee64803ad..4712d9f66b 100644
> --- a/gmon/gmon.c
> +++ b/gmon/gmon.c
> @@ -132,7 +132,7 @@ __monstartup (u_long lowpc, u_long highpc)
>    p->lowpc = ROUNDDOWN(lowpc, HISTFRACTION * sizeof(HISTCOUNTER));
>    p->highpc = ROUNDUP(highpc, HISTFRACTION * sizeof(HISTCOUNTER));
>    p->textsize = p->highpc - p->lowpc;
> -  p->kcountsize = ROUNDUP(p->textsize / HISTFRACTION, sizeof(*p->froms));
> +  p->kcountsize = ROUNDUP(p->textsize / HISTFRACTION, sizeof(*p->kcount));
>    p->hashfraction = HASHFRACTION;
>    p->log_hashfraction = -1;
>    /* The following test must be kept in sync with the corresponding
> @@ -142,7 +142,7 @@ __monstartup (u_long lowpc, u_long highpc)
>  	 instead of integer division.  Precompute shift amount. */
>        p->log_hashfraction = ffs(p->hashfraction * sizeof(*p->froms)) - 1;
>    }
> -  p->fromssize = p->textsize / HASHFRACTION;
> +  p->fromssize = ROUNDUP(p->textsize / HASHFRACTION, sizeof(*p->froms));
>    p->tolimit = p->textsize * ARCDENSITY / 100;
>    if (p->tolimit < MINARCS)
>      p->tolimit = MINARCS;
  

On 2023-02-04 06:41, Леонид Юрьев (Leonid Yuriev) wrote:
> The `__monstartup()` allocates a buffer used to store all the data
> accumulated by the monitor.
> 
> The size of this buffer depends on the size of the internal structures
> used and the address range for which the monitor is activated, as well
> as on the maximum density of call instuctions and/or callable functions
> that could be potentially on a segment of executable code.
> 
> In particular a hash table of arcs is placed at the end of this buffer.
> The size of this hash table is calculated in bytes as
>     p->fromssize = p->textsize / HASHFRACTION;
> 
> but actually should be
>     p->fromssize = ROUNDUP(p->textsize / HASHFRACTION, sizeof(*p->froms));
> 
> Another minor error seems a related typo in the calculation of `kcountsize`.
> 
> This results in writing beyond the end of the allocated buffer when an
> added arc corresponds to a call near from the end of the monitored
> address range, since `_mcount()` check the incoming caller address for
> monitored range but not the intermediate result hash-like index that
> uses to write into the table.
> 
> It should be noted that when the results are output to `gmon.out`, the
> table is read to the last element calculated from the allocated size in
> bytes, so the arcs stored outside the buffer boundary did not fall into
> `gprof` for analysis. Thus this "feature" help me to found this bug
> during working with https://sourceware.org/bugzilla/show_bug.cgi?id=29438
> 
> Just in case, I will explicitly note that the problem breaks the
> `make test t=gmon/tst-gmon-dso` added for Bug 29438.
> There, the arc of the `f3()` call disappears from the output, since in
> the DSO case, the call to `f3` is located close to the end of the
> monitored range.
> 
> Signed-off-by: Леонид Юрьев (Leonid Yuriev) <leo@yuriev.ru>

Adding a quick note here since this got raised as a security issue: this 
is a bug, but I don't see any security impact here since the inputs that 
cause this are trusted, coming from addresses of a profiled application. 
We'll be rejecting this CVE.

I'll leave the actual patch review and fix incorporation to DJ, who has 
volunteered to look at it.

Thanks,
Sid
  

›µ¾½¸´
®€Œµ² (Leonid Yuriev) <leo@yuriev.ru>
writes:
>    p->lowpc = ROUNDDOWN(lowpc, HISTFRACTION * sizeof(HISTCOUNTER));
>    p->highpc = ROUNDUP(highpc, HISTFRACTION * sizeof(HISTCOUNTER));
>    p->textsize = p->highpc - p->lowpc;
> -  p->kcountsize = ROUNDUP(p->textsize / HISTFRACTION, sizeof(*p->froms));
> +  p->kcountsize = ROUNDUP(p->textsize / HISTFRACTION, sizeof(*p->kcount));

I think the purpose here isn't to make sure the last entry fits, as that
it handled when lowpc and highpc are rounded.  I think the purpose here
is to make sure that the *next* portion of the buffer (p->froms) is
suitably aligned for its type.

> -  p->fromssize = p->textsize / HASHFRACTION;
> +  p->fromssize = ROUNDUP(p->textsize / HASHFRACTION, sizeof(*p->froms));

This part looks OK to me.