[v2,06/10] Avoid undefined behavior in parse_number
Commit Message
-fsanitize=undefined pointed out that c-exp.y relied on undefined
behavior here:
if (c != 'l' && c != 'u')
n *= base;
...when a large hex constant "just fit" into a LONGEST, causing the
high bit to be set.
This fixes the problem by having the function work in an unsigned
type.
gdb/ChangeLog
2018-10-01 Tom Tromey <tom@tromey.com>
* c-exp.y (parse_number): Work in unsigned. Remove casts.
---
gdb/ChangeLog | 4 ++++
gdb/c-exp.y | 10 ++++------
2 files changed, 8 insertions(+), 6 deletions(-)
@@ -1760,10 +1760,8 @@ static int
parse_number (struct parser_state *par_state,
const char *buf, int len, int parsed_float, YYSTYPE *putithere)
{
- /* FIXME: Shouldn't these be unsigned? We don't deal with negative values
- here, and we do kind of silly things like cast to unsigned. */
- LONGEST n = 0;
- LONGEST prevn = 0;
+ ULONGEST n = 0;
+ ULONGEST prevn = 0;
ULONGEST un;
int i = 0;
@@ -1922,7 +1920,7 @@ parse_number (struct parser_state *par_state,
on 0x123456789 when LONGEST is 32 bits. */
if (c != 'l' && c != 'u' && n != 0)
{
- if ((unsigned_p && (ULONGEST) prevn >= (ULONGEST) n))
+ if (unsigned_p && prevn >= n)
error (_("Numeric constant too large."));
}
prevn = n;
@@ -1940,7 +1938,7 @@ parse_number (struct parser_state *par_state,
the case where it is we just always shift the value more than
once, with fewer bits each time. */
- un = (ULONGEST)n >> 2;
+ un = n >> 2;
if (long_p == 0
&& (un >> (gdbarch_int_bit (parse_gdbarch (par_state)) - 2)) == 0)
{