[pushed] Fix floating conversion buffer overrun when host/target format matches
Commit Message
Running the testsuite with a gdb configured with --enable-libmcheck
reveals a problem:
(gdb) ptype 3 * 2.0
type = <12-byte float>
memory clobbered past end of allocated block
ERROR: Process no longer exists
UNRESOLVED: gdb.ada/ptype_arith_binop.exp: ptype 3 * 2.0
(gdb) PASS: gdb.dlang/expression.exp: ptype 0x1.FFFFFFFFFFFFFp1023
ptype 0x1p-52L
type = real
memory clobbered past end of allocated block
ERROR: Process no longer exists
UNRESOLVED: gdb.dlang/expression.exp: ptype 0x1p-52L
Even though this shows up with Ada and D, it's easy to reproduce in C
too. We just need to print a long double, when the current arch is
32-bit, which is the default when gdb starts up:
$ ./gdb -q -ex "ptype 1.0L"
type = long double
memory clobbered past end of allocated block
Aborted (core dumped)
Valgrind shows:
==22159== Invalid write of size 8
==22159== at 0x8464A9: floatformat_from_doublest (doublest.c:756)
==22159== by 0x846822: store_typed_floating (doublest.c:867)
==22159== by 0x6A7959: value_from_double (value.c:3662)
==22159== by 0x6A9F2D: evaluate_subexp_standard (eval.c:745)
==22159== by 0x7F31AF: evaluate_subexp_c (c-lang.c:716)
==22159== by 0x6A8986: evaluate_subexp (eval.c:79)
==22159== by 0x6A8BA3: evaluate_type (eval.c:174)
==22159== by 0x817CCF: whatis_exp (typeprint.c:456)
==22159== by 0x817EAA: ptype_command (typeprint.c:508)
==22159== by 0x5F267B: do_cfunc (cli-decode.c:105)
==22159== by 0x5F5618: cmd_func (cli-decode.c:1885)
==22159== by 0x83622A: execute_command (top.c:475)
==22159== Address 0x8c6cb28 is 8 bytes inside a block of size 12 alloc'd
==22159== at 0x4C2AA98: calloc (vg_replace_malloc.c:711)
==22159== by 0x87384A: xcalloc (common-utils.c:83)
==22159== by 0x873889: xzalloc (common-utils.c:93)
==22159== by 0x6A34CB: allocate_value_contents (value.c:1036)
==22159== by 0x6A3501: allocate_value (value.c:1047)
==22159== by 0x6A790A: value_from_double (value.c:3656)
==22159== by 0x6A9F2D: evaluate_subexp_standard (eval.c:745)
==22159== by 0x7F31AF: evaluate_subexp_c (c-lang.c:716)
==22159== by 0x6A8986: evaluate_subexp (eval.c:79)
==22159== by 0x6A8BA3: evaluate_type (eval.c:174)
==22159== by 0x817CCF: whatis_exp (typeprint.c:456)
==22159== by 0x817EAA: ptype_command (typeprint.c:508)
==22159==
type = long double
(gdb)
Even if the target and host floating-point formats match, the length
of the types might still be different. On x86, long double is the
80-bit extended precision type on both 32-bit and 64-bit ABIs, but by
default it is stored as 12 bytes on 32-bit, and 16 bytes on 64-bit,
for alignment reasons. Several places in doublest.c already consider
this, but floatformat_to_doublest and floatformat_from_doublest miss
it. E.g., convert_typed_floating and store_typed_floating,
Tested on x86-64 Fedora 23 with --enable-libmcheck, where it fixes the
crashed above.
gdb/ChangeLog:
2016-03-09 Pedro Alves <palves@redhat.com>
* doublest.c: Extend comments.
(floatformat_to_doublest, floatformat_from_doublest): Copy the
floatformat's total size, not the host type's size.
---
gdb/ChangeLog | 6 ++++++
gdb/doublest.c | 30 ++++++++++++++++++++----------
2 files changed, 26 insertions(+), 10 deletions(-)
@@ -1,5 +1,11 @@
2016-03-09 Pedro Alves <palves@redhat.com>
+ * doublest.c: Extend comments.
+ (floatformat_to_doublest, floatformat_from_doublest): Copy the
+ floatformat's total size, not the host type's size.
+
+2016-03-09 Pedro Alves <palves@redhat.com>
+
* doublest.c (floatformat_totalsize_bytes): New function.
(floatformat_from_type): Assert that the type's length is at least
as long as the floatformat's totalsize.
@@ -686,7 +686,15 @@ floatformat_mantissa (const struct floatformat *fmt,
If the host and target formats agree, we just copy the raw data
into the appropriate type of variable and return, letting the host
increase precision as necessary. Otherwise, we call the conversion
- routine and let it do the dirty work. */
+ routine and let it do the dirty work. Note that even if the target
+ and host floating-point formats match, the length of the types
+ might still be different, so the conversion routines must make sure
+ to not overrun any buffers. For example, on x86, long double is
+ the 80-bit extended precision type on both 32-bit and 64-bit ABIs,
+ but by default it is stored as 12 bytes on 32-bit, and 16 bytes on
+ 64-bit, for alignment reasons. See comment in store_typed_floating
+ for a discussion about zeroing out remaining bytes in the target
+ buffer. */
static const struct floatformat *host_float_format = GDB_HOST_FLOAT_FORMAT;
static const struct floatformat *host_double_format = GDB_HOST_DOUBLE_FORMAT;
@@ -707,25 +715,26 @@ floatformat_to_doublest (const struct floatformat *fmt,
const void *in, DOUBLEST *out)
{
gdb_assert (fmt != NULL);
+
if (fmt == host_float_format)
{
- float val;
+ float val = 0;
- memcpy (&val, in, sizeof (val));
+ memcpy (&val, in, floatformat_totalsize_bytes (fmt));
*out = val;
}
else if (fmt == host_double_format)
{
- double val;
+ double val = 0;
- memcpy (&val, in, sizeof (val));
+ memcpy (&val, in, floatformat_totalsize_bytes (fmt));
*out = val;
}
else if (fmt == host_long_double_format)
{
- long double val;
+ long double val = 0;
- memcpy (&val, in, sizeof (val));
+ memcpy (&val, in, floatformat_totalsize_bytes (fmt));
*out = val;
}
else
@@ -737,23 +746,24 @@ floatformat_from_doublest (const struct floatformat *fmt,
const DOUBLEST *in, void *out)
{
gdb_assert (fmt != NULL);
+
if (fmt == host_float_format)
{
float val = *in;
- memcpy (out, &val, sizeof (val));
+ memcpy (out, &val, floatformat_totalsize_bytes (fmt));
}
else if (fmt == host_double_format)
{
double val = *in;
- memcpy (out, &val, sizeof (val));
+ memcpy (out, &val, floatformat_totalsize_bytes (fmt));
}
else if (fmt == host_long_double_format)
{
long double val = *in;
- memcpy (out, &val, sizeof (val));
+ memcpy (out, &val, floatformat_totalsize_bytes (fmt));
}
else
convert_doublest_to_floatformat (fmt, in, out);