[v2,04/10] Avoid undefined behavior in extract_integer

Message ID 20181002044420.17628-5-tom@tromey.com
State New, archived
Headers

Commit Message

Tom Tromey Oct. 2, 2018, 4:44 a.m. UTC
  -fsanitize=undefined showed that extract_integer could left-shift a
negative value, which is undefined.  This patch fixes the problem by
doing all the work in an unsigned type.  This relies on
implementation-defined behavior, but I tend to think we are on safe
ground there.  (Also, if need be, violations of this could probably be
detected, either by configure or by a static_assert.)

gdb/ChangeLog
2018-10-01  Tom Tromey  <tom@tromey.com>

	* findvar.c (extract_integer): Do work in an unsigned type.
---
 gdb/ChangeLog | 4 ++++
 gdb/findvar.c | 2 +-
 2 files changed, 5 insertions(+), 1 deletion(-)
  

Patch

diff --git a/gdb/findvar.c b/gdb/findvar.c
index 9256833ab6..be6c9d6f60 100644
--- a/gdb/findvar.c
+++ b/gdb/findvar.c
@@ -50,7 +50,7 @@  template<typename T, typename>
 T
 extract_integer (const gdb_byte *addr, int len, enum bfd_endian byte_order)
 {
-  T retval = 0;
+  typename std::make_unsigned<T>::type retval = 0;
   const unsigned char *p;
   const unsigned char *startaddr = addr;
   const unsigned char *endaddr = startaddr + len;