Clear upper bits during sign extension
Commit Message
Doug Evans <dje@google.com> writes:
> It's not immediately clear to this reader that undefined behaviour is
> avoided here.
> E.g., what if sizeof (LONGEST) == 8 && bit == 64.
How about returning VALUE simply in this case? like the patch below,
@@ -3028,10 +3028,13 @@ gdb_sign_extend (LONGEST value, int bit)
{
gdb_assert (bit >= 1 && bit <= 8 * sizeof (LONGEST));
- if (((value >> (bit - 1)) & 1) != 0)
+ if (bit < 8 * sizeof (LONGEST) && ((value >> (bit - 1)) & 1) != 0)
{
LONGEST signbit = ((LONGEST) 1) << (bit - 1);
+ /* Clear upper bits from bit BIT. */
+ value &= (signbit << 1) - 1;
+
value = (value ^ signbit) - signbit;
}