From patchwork Thu Feb 4 17:29:28 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Mahoney X-Patchwork-Id: 10739 Received: (qmail 127843 invoked by alias); 4 Feb 2016 17:43:27 -0000 Mailing-List: contact gdb-patches-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: gdb-patches-owner@sourceware.org Delivered-To: mailing list gdb-patches@sourceware.org Received: (qmail 127816 invoked by uid 89); 4 Feb 2016 17:43:25 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.2 spammy=is_unsigned, Hx-languages-length:1980 X-HELO: mx2.suse.de Received: from mx2.suse.de (HELO mx2.suse.de) (195.135.220.15) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (CAMELLIA256-SHA encrypted) ESMTPS; Thu, 04 Feb 2016 17:43:23 +0000 Received: from relay2.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 6209AAD34 for ; Thu, 4 Feb 2016 17:29:37 +0000 (UTC) Received: by starscream.home.jeffm.io (Postfix, from userid 1000) id 774FB89958; Thu, 4 Feb 2016 12:29:34 -0500 (EST) From: jeffm@suse.com To: gdb-patches@sourceware.org Cc: Jeff Mahoney Subject: [PATCH 2/7] py-value: properly handle unsigned and pointer types Date: Thu, 4 Feb 2016 12:29:28 -0500 Message-Id: <1454606973-31017-3-git-send-email-jeffm@suse.com> In-Reply-To: <1454606973-31017-1-git-send-email-jeffm@suse.com> References: <1454606973-31017-1-git-send-email-jeffm@suse.com> X-IsSubscribed: yes From: Jeff Mahoney GDB passes signed long values into python whether they're signed or not. This results in a situation where the caller needs to convert it back to an unsigned value, which requires knowledge of the underlying size of the value. The information to do that is available in the API, but it's unnecessary. We know it's an unsigned value so pass it as an unsigned value. --- gdb/python/py-value.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/gdb/python/py-value.c b/gdb/python/py-value.c index 140aaf5..c557d95 100644 --- a/gdb/python/py-value.c +++ b/gdb/python/py-value.c @@ -1475,6 +1475,7 @@ valpy_int (PyObject *self) { struct value *value = ((value_object *) self)->value; struct type *type = value_type (value); + int is_unsigned = 0; LONGEST l = 0; TRY @@ -1482,6 +1483,9 @@ valpy_int (PyObject *self) if (!is_integral_type (type)) error (_("Cannot convert value to int.")); + if (TYPE_CODE (type) == TYPE_CODE_PTR || + TYPE_UNSIGNED(type)) + is_unsigned = 1; l = value_as_long (value); } CATCH (except, RETURN_MASK_ALL) @@ -1490,6 +1494,8 @@ valpy_int (PyObject *self) } END_CATCH + if (is_unsigned) + return gdb_py_object_from_ulongest ((ULONGEST)l); return gdb_py_object_from_longest (l); } #endif @@ -1500,6 +1506,7 @@ valpy_long (PyObject *self) { struct value *value = ((value_object *) self)->value; struct type *type = value_type (value); + int is_unsigned = 0; LONGEST l = 0; TRY @@ -1510,6 +1517,9 @@ valpy_long (PyObject *self) && TYPE_CODE (type) != TYPE_CODE_PTR) error (_("Cannot convert value to long.")); + if (TYPE_CODE (type) == TYPE_CODE_PTR || + TYPE_UNSIGNED(type)) + is_unsigned = 1; l = value_as_long (value); } CATCH (except, RETURN_MASK_ALL) @@ -1518,6 +1528,8 @@ valpy_long (PyObject *self) } END_CATCH + if (is_unsigned) + return gdb_py_long_from_ulongest ((ULONGEST)l); return gdb_py_long_from_longest (l); }