diff mbox series

libctf: ctf_member_next needs to return (ssize_t)-1 on error

Message ID 20230824113249.1197514-1-torbjorn.svensson@foss.st.com
State New
Headers
Series libctf: ctf_member_next needs to return (ssize_t)-1 on error |

Checks

Context Check Description
linaro-tcwg-bot/tcwg_binutils_build--master-arm success Testing passed
linaro-tcwg-bot/tcwg_binutils_build--master-aarch64 success Testing passed
linaro-tcwg-bot/tcwg_binutils_check--master-aarch64 success Testing passed
linaro-tcwg-bot/tcwg_binutils_check--master-arm success Testing passed

Commit Message

Torbjorn SVENSSON Aug. 24, 2023, 11:32 a.m. UTC 
  The function ctf_member_next should return (ssize_t)-1 on
error. As the function ctf_set_errno returns (ctf_id_t)-1L and that is
then casted to "unsigned long" as it's the return type of the function,
it's not compatible and causes the value 0xffffffff to be returned on
64-bit Windows builds. As a result, the check for a negative value in
ctf_dedup_rhash_type will never be true and a resulting infinit loop is
created.

This was found testing an arm-none-eabi toolchain built with
x86_64-w64-mingw32. If the same source tree is built with
i686-w64-mingw32, everything appears to be working correctly.

Signed-off-by: Torbjörn SVENSSON <torbjorn.svensson@foss.st.com>
Co-Authored-By: Yvan ROUX <yvan.roux@foss.st.com>
---
 libctf/ctf-types.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

Comments

Alan Modra Aug. 25, 2023, 2:22 a.m. UTC | #1 
On Thu, Aug 24, 2023 at 01:32:49PM +0200, Torbjörn SVENSSON via Binutils wrote:
> diff --git a/libctf/ctf-types.c b/libctf/ctf-types.c
> index c20ff825d9a..058b647ba9a 100644
> --- a/libctf/ctf-types.c
> +++ b/libctf/ctf-types.c
> @@ -233,7 +233,8 @@ ctf_member_next (ctf_dict_t *fp, ctf_id_t type, ctf_next_t **it,
>   end_iter:
>    ctf_next_destroy (i);
>    *it = NULL;
> -  return ctf_set_errno (ofp, ECTF_NEXT_END);
> +  ctf_set_errno (ofp, ECTF_NEXT_END);
> +  return -1;
>  }
>  
>  /* Iterate over the members of an ENUM.  We pass the string name and associated

This isn't the correct fix.  There are many uses of ctf_set_errno,
even in the function you are patching.  Some other fix is needed to
cope with unsigned long being smaller than ssize_t for the Microsoft
64-bit ABIs.
diff mbox series

Patch

diff --git a/libctf/ctf-types.c b/libctf/ctf-types.c
index c20ff825d9a..058b647ba9a 100644
--- a/libctf/ctf-types.c
+++ b/libctf/ctf-types.c
@@ -233,7 +233,8 @@  ctf_member_next (ctf_dict_t *fp, ctf_id_t type, ctf_next_t **it,
  end_iter:
   ctf_next_destroy (i);
   *it = NULL;
-  return ctf_set_errno (ofp, ECTF_NEXT_END);
+  ctf_set_errno (ofp, ECTF_NEXT_END);
+  return -1;
 }
 
 /* Iterate over the members of an ENUM.  We pass the string name and associated