File name too long causing failure to open temporary head file in dlltool

  From: Jiaying Song <jiaying.song.cn@windriver.com>

During the execution of the command: i686-w64-mingw32-dlltool
--input-def $def_filepath --output-delaylib $filepath --dllname qemu.exe
An error occurred:
i686-w64-mingw32-dlltool: failed to open temporary head file: ..._w64_mingw32_nativesdk_qemu_8_2_2_build_plugins_libqemu_plugin_api_a_h.s

Due to the path length exceeding the Linux system's file name length
limit (NAME_MAX=255), the temporary file name generated by the
i686-w64-mingw32-dlltool command becomes too long to open. To address
this, a new temporary file name prefix is generated using tmp_prefix =
prefix_encode ("d", getpid()), ensuring that the file name does not
exceed the system's length limit.

Signed-off-by: Jiaying Song <jiaying.song.cn@windriver.com>
---
 binutils/dlltool.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)
  

On 13.08.2024 04:31, jiaying.song.cn@windriver.com wrote:
> --- a/binutils/dlltool.c
> +++ b/binutils/dlltool.c
> @@ -4099,11 +4099,18 @@ main (int ac, char **av)
>        if (imp_name || delayimp_name)
>          {
>            const char *input = imp_name ? imp_name : delayimp_name;
> -          tmp_prefix = xmalloc (strlen (input) + 2);
> -          sprintf (tmp_prefix, "%s_", input);
> -          for (i = 0; tmp_prefix[i]; i++)
> -            if (!ISALNUM (tmp_prefix[i]))
> -              tmp_prefix[i] = '_';
> +          if ((strlen(input) + 2) > NAME_MAX)
> +            {
> +              tmp_prefix = prefix_encode ("d", getpid ());

I think it would be nice if this function invocation would be folded
with the other one, just out of patch context (as long as that's easy
to arrange for, which it looks like it is).

> +            }
> +          else
> +            {
> +              tmp_prefix = xmalloc (strlen (input) + 2);
> +              sprintf (tmp_prefix, "%s_", input);
> +              for (i = 0; tmp_prefix[i]; i++)
> +              if (!ISALNUM (tmp_prefix[i]))
> +                tmp_prefix[i] = '_';

Please don't screw up indentation. (I notice the whole code block
suffers from the lack of use of hard tabs. As - for the comment
above - you'll re-arrange it anyway, please can you take care of
that as well at the same time?)

Jan

> +            }
>          }
>        else
>          {
  

On Tue, Aug 13, 2024 at 10:31:21AM +0800, jiaying.song.cn@windriver.com wrote:
> From: Jiaying Song <jiaying.song.cn@windriver.com>
> 
> During the execution of the command: i686-w64-mingw32-dlltool
> --input-def $def_filepath --output-delaylib $filepath --dllname qemu.exe
> An error occurred:
> i686-w64-mingw32-dlltool: failed to open temporary head file: ..._w64_mingw32_nativesdk_qemu_8_2_2_build_plugins_libqemu_plugin_api_a_h.s
> 
> Due to the path length exceeding the Linux system's file name length
> limit (NAME_MAX=255), the temporary file name generated by the
> i686-w64-mingw32-dlltool command becomes too long to open. To address
> this, a new temporary file name prefix is generated using tmp_prefix =
> prefix_encode ("d", getpid()), ensuring that the file name does not
> exceed the system's length limit.
> 
> Signed-off-by: Jiaying Song <jiaying.song.cn@windriver.com>
> ---
>  binutils/dlltool.c | 17 ++++++++++++-----
>  1 file changed, 12 insertions(+), 5 deletions(-)
> 
> diff --git a/binutils/dlltool.c b/binutils/dlltool.c
> index 066c99a4..acaeb5a7 100644
> --- a/binutils/dlltool.c
> +++ b/binutils/dlltool.c
> @@ -4099,11 +4099,18 @@ main (int ac, char **av)
>        if (imp_name || delayimp_name)
>          {
>            const char *input = imp_name ? imp_name : delayimp_name;
> -          tmp_prefix = xmalloc (strlen (input) + 2);
> -          sprintf (tmp_prefix, "%s_", input);
> -          for (i = 0; tmp_prefix[i]; i++)
> -            if (!ISALNUM (tmp_prefix[i]))
> -              tmp_prefix[i] = '_';
> +          if ((strlen(input) + 2) > NAME_MAX)
> +            {
> +              tmp_prefix = prefix_encode ("d", getpid ());
> +            }
> +          else
> +            {
> +              tmp_prefix = xmalloc (strlen (input) + 2);
> +              sprintf (tmp_prefix, "%s_", input);
> +              for (i = 0; tmp_prefix[i]; i++)
> +              if (!ISALNUM (tmp_prefix[i]))
> +                tmp_prefix[i] = '_';
> +            }
>          }
>        else
>          {
> -- 
> 2.25.1

Thanks, I'm committing a slightly changed version.

diff --git a/binutils/dlltool.c b/binutils/dlltool.c
index d75a99d8b3d..6dc16a9ed84 100644
--- a/binutils/dlltool.c
+++ b/binutils/dlltool.c
@@ -4068,9 +4068,9 @@ main (int ac, char **av)
   if (tmp_prefix == NULL)
     {
       /* If possible use a deterministic prefix.  */
-      if (imp_name || delayimp_name)
+      const char *input = imp_name ? imp_name : delayimp_name;
+      if (input && strlen (input) + 2 <= NAME_MAX)
 	{
-	  const char *input = imp_name ? imp_name : delayimp_name;
 	  tmp_prefix = xmalloc (strlen (input) + 2);
 	  sprintf (tmp_prefix, "%s_", input);
 	  for (i = 0; tmp_prefix[i]; i++)
@@ -4078,9 +4078,7 @@ main (int ac, char **av)
 	      tmp_prefix[i] = '_';
 	}
       else
-	{
-	  tmp_prefix = prefix_encode ("d", getpid ());
-	}
+	tmp_prefix = prefix_encode ("d", getpid ());
     }
 
   mangle_defs ();