Support gzip compressed exec and core files in gdb

Message ID 550A1F4D.5030107@eagerm.com
State New, archived
Headers

Commit Message

Michael Eager March 19, 2015, 12:58 a.m. UTC
  On 03/10/15 16:01, Michael Eager wrote:
> Add support to automatically unzip compressed executable and core files.
> Files will be uncompressed into temporary directory (/tmp or $TMPDIR)
> and are deleted when GDB exits.  This should be transparent to users,
> except for disk space requirements.  The name of the uncompressed file is
> mentioned, but all references to the file in GDB messages is to the file
> which the user specified.
>
> This operation cannot be done completely by BFD because BFD allows an opened
> file to be passed to it for processing.  GDB uses this functionality.
>
> BFD:
>    * bfd-in2.h: Regenerate.
>    * bfd.c (struct bfd): Add uncompressed_filename.
>    * bfdio.c (bfd_get_mtime): Set bfd->mtime_set to true.
>    * cache.c (bfd_open): Open previously created uncompressed file.
>
> GDB:
>    * common/filestuff.c (struct compressed_file_cache_search, eq_compressed_file,
>    is_gzip, decompress_gzip, do_compressed_cleanup, gdb_uncompress): New.
>    * common/filestuff.h (gdb_uncompress): Declare.
>    * corelow.c (core_open): Uncompress core file.
>    * exec.c (exec_file_attach): Uncompress exe file.
>    * symfile.c (symfile_bfd_open): Uncompress sym (exe) file.
>
> GDB/DOC:
>    * gdb.texinfo: Mention gzipped exec and core files.
>

Revised patch attached.  After Jan's patch to not close cached files,
I was able to eliminate the changes to BFD.  Moved uncompress code to
utils.c.  Added NEWS entry.

gdb/ChangeLog:
   * utils.c (struct compressed_file_cache_search, eq_compressed_file,
   is_gzip, decompress_gzip, do_compressed_cleanup, gdb_uncompress): New.
   * utils.h (gdb_uncompress): Declare.
   * corelow.c (core_open): Uncompress core file.
   * exec.c (exec_file_attach): Uncompress exec file.
   * symfile.c (symfile_bfd_open): Uncompress sym (exec) file.
   * NEWS: Mention new functionality.

gdb/doc:
   * gdb.texinfo (Files): Mention gzipped exec and core files.
  

Comments

Eli Zaretskii March 19, 2015, 3:45 a.m. UTC | #1
> Date: Wed, 18 Mar 2015 17:58:53 -0700
> From: Michael Eager <eager@eagerm.com>
> 
> Revised patch attached.  After Jan's patch to not close cached files,
> I was able to eliminate the changes to BFD.  Moved uncompress code to
> utils.c.  Added NEWS entry.

The NEWS entry is OK.
  
Mike Frysinger March 20, 2015, 10:16 p.m. UTC | #2
On 18 Mar 2015 17:58, Michael Eager wrote:
> --- a/gdb/utils.c
> +++ b/gdb/utils.c
>
> +#define COMPRESS_BUF_SIZE (1024*1024)

space around the *

would be nice to have a comment noting where the 1MiB comes from

> +static int
> +decompress_gzip (const char *filename, FILE *tmp)
> +{
> +#ifdef HAVE_ZLIB_H
> +  char *buf = xmalloc (COMPRESS_BUF_SIZE);
> +  gzFile compressed = gzopen (filename, "r");
> +  int count, res;
> +
> +  if (buf == NULL || compressed == NULL)
> +    {
> +      fprintf_filtered (gdb_stderr, _("error copying gzip file\n"));
> +      free (buf);
> +      return 0;
> +    }
> +
> +  while ((count = gzread (compressed, buf, COMPRESS_BUF_SIZE)))
> +    {
> +      res = fwrite (buf, 1, count, tmp);
> +      if (res != count)
> +	{
> +	  fprintf_filtered (gdb_stderr, _("error decompressing gzip file\n"));
> +          free (buf);

this line needs to use a tab to indent

> +/* If file is compressed, uncompress it into a temporary.  */
> +
> +int
> +gdb_uncompress (const char *filename, char **uncompressed_filename)
> +{
> +  FILE *handle;
> +  struct compressed_file_cache_search search, *found;
> +  struct stat st;
> +  hashval_t hash;
> +  void **slot;
> +  static unsigned char buffer[1024];
> +  size_t count;
> +  enum {NONE, GZIP, BZIP2} file_compression = NONE;

shouldn't this enum be declared outside this func ?

> +  if (found)
> +    {
> +      /* We previously uncompressed the file.  */
> +      if (found->mtime == st.st_mtime)
> +        {

this brace needs to indent w/a tab

> +	  /* Return file if compressed file not changed.  */
> +	  *uncompressed_filename = found->uncompressed_filename;
> +	  return 1;
> +	}
> +      else
> +        {

same here

> +	  /* Delete old uncompressed file.  */
> +	  unlink (found->uncompressed_filename);
> +	  xfree (found->filename);
> +	  xfree (found->uncompressed_filename);
> +        }

and here

> +  /* Create temporary file name for uncompressed file.  */
> +  if (!(tmpdir = getenv ("TMPDIR")))
> +    tmpdir = "/tmp";
> +
> +  if (!asprintf (&template, "%s/%s-XXXXXX", tmpdir, basename (filename)))
> +    return 0;
> +
> +  decomp_fd = mkstemp (template);

ignoring that assigningments in if statements are frowned upon, looks like you 
can just use make_temp_file(NULL) from libiberty

you would also leak the fopen() handle here.  probably want to go through this 
code again looking for such leaks.  and maybe try breaking the func up into more 
utility related chunks when possible.

> +  if (decomp_fd != -1)
> +    {
> +      decomp_file = fdopen (decomp_fd, "w+b");

FOPEN_WUB

> +      if (file_compression == GZIP)
> +        {

you've got more missing tabs in this file.  i'm going to stop checking.
-mike
  

Patch

From e0946586d235a45b5c314e7af35970ed79317e43 Mon Sep 17 00:00:00 2001
From: Michael Eager <eager@eagercon.com>
Date: Wed, 18 Mar 2015 11:08:38 -0700
Subject: [PATCH] GDB support compressed exec and core files.

Add support to automatically unzip compressed executable and core files.
Files will be uncompressed into temporary directory (/tmp or $TMPDIR)
and are deleted when GDB exits.  This should be transparent to users,
except for disk space requirements.  The name of the uncompressed file is
mentioned, but all references to the file in GDB messages is to the file
which the user specified.

gdb/ChangeLog:
  * utils.c (struct compressed_file_cache_search, eq_compressed_file,
  is_gzip, decompress_gzip, do_compressed_cleanup, gdb_uncompress): New.
  * utils.h (gdb_uncompress): Declare.
  * corelow.c (core_open): Uncompress core file.
  * exec.c (exec_file_attach): Uncompress exec file.
  * symfile.c (symfile_bfd_open): Uncompress sym (exec) file.
  * NEWS: Mention new functionality.

gdb/doc:
  * gdb.texinfo (Files): Mention gzipped exec and core files.
---
 gdb/NEWS            |   3 +
 gdb/corelow.c       |   9 +++
 gdb/doc/gdb.texinfo |   5 ++
 gdb/exec.c          |  14 +++-
 gdb/symfile.c       |  13 +++-
 gdb/utils.c         | 199 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 gdb/utils.h         |   4 ++
 7 files changed, 245 insertions(+), 2 deletions(-)

diff --git a/gdb/NEWS b/gdb/NEWS
index bda4a35..b132196 100644
--- a/gdb/NEWS
+++ b/gdb/NEWS
@@ -89,6 +89,9 @@  vFile:fstat:
 
 * Removed targets and native configurations
 
+* GDB will automatically uncompress executable and core files
+  which have been compressed using gzip.
+
 HP/PA running HP-UX           hppa*-*-hpux*
 Itanium running HP-UX         ia64-*-hpux*
 
diff --git a/gdb/corelow.c b/gdb/corelow.c
index 9218003..0314d1f 100644
--- a/gdb/corelow.c
+++ b/gdb/corelow.c
@@ -279,6 +279,7 @@  core_open (const char *arg, int from_tty)
   int scratch_chan;
   int flags;
   char *filename;
+  char *uncompressed_filename;
 
   target_preopen (from_tty);
   if (!arg)
@@ -316,6 +317,14 @@  core_open (const char *arg, int from_tty)
   if (temp_bfd == NULL)
     perror_with_name (filename);
 
+  if (!write_files && gdb_uncompress (filename, &uncompressed_filename))
+    {
+      close (scratch_chan);
+      scratch_chan = gdb_open_cloexec (uncompressed_filename, flags, 0);
+      temp_bfd = gdb_bfd_fopen (uncompressed_filename, gnutarget,
+				FOPEN_RB, scratch_chan);
+    }
+
   if (!bfd_check_format (temp_bfd, bfd_core)
       && !gdb_check_format (temp_bfd))
     {
diff --git a/gdb/doc/gdb.texinfo b/gdb/doc/gdb.texinfo
index 9e71642..4c9c3f3 100644
--- a/gdb/doc/gdb.texinfo
+++ b/gdb/doc/gdb.texinfo
@@ -17375,6 +17375,11 @@  via @code{gdbserver} (@pxref{Server, file, Using the @code{gdbserver}
 Program}).  In these situations the @value{GDBN} commands to specify
 new files are useful.
 
+Executable and core files may be compressed using @command{gzip}.  These
+files will be uncompressed into temporary files in the system-wide
+temporary directory (e.g., @file{/tmp} on some systems).  The files will
+have a unique name and will be deleted when @value{GDBN} terminates.
+
 @table @code
 @cindex executable file
 @kindex file
diff --git a/gdb/exec.c b/gdb/exec.c
index b1f6157..4e550e1 100644
--- a/gdb/exec.c
+++ b/gdb/exec.c
@@ -155,6 +155,7 @@  void
 exec_file_attach (const char *filename, int from_tty)
 {
   struct cleanup *cleanups;
+  char *uncompressed_filename = NULL;
 
   /* First, acquire a reference to the current exec_bfd.  We release
      this at the end of the function; but acquiring it now lets the
@@ -209,7 +210,18 @@  exec_file_attach (const char *filename, int from_tty)
 	exec_bfd = gdb_bfd_fopen (canonical_pathname, gnutarget,
 				  FOPEN_RUB, scratch_chan);
       else
-	exec_bfd = gdb_bfd_open (canonical_pathname, gnutarget, scratch_chan);
+	{
+	  if (!gdb_uncompress (canonical_pathname, &uncompressed_filename))
+	    exec_bfd = gdb_bfd_open (canonical_pathname, gnutarget, scratch_chan);
+          else
+	    {
+	      close (scratch_chan);
+	      scratch_chan = openp ("", 0, uncompressed_filename,
+				    O_RDONLY | O_BINARY, &scratch_pathname);
+
+	      exec_bfd = gdb_bfd_open (uncompressed_filename, gnutarget, scratch_chan);
+	    }
+	}
 
       if (!exec_bfd)
 	{
diff --git a/gdb/symfile.c b/gdb/symfile.c
index bd3a366..e24517f 100644
--- a/gdb/symfile.c
+++ b/gdb/symfile.c
@@ -56,6 +56,7 @@ 
 #include "stack.h"
 #include "gdb_bfd.h"
 #include "cli/cli-utils.h"
+#include "utils.h"
 
 #include <sys/types.h>
 #include <fcntl.h>
@@ -1744,6 +1745,7 @@  symfile_bfd_open (const char *cname)
   bfd *sym_bfd;
   int desc;
   char *name, *absolute_name;
+  char *uncompressed_filename;
   struct cleanup *back_to;
 
   if (remote_filename_p (cname))
@@ -1788,7 +1790,16 @@  symfile_bfd_open (const char *cname)
   name = absolute_name;
   back_to = make_cleanup (xfree, name);
 
-  sym_bfd = gdb_bfd_open (name, gnutarget, desc);
+  if (!gdb_uncompress (name, &uncompressed_filename))
+    sym_bfd = gdb_bfd_open (name, gnutarget, desc);
+  else
+    {
+      close (desc);
+      desc = openp ("", 0, uncompressed_filename, O_RDONLY | O_BINARY, &absolute_name);
+
+      sym_bfd = gdb_bfd_open (uncompressed_filename, gnutarget, desc);
+    }
+
   if (!sym_bfd)
     error (_("`%s': can't open to read symbols: %s."), name,
 	   bfd_errmsg (bfd_get_error ()));
diff --git a/gdb/utils.c b/gdb/utils.c
index 7172bba..35e7c02 100644
--- a/gdb/utils.c
+++ b/gdb/utils.c
@@ -29,6 +29,10 @@ 
 #include <sys/resource.h>
 #endif /* HAVE_SYS_RESOURCE_H */
 
+#ifdef HAVE_ZLIB_H
+#include <zlib.h>
+#endif
+
 #ifdef TUI
 #include "tui/tui.h"		/* For tui_get_command_dimension.   */
 #endif
@@ -3486,6 +3490,201 @@  gdb_filename_fnmatch (const char *pattern, const char *string, int flags)
 
   return fnmatch (pattern, string, flags);
 }
+
+#ifdef HAVE_ZLIB_H
+/* Hash table of compressed files.  */
+
+static htab_t compressed_file_cache;
+
+struct compressed_file_cache_search
+{
+  char *filename;
+  char *uncompressed_filename;
+  time_t mtime;
+};
+
+static int
+eq_compressed_file (const void *a, const void *b)
+{
+  const struct compressed_file_cache_search *entry = a;
+  const struct compressed_file_cache_search *search = b;
+
+  return (strcmp (entry->filename, search->filename) == 0);
+}
+#endif
+
+/* Test if file is compressed with gzip.  */
+
+static inline int
+is_gzip (unsigned char *buf)
+{
+  return (buf[0] == 037 && buf[1] == 0213);	/* From /usr/share/magic.  */
+}
+
+#define COMPRESS_BUF_SIZE (1024*1024)
+static int
+decompress_gzip (const char *filename, FILE *tmp)
+{
+#ifdef HAVE_ZLIB_H
+  char *buf = xmalloc (COMPRESS_BUF_SIZE);
+  gzFile compressed = gzopen (filename, "r");
+  int count, res;
+
+  if (buf == NULL || compressed == NULL)
+    {
+      fprintf_filtered (gdb_stderr, _("error copying gzip file\n"));
+      free (buf);
+      return 0;
+    }
+
+  while ((count = gzread (compressed, buf, COMPRESS_BUF_SIZE)))
+    {
+      res = fwrite (buf, 1, count, tmp);
+      if (res != count)
+	{
+	  fprintf_filtered (gdb_stderr, _("error decompressing gzip file\n"));
+          free (buf);
+	  return 0;
+	}
+    }
+
+  gzclose (compressed);
+  free (buf);
+  return 1;
+#else
+  return 0;
+#endif
+}
+
+/* Delete uncompressed temp file when terminating.  */
+static void
+do_compressed_cleanup (void *filename)
+{
+  unlink (filename);
+  xfree (filename);
+}
+
+/* If file is compressed, uncompress it into a temporary.  */
+
+int
+gdb_uncompress (const char *filename, char **uncompressed_filename)
+{
+  FILE *handle;
+  struct compressed_file_cache_search search, *found;
+  struct stat st;
+  hashval_t hash;
+  void **slot;
+  static unsigned char buffer[1024];
+  size_t count;
+  enum {NONE, GZIP, BZIP2} file_compression = NONE;
+  int decomp_fd;
+  FILE *decomp_file;
+  int ret = 0;
+  char *tmpdir, *p;
+  char *template;
+
+  if (compressed_file_cache == NULL)
+    compressed_file_cache = htab_create_alloc (1, htab_hash_string,
+					       eq_compressed_file,
+					       NULL, xcalloc, xfree);
+
+  if (stat (filename, &st) < 0)
+    return 0;
+
+  search.filename = (char *) filename;
+  search.uncompressed_filename = NULL;
+
+  hash = htab_hash_string (filename);
+  found = htab_find_with_hash (compressed_file_cache, &search, hash);
+
+  if (found)
+    {
+      /* We previously uncompressed the file.  */
+      if (found->mtime == st.st_mtime)
+        {
+	  /* Return file if compressed file not changed.  */
+	  *uncompressed_filename = found->uncompressed_filename;
+	  return 1;
+	}
+      else
+        {
+	  /* Delete old uncompressed file.  */
+	  unlink (found->uncompressed_filename);
+	  xfree (found->filename);
+	  xfree (found->uncompressed_filename);
+        }
+    }
+
+  handle = fopen (filename, FOPEN_RB);
+  if (handle == NULL)
+    return 0;
+
+  count = fread (buffer, 1, sizeof buffer, handle);
+  if (count > 0)
+    {
+      if (is_gzip (buffer))
+	file_compression = GZIP;
+    }
+
+  fclose (handle);
+
+  if (file_compression == NONE)
+    return 0;
+
+  /* Create temporary file name for uncompressed file.  */
+  if (!(tmpdir = getenv ("TMPDIR")))
+    tmpdir = "/tmp";
+
+  if (!asprintf (&template, "%s/%s-XXXXXX", tmpdir, basename (filename)))
+    return 0;
+
+  decomp_fd = mkstemp (template);
+  if (decomp_fd != -1)
+    {
+      decomp_file = fdopen (decomp_fd, "w+b");
+
+      if (file_compression == GZIP)
+        {
+	  printf (_("Decompressing %s to %s\n"), filename, template);
+	  ret = decompress_gzip (filename, decomp_file);
+	}
+      else
+        {
+          xfree (template);
+          return 0;
+        }
+      fclose (decomp_file);
+
+      if (ret)
+	{
+	  if (!found)
+	    {
+	      slot = htab_find_slot_with_hash (compressed_file_cache,
+					       &search, hash, INSERT);
+	      gdb_assert (slot && !*slot);
+	      found = xmalloc (sizeof (struct compressed_file_cache_search));
+	      *slot = found;
+	    }
+	  found->filename = strdup (filename);
+	  found->mtime = st.st_mtime;
+	  found->uncompressed_filename = template;
+	}
+    }
+  else
+    {
+      warning (_("Decompression failed\n"));
+      xfree (template);
+      return 0;
+    }
+
+  *uncompressed_filename = template;
+
+  /* Schedule delete of temp file when gdb ends.  */
+  make_final_cleanup (do_compressed_cleanup, xstrdup (template));
+
+  return 1;
+}
+
 
 /* Provide a prototype to silence -Wmissing-prototypes.  */
 extern initialize_file_ftype _initialize_utils;
diff --git a/gdb/utils.h b/gdb/utils.h
index b8e1aff..10d9c08 100644
--- a/gdb/utils.h
+++ b/gdb/utils.h
@@ -367,4 +367,8 @@  extern void dump_core (void);
 
 extern char *make_hex_string (const gdb_byte *data, size_t length);
 
+/* Uncompress file if compressed.  */
+
+int gdb_uncompress (const char *filename, char **uncompressed_filename);
+
 #endif /* UTILS_H */
-- 
1.8.1.4