[15/16] Highlight source code using GNU Source Highlight

  This changes gdb to highlight source using GNU Source Highlight, if it
is available.

This affects the output of the "list" command and also the TUI source
window.

No new test because I didn't see a way to make it work when Source
Highlight is not found.

gdb/ChangeLog
2018-11-27  Tom Tromey  <tom@tromey.com>

	* utils.h (can_emit_style_escape): Declare.
	* utils.c (can_emit_style_escape): No longer static.
	* cli/cli-style.c (set_style_enabled): New function.
	(_initialize_cli_style): Use it.
	* tui/tui-winsource.c (tui_show_source_line): Use tui_puts.
	(tui_alloc_source_buffer): Change how source lines are allocated.
	* tui/tui-source.c (copy_source_line): New function.
	(tui_set_source_content): Use source cache.
	* tui/tui-io.h (tui_puts): Update.
	* tui/tui-io.c (tui_puts_internal): Add window parameter.
	(tui_puts): Likewise.
	(tui_redisplay_readline): Update.
	* tui/tui-data.c (free_content_elements): Change how source window
	contents are freed.
	* source.c (forget_cached_source_info): Clear the source cache.
	(print_source_lines_base): Use the source cache.
	* source-cache.h: New file.
	* source-cache.c: New file.
	* configure.ac: Check for GNU Source Highlight library.
	* configure: Update.
	* config.in: Update.
	* Makefile.in (SRCHIGH_LIBS, SRCHIGH_CFLAGS): New variables.
	(INTERNAL_CFLAGS_BASE): Add SRCHIGH_CFLAGS.
	(CLIBS): Add SRCHIGH_LIBS.
	(COMMON_SFILES): Add source-cache.c.
	(HFILES_NO_SRCDIR): Add source-cache.h.
---
 gdb/ChangeLog           |  29 ++++
 gdb/Makefile.in         |  12 +-
 gdb/cli/cli-style.c     |   9 +-
 gdb/config.in           |   3 +
 gdb/configure           |  30 ++++
 gdb/configure.ac        |  23 +++
 gdb/source-cache.c      | 209 +++++++++++++++++++++++++++
 gdb/source-cache.h      |  79 +++++++++++
 gdb/source.c            |  38 ++---
 gdb/tui/tui-data.c      |   5 +-
 gdb/tui/tui-io.c        |  11 +-
 gdb/tui/tui-io.h        |   4 +-
 gdb/tui/tui-source.c    | 302 +++++++++++++++++++---------------------
 gdb/tui/tui-winsource.c |  17 +--
 gdb/utils.c             |   4 +-
 gdb/utils.h             |   4 +
 16 files changed, 576 insertions(+), 203 deletions(-)
 create mode 100644 gdb/source-cache.c
 create mode 100644 gdb/source-cache.h
  

On Tue, Nov 27, 2018 at 4:16 PM Tom Tromey <tom@tromey.com> wrote:
>
> This changes gdb to highlight source using GNU Source Highlight, if it
> is available.
>
> This affects the output of the "list" command and also the TUI source
> window.
>
> No new test because I didn't see a way to make it work when Source
> Highlight is not found.

I know this patch has been in for almost a year now.  But I have a
question, is there a way to build GNU Source Highlight as a static
library and have it built at the top level?  Like what is done for
GMP/MPFR for GCC?  It would reduce how I built a gdb that is for a
non-native target.
If it cannot be compiled at a top-level, does it support as a static
library?  I have not looked fully though.  The reason is I want to
support one gdb that can be run on multiple (some older) distros.

Thanks,
Andrew Pinski


>
> gdb/ChangeLog
> 2018-11-27  Tom Tromey  <tom@tromey.com>
>
>         * utils.h (can_emit_style_escape): Declare.
>         * utils.c (can_emit_style_escape): No longer static.
>         * cli/cli-style.c (set_style_enabled): New function.
>         (_initialize_cli_style): Use it.
>         * tui/tui-winsource.c (tui_show_source_line): Use tui_puts.
>         (tui_alloc_source_buffer): Change how source lines are allocated.
>         * tui/tui-source.c (copy_source_line): New function.
>         (tui_set_source_content): Use source cache.
>         * tui/tui-io.h (tui_puts): Update.
>         * tui/tui-io.c (tui_puts_internal): Add window parameter.
>         (tui_puts): Likewise.
>         (tui_redisplay_readline): Update.
>         * tui/tui-data.c (free_content_elements): Change how source window
>         contents are freed.
>         * source.c (forget_cached_source_info): Clear the source cache.
>         (print_source_lines_base): Use the source cache.
>         * source-cache.h: New file.
>         * source-cache.c: New file.
>         * configure.ac: Check for GNU Source Highlight library.
>         * configure: Update.
>         * config.in: Update.
>         * Makefile.in (SRCHIGH_LIBS, SRCHIGH_CFLAGS): New variables.
>         (INTERNAL_CFLAGS_BASE): Add SRCHIGH_CFLAGS.
>         (CLIBS): Add SRCHIGH_LIBS.
>         (COMMON_SFILES): Add source-cache.c.
>         (HFILES_NO_SRCDIR): Add source-cache.h.
> ---
>  gdb/ChangeLog           |  29 ++++
>  gdb/Makefile.in         |  12 +-
>  gdb/cli/cli-style.c     |   9 +-
>  gdb/config.in           |   3 +
>  gdb/configure           |  30 ++++
>  gdb/configure.ac        |  23 +++
>  gdb/source-cache.c      | 209 +++++++++++++++++++++++++++
>  gdb/source-cache.h      |  79 +++++++++++
>  gdb/source.c            |  38 ++---
>  gdb/tui/tui-data.c      |   5 +-
>  gdb/tui/tui-io.c        |  11 +-
>  gdb/tui/tui-io.h        |   4 +-
>  gdb/tui/tui-source.c    | 302 +++++++++++++++++++---------------------
>  gdb/tui/tui-winsource.c |  17 +--
>  gdb/utils.c             |   4 +-
>  gdb/utils.h             |   4 +
>  16 files changed, 576 insertions(+), 203 deletions(-)
>  create mode 100644 gdb/source-cache.c
>  create mode 100644 gdb/source-cache.h
>
> diff --git a/gdb/Makefile.in b/gdb/Makefile.in
> index 54baf96ea3..50d429d792 100644
> --- a/gdb/Makefile.in
> +++ b/gdb/Makefile.in
> @@ -194,6 +194,10 @@ LIBIPT = @LIBIPT@
>  # Where is libmpfr?  This will be empty if libmpfr was not available.
>  LIBMPFR = @LIBMPFR@
>
> +# GNU source highlight library.
> +SRCHIGH_LIBS = @SRCHIGH_LIBS@
> +SRCHIGH_CFLAGS = @SRCHIGH_CFLAGS@
> +
>  WARN_CFLAGS = @WARN_CFLAGS@
>  WERROR_CFLAGS = @WERROR_CFLAGS@
>  GDB_WARN_CFLAGS = $(WARN_CFLAGS)
> @@ -565,7 +569,8 @@ INTERNAL_CFLAGS_BASE = \
>         $(CXXFLAGS) $(GLOBAL_CFLAGS) $(PROFILE_CFLAGS) \
>         $(GDB_CFLAGS) $(OPCODES_CFLAGS) $(READLINE_CFLAGS) $(ZLIBINC) \
>         $(BFD_CFLAGS) $(INCLUDE_CFLAGS) $(LIBDECNUMBER_CFLAGS) \
> -       $(INTL_CFLAGS) $(INCGNU) $(ENABLE_CFLAGS) $(INTERNAL_CPPFLAGS)
> +       $(INTL_CFLAGS) $(INCGNU) $(ENABLE_CFLAGS) $(INTERNAL_CPPFLAGS) \
> +       $(SRCHIGH_CFLAGS)
>  INTERNAL_WARN_CFLAGS = $(INTERNAL_CFLAGS_BASE) $(GDB_WARN_CFLAGS)
>  INTERNAL_CFLAGS = $(INTERNAL_WARN_CFLAGS) $(GDB_WERROR_CFLAGS)
>
> @@ -591,7 +596,8 @@ CLIBS = $(SIM) $(READLINE) $(OPCODES) $(BFD) $(ZLIB) $(INTL) $(LIBIBERTY) $(LIBD
>         $(XM_CLIBS) $(GDBTKLIBS) \
>         @LIBS@ @GUILE_LIBS@ @PYTHON_LIBS@ \
>         $(LIBEXPAT) $(LIBLZMA) $(LIBBABELTRACE) $(LIBIPT) \
> -       $(LIBIBERTY) $(WIN32LIBS) $(LIBGNU) $(LIBICONV) $(LIBMPFR)
> +       $(LIBIBERTY) $(WIN32LIBS) $(LIBGNU) $(LIBICONV) $(LIBMPFR) \
> +       $(SRCHIGH_LIBS)
>  CDEPS = $(NAT_CDEPS) $(SIM) $(BFD) $(READLINE_DEPS) \
>         $(OPCODES) $(INTL_DEPS) $(LIBIBERTY) $(CONFIG_DEPS) $(LIBGNU)
>
> @@ -1101,6 +1107,7 @@ COMMON_SFILES = \
>         solib.c \
>         solib-target.c \
>         source.c \
> +       source-cache.c \
>         stabsread.c \
>         stack.c \
>         std-regs.c \
> @@ -1376,6 +1383,7 @@ HFILES_NO_SRCDIR = \
>         solib-target.h \
>         solist.h \
>         source.h \
> +       source-cache.h \
>         sparc-nat.h \
>         sparc-ravenscar-thread.h \
>         sparc-tdep.h \
> diff --git a/gdb/cli/cli-style.c b/gdb/cli/cli-style.c
> index 74e3958eb8..16415e0f88 100644
> --- a/gdb/cli/cli-style.c
> +++ b/gdb/cli/cli-style.c
> @@ -20,6 +20,7 @@
>  #include "defs.h"
>  #include "cli/cli-cmds.h"
>  #include "cli/cli-style.h"
> +#include "source-cache.h"
>
>  /* True if styling is enabled.  */
>
> @@ -216,6 +217,12 @@ show_style (const char *arg, int from_tty)
>  {
>  }
>
> +static void
> +set_style_enabled  (const char *args, int from_tty, struct cmd_list_element *c)
> +{
> +  g_source_cache.clear ();
> +}
> +
>  static void
>  show_style_enabled (struct ui_file *file, int from_tty,
>                     struct cmd_list_element *c, const char *value)
> @@ -245,7 +252,7 @@ Configure various style-related variables, such as colors"),
>  Set whether CLI styling is enabled."), _("\
>  Show whether CLI is enabled."), _("\
>  If enabled, output to the terminal is styled."),
> -                          NULL, show_style_enabled,
> +                          set_style_enabled, show_style_enabled,
>                            &style_set_list, &style_show_list);
>
>    file_name_style.add_setshow_commands ("filename", no_class,
> diff --git a/gdb/config.in b/gdb/config.in
> index 760db6b320..ea907d2b56 100644
> --- a/gdb/config.in
> +++ b/gdb/config.in
> @@ -426,6 +426,9 @@
>  /* Define to 1 if the system has the type `socklen_t'. */
>  #undef HAVE_SOCKLEN_T
>
> +/* Define to 1 if the source-highlight library is available */
> +#undef HAVE_SOURCE_HIGHLIGHT
> +
>  /* Define to 1 if you have the <stdint.h> header file. */
>  #undef HAVE_STDINT_H
>
> diff --git a/gdb/configure b/gdb/configure
> index 44df6ebcfb..811664ed40 100755
> --- a/gdb/configure
> +++ b/gdb/configure
> @@ -701,6 +701,8 @@ ALLOCA
>  LTLIBIPT
>  LIBIPT
>  HAVE_LIBIPT
> +SRCHIGH_CFLAGS
> +SRCHIGH_LIBS
>  HAVE_GUILE_FALSE
>  HAVE_GUILE_TRUE
>  GUILE_LIBS
> @@ -11469,6 +11471,34 @@ else
>  fi
>
>
> +# ---------------------------- #
> +# Check for source highlight.  #
> +# ---------------------------- #
> +
> +SRCHIGH_LIBS=
> +SRCHIGH_CFLAGS=
> +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for source highlight" >&5
> +$as_echo_n "checking for source highlight... " >&6; }
> +if test "${pkg_config_prog_path}" = "missing"; then
> +   { $as_echo "$as_me:${as_lineno-$LINENO}: result: no - pkg-config not found" >&5
> +$as_echo "no - pkg-config not found" >&6; }
> +else
> +   if ${pkg_config_prog_path} --exists source-highlight; then
> +      SRCHIGH_CFLAGS=`${pkg_config_prog_path} --cflags source-highlight`
> +      SRCHIGH_LIBS=`${pkg_config_prog_path} --libs source-highlight`
> +
> +$as_echo "#define HAVE_SOURCE_HIGHLIGHT 1" >>confdefs.h
> +
> +      { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
> +$as_echo "yes" >&6; }
> +   else
> +      { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
> +$as_echo "no" >&6; }
> +   fi
> +fi
> +
> +
> +
>  # --------------------- #
>  # Check for libmcheck.  #
>  # --------------------- #
> diff --git a/gdb/configure.ac b/gdb/configure.ac
> index 56cd0927bb..7d2d39b710 100644
> --- a/gdb/configure.ac
> +++ b/gdb/configure.ac
> @@ -1226,6 +1226,29 @@ AC_SUBST(GUILE_CPPFLAGS)
>  AC_SUBST(GUILE_LIBS)
>  AM_CONDITIONAL(HAVE_GUILE, test "${have_libguile}" != no)
>
> +# ---------------------------- #
> +# Check for source highlight.  #
> +# ---------------------------- #
> +
> +SRCHIGH_LIBS=
> +SRCHIGH_CFLAGS=
> +AC_MSG_CHECKING([for the source-highlight library])
> +if test "${pkg_config_prog_path}" = "missing"; then
> +   AC_MSG_RESULT([no - pkg-config not found])
> +else
> +   if ${pkg_config_prog_path} --exists source-highlight; then
> +      SRCHIGH_CFLAGS=`${pkg_config_prog_path} --cflags source-highlight`
> +      SRCHIGH_LIBS=`${pkg_config_prog_path} --libs source-highlight`
> +      AC_DEFINE([HAVE_SOURCE_HIGHLIGHT], 1,
> +                [Define to 1 if the source-highlight library is available])
> +      AC_MSG_RESULT([yes])
> +   else
> +      AC_MSG_RESULT([no])
> +   fi
> +fi
> +AC_SUBST(SRCHIGH_LIBS)
> +AC_SUBST(SRCHIGH_CFLAGS)
> +
>  # --------------------- #
>  # Check for libmcheck.  #
>  # --------------------- #
> diff --git a/gdb/source-cache.c b/gdb/source-cache.c
> new file mode 100644
> index 0000000000..56851806e4
> --- /dev/null
> +++ b/gdb/source-cache.c
> @@ -0,0 +1,209 @@
> +/* Cache of styled source file text
> +   Copyright (C) 2018 Free Software Foundation, Inc.
> +
> +   This file is part of GDB.
> +
> +   This program is free software; you can redistribute it and/or modify
> +   it under the terms of the GNU General Public License as published by
> +   the Free Software Foundation; either version 3 of the License, or
> +   (at your option) any later version.
> +
> +   This program is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +   GNU General Public License for more details.
> +
> +   You should have received a copy of the GNU General Public License
> +   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
> +
> +#include "defs.h"
> +#include "source-cache.h"
> +#include "common/scoped_fd.h"
> +#include "source.h"
> +#include "cli/cli-style.h"
> +
> +#ifdef HAVE_SOURCE_HIGHLIGHT
> +#include <fstream>
> +#include <sstream>
> +#include <srchilite/sourcehighlight.h>
> +#include <srchilite/langmap.h>
> +#endif
> +
> +/* The number of source files we'll cache.  */
> +
> +#define MAX_ENTRIES 5
> +
> +/* See source-cache.h.  */
> +
> +source_cache g_source_cache;
> +
> +/* See source-cache.h.  */
> +
> +bool
> +source_cache::get_plain_source_lines (struct symtab *s, int first_line,
> +                                     int last_line, std::string *lines)
> +{
> +  scoped_fd desc (open_source_file (s));
> +  if (desc.get () < 0)
> +    return false;
> +
> +  if (s->line_charpos == 0)
> +    find_source_lines (s, desc.get ());
> +
> +  if (first_line < 1 || first_line > s->nlines || last_line < 1)
> +    return false;
> +
> +  if (lseek (desc.get (), s->line_charpos[first_line - 1], SEEK_SET) < 0)
> +    perror_with_name (symtab_to_filename_for_display (s));
> +
> +  int last_charpos;
> +  if (last_line >= s->nlines)
> +    {
> +      struct stat st;
> +
> +      if (fstat (desc.get (), &st) < 0)
> +       perror_with_name (symtab_to_filename_for_display (s));
> +      /* We could cache this in line_charpos... */
> +      last_charpos = st.st_size;
> +    }
> +  else
> +    last_charpos = s->line_charpos[last_line];
> +
> +  lines->resize (last_charpos - s->line_charpos[first_line - 1]);
> +  if (myread (desc.get (), &(*lines)[0], lines->size ()) < 0)
> +    perror_with_name (symtab_to_filename_for_display (s));
> +
> +  return true;
> +}
> +
> +/* See source-cache.h.  */
> +
> +bool
> +source_cache::extract_lines (const struct source_text &text, int first_line,
> +                            int last_line, std::string *lines)
> +{
> +  int lineno = 1;
> +  std::string::size_type pos = 0;
> +  std::string::size_type first_pos = std::string::npos;
> +
> +  while (pos != std::string::npos && lineno <= last_line)
> +    {
> +      std::string::size_type new_pos = text.contents.find ('\n', pos);
> +
> +      if (lineno == first_line)
> +       first_pos = pos;
> +
> +      pos = new_pos;
> +      if (lineno == last_line || pos == std::string::npos)
> +       {
> +         if (pos == std::string::npos)
> +           pos = text.contents.size ();
> +         *lines = text.contents.substr (first_pos, pos - first_pos);
> +         return true;
> +       }
> +      ++lineno;
> +      ++pos;
> +    }
> +
> +  return false;
> +}
> +
> +/* Return the Source Highlight language name, given a gdb language
> +   LANG.  Returns NULL if the language is not known.  */
> +
> +static const char *
> +get_language_name (enum language lang)
> +{
> +  switch (lang)
> +    {
> +    case language_c:
> +    case language_objc:
> +      return "c.lang";
> +
> +    case language_cplus:
> +      return "cpp.lang";
> +
> +    case language_d:
> +      return "d.lang";
> +
> +    case language_go:
> +      return "go.lang";
> +
> +    case language_fortran:
> +      return "fortran.lang";
> +
> +    case language_m2:
> +      /* Not handled by Source Highlight.  */
> +      break;
> +
> +    case language_asm:
> +      return "asm.lang";
> +
> +    case language_pascal:
> +      return "pascal.lang";
> +
> +    case language_opencl:
> +      /* Not handled by Source Highlight.  */
> +      break;
> +
> +    case language_rust:
> +      /* Not handled by Source Highlight.  */
> +      break;
> +
> +    case language_ada:
> +      return "ada.lang";
> +
> +    default:
> +      break;
> +    }
> +
> +  return nullptr;
> +}
> +
> +/* See source-cache.h.  */
> +
> +bool
> +source_cache::get_source_lines (struct symtab *s, int first_line,
> +                               int last_line, std::string *lines)
> +{
> +  if (first_line < 1 || last_line < 1 || first_line > last_line)
> +    return false;
> +
> +#ifdef HAVE_SOURCE_HIGHLIGHT
> +  if (can_emit_style_escape (gdb_stdout))
> +    {
> +      const char *fullname = symtab_to_fullname (s);
> +
> +      for (const auto &item : m_source_map)
> +       {
> +         if (item.fullname == fullname)
> +           return extract_lines (item, first_line, last_line, lines);
> +       }
> +
> +      const char *lang_name = get_language_name (SYMTAB_LANGUAGE (s));
> +      if (lang_name != nullptr)
> +       {
> +         std::ifstream input (fullname);
> +         if (input.is_open ())
> +           {
> +             srchilite::SourceHighlight highlighter ("esc.outlang");
> +             highlighter.setStyleFile("esc.style");
> +
> +             std::ostringstream output;
> +             highlighter.highlight (input, output, lang_name, fullname);
> +
> +             source_text result = { fullname, output.str () };
> +             m_source_map.push_back (std::move (result));
> +
> +             if (m_source_map.size () > MAX_ENTRIES)
> +               m_source_map.erase (m_source_map.begin ());
> +
> +             return extract_lines (m_source_map.back (), first_line,
> +                                   last_line, lines);
> +           }
> +       }
> +    }
> +#endif /* HAVE_SOURCE_HIGHLIGHT */
> +
> +  return get_plain_source_lines (s, first_line, last_line, lines);
> +}
> diff --git a/gdb/source-cache.h b/gdb/source-cache.h
> new file mode 100644
> index 0000000000..2236d38846
> --- /dev/null
> +++ b/gdb/source-cache.h
> @@ -0,0 +1,79 @@
> +/* Cache of styled source file text
> +   Copyright (C) 2018 Free Software Foundation, Inc.
> +
> +   This file is part of GDB.
> +
> +   This program is free software; you can redistribute it and/or modify
> +   it under the terms of the GNU General Public License as published by
> +   the Free Software Foundation; either version 3 of the License, or
> +   (at your option) any later version.
> +
> +   This program is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +   GNU General Public License for more details.
> +
> +   You should have received a copy of the GNU General Public License
> +   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
> +
> +#ifndef SOURCE_CACHE_H
> +#define SOURCE_CACHE_H
> +
> +/* This caches highlighted source text, keyed by the source file's
> +   full name.  A size-limited LRU cache is used.
> +
> +   Highlighting depends on the GNU Source Highlight library.  When not
> +   available, this cache will fall back on reading plain text from the
> +   appropriate file.  */
> +class source_cache
> +{
> +public:
> +
> +  source_cache ()
> +  {
> +  }
> +
> +  /* Get the source text for the source file in symtab S.  FIRST_LINE
> +     and LAST_LINE are the first and last lines to return; line
> +     numbers are 1-based.  If the file cannot be read, false is
> +     returned.  Otherwise, LINES is set to the desired text.  The
> +     returned text may include ANSI terminal escapes.  */
> +  bool get_source_lines (struct symtab *s, int first_line,
> +                        int last_line, std::string *lines);
> +
> +  /* Remove all the items from the source cache.  */
> +  void clear ()
> +  {
> +    m_source_map.clear ();
> +  }
> +
> +private:
> +
> +  /* One element in the cache.  */
> +  struct source_text
> +  {
> +    /* The full name of the file.  */
> +    std::string fullname;
> +    /* The contents of the file.  */
> +    std::string contents;
> +  };
> +
> +  /* A helper function for get_source_lines that is used when the
> +     source lines are not highlighted.  The arguments and return value
> +     are as for get_source_lines.  */
> +  bool get_plain_source_lines (struct symtab *s, int first_line,
> +                              int last_line, std::string *lines);
> +  /* A helper function for get_plain_source_lines that extracts the
> +     desired source lines from TEXT, putting them into LINES.  The
> +     arguments and return value are as for get_source_lines.  */
> +  bool extract_lines (const struct source_text &text, int first_line,
> +                     int last_line, std::string *lines);
> +
> +  /* The contents of the cache.  */
> +  std::vector<source_text> m_source_map;
> +};
> +
> +/* The global source cache.  */
> +extern source_cache g_source_cache;
> +
> +#endif /* SOURCE_CACHE_H */
> diff --git a/gdb/source.c b/gdb/source.c
> index 7552496f78..aa7eb8a6d4 100644
> --- a/gdb/source.c
> +++ b/gdb/source.c
> @@ -45,6 +45,7 @@
>  #include "common/scoped_fd.h"
>  #include <algorithm>
>  #include "common/pathstuff.h"
> +#include "source-cache.h"
>
>  #define OPEN_MODE (O_RDONLY | O_BINARY)
>  #define FDOPEN_MODE FOPEN_RB
> @@ -393,6 +394,7 @@ forget_cached_source_info (void)
>        forget_cached_source_info_for_objfile (objfile);
>      }
>
> +  g_source_cache.clear ();
>    last_source_visited = NULL;
>  }
>
> @@ -1344,25 +1346,18 @@ print_source_lines_base (struct symtab *s, int line, int stopline,
>
>    last_source_error = 0;
>
> -  if (s->line_charpos == 0)
> -    find_source_lines (s, desc.get ());
> -
> -  if (line < 1 || line > s->nlines)
> +  std::string lines;
> +  if (!g_source_cache.get_source_lines (s, line, stopline - 1, &lines))
>      error (_("Line number %d out of range; %s has %d lines."),
>            line, symtab_to_filename_for_display (s), s->nlines);
>
> -  if (lseek (desc.get (), s->line_charpos[line - 1], 0) < 0)
> -    perror_with_name (symtab_to_filename_for_display (s));
> -
> -  gdb_file_up stream = desc.to_file (FDOPEN_MODE);
> -  clearerr (stream.get ());
> -
> +  const char *iter = lines.c_str ();
>    while (nlines-- > 0)
>      {
>        char buf[20];
>
> -      c = fgetc (stream.get ());
> -      if (c == EOF)
> +      c = *iter++;
> +      if (c == '\0')
>         break;
>        last_line_listed = current_source_line;
>        if (flags & PRINT_SOURCE_LINES_FILENAME)
> @@ -1374,7 +1369,7 @@ print_source_lines_base (struct symtab *s, int line, int stopline,
>        uiout->text (buf);
>        do
>         {
> -         if (c < 040 && c != '\t' && c != '\n' && c != '\r')
> +         if (c < 040 && c != '\t' && c != '\n' && c != '\r' && c != '\033')
>             {
>               xsnprintf (buf, sizeof (buf), "^%c", c + 0100);
>               uiout->text (buf);
> @@ -1384,12 +1379,13 @@ print_source_lines_base (struct symtab *s, int line, int stopline,
>           else if (c == '\r')
>             {
>               /* Skip a \r character, but only before a \n.  */
> -             int c1 = fgetc (stream.get ());
> -
> -             if (c1 != '\n')
> +             if (iter[1] == '\n')
> +               {
> +                 ++iter;
> +                 c = '\n';
> +               }
> +             else
>                 printf_filtered ("^%c", c + 0100);
> -             if (c1 != EOF)
> -               ungetc (c1, stream.get ());
>             }
>           else
>             {
> @@ -1397,8 +1393,12 @@ print_source_lines_base (struct symtab *s, int line, int stopline,
>               uiout->text (buf);
>             }
>         }
> -      while (c != '\n' && (c = fgetc (stream.get ())) >= 0);
> +      while (c != '\n' && (c = *iter++) != '\0');
> +      if (c == '\0')
> +       break;
>      }
> +  if (lines.back () != '\n')
> +    uiout->text ("\n");
>  }
>
>  /* Show source lines from the file of symtab S, starting with line
> diff --git a/gdb/tui/tui-data.c b/gdb/tui/tui-data.c
> index 4391f0d100..2b95f197fd 100644
> --- a/gdb/tui/tui-data.c
> +++ b/gdb/tui/tui-data.c
> @@ -838,7 +838,7 @@ free_content_elements (tui_win_content content,
>      {
>        int i;
>
> -      if (type == SRC_WIN || type == DISASSEM_WIN)
> +      if (type == DISASSEM_WIN)
>         {
>           /* Free whole source block.  */
>           xfree (content[0]->which_element.source.line);
> @@ -854,6 +854,9 @@ free_content_elements (tui_win_content content,
>                 {
>                   switch (type)
>                     {
> +                   case SRC_WIN:
> +                     xfree (element->which_element.source.line);
> +                     break;
>                     case DATA_WIN:
>                       xfree (element);
>                       break;
> diff --git a/gdb/tui/tui-io.c b/gdb/tui/tui-io.c
> index b1099246dd..0c7b88f1c7 100644
> --- a/gdb/tui/tui-io.c
> +++ b/gdb/tui/tui-io.c
> @@ -367,9 +367,8 @@ tui_write (const char *buf, size_t length)
>  }
>
>  static void
> -tui_puts_internal (const char *string, int *height)
> +tui_puts_internal (WINDOW *w, const char *string, int *height)
>  {
> -  WINDOW *w = TUI_CMD_WIN->generic.handle;
>    char c;
>    int prev_col = 0;
>
> @@ -410,9 +409,11 @@ tui_puts_internal (const char *string, int *height)
>     necessary.  */
>
>  void
> -tui_puts (const char *string)
> +tui_puts (const char *string, WINDOW *w)
>  {
> -  tui_puts_internal (string, nullptr);
> +  if (w == nullptr)
> +    w = TUI_CMD_WIN->generic.handle;
> +  tui_puts_internal (w, string, nullptr);
>  }
>
>  /* Readline callback.
> @@ -453,7 +454,7 @@ tui_redisplay_readline (void)
>    prev_col = 0;
>    height = 1;
>    if (prompt != nullptr)
> -    tui_puts_internal (prompt, &height);
> +    tui_puts_internal (TUI_CMD_WIN->generic.handle, prompt, &height);
>
>    prev_col = getcurx (w);
>    for (in = 0; in <= rl_end; in++)
> diff --git a/gdb/tui/tui-io.h b/gdb/tui/tui-io.h
> index 11752d0845..95887639ec 100644
> --- a/gdb/tui/tui-io.h
> +++ b/gdb/tui/tui-io.h
> @@ -22,11 +22,13 @@
>  #ifndef TUI_IO_H
>  #define TUI_IO_H
>
> +#include "gdb_curses.h"
> +
>  struct ui_out;
>  class cli_ui_out;
>
>  /* Print the string in the curses command window.  */
> -extern void tui_puts (const char *);
> +extern void tui_puts (const char *, WINDOW * = nullptr);
>
>  /* Print LENGTH characters from the buffer pointed to by BUF to the
>     curses command window.  */
> diff --git a/gdb/tui/tui-source.c b/gdb/tui/tui-source.c
> index 3c4f06b01a..260b274297 100644
> --- a/gdb/tui/tui-source.c
> +++ b/gdb/tui/tui-source.c
> @@ -28,14 +28,90 @@
>  #include "symtab.h"
>  #include "objfiles.h"
>  #include "filenames.h"
> +#include "source-cache.h"
>
>  #include "tui/tui.h"
>  #include "tui/tui-data.h"
> +#include "tui/tui-io.h"
>  #include "tui/tui-stack.h"
>  #include "tui/tui-winsource.h"
>  #include "tui/tui-source.h"
>  #include "gdb_curses.h"
>
> +/* A helper function for tui_set_source_content that extracts some
> +   source text from PTR.  LINE_NO is the line number; FIRST_COL is the
> +   first column to extract, and LINE_WIDTH is the number of characters
> +   to display.  Returns a string holding the desired text.  */
> +
> +static std::string
> +copy_source_line (const char **ptr, int line_no, int first_col,
> +                 int line_width)
> +{
> +  const char *lineptr = *ptr;
> +
> +  /* Init the line with the line number.  */
> +  std::string result = string_printf ("%-6d", line_no);
> +  int len = result.size ();
> +  len = len - ((len / tui_tab_width) * tui_tab_width);
> +  result.append (len, ' ');
> +
> +  int column = 0;
> +  char c;
> +  do
> +    {
> +      int skip_bytes;
> +
> +      c = *lineptr;
> +      if (c == '\033' && skip_ansi_escape (lineptr, &skip_bytes))
> +       {
> +         /* We always have to preserve escapes.  */
> +         result.append (lineptr, lineptr + skip_bytes);
> +         lineptr += skip_bytes;
> +         continue;
> +       }
> +
> +      ++lineptr;
> +      ++column;
> +      /* We have to process all the text in order to pick up all the
> +        escapes.  */
> +      if (column < first_col || column > first_col + line_width)
> +       continue;
> +
> +      if (c == '\n' || c == '\r' || c == '\0')
> +       {
> +         /* Nothing.  */
> +       }
> +      else if (c < 040 && c != '\t')
> +       {
> +         result.push_back ('^');
> +         result.push_back (c + 0100);
> +       }
> +      else if (c == 0177)
> +       {
> +         result.push_back ('^');
> +         result.push_back ('?');
> +       }
> +      else if (c == '\t')
> +       {
> +         int j, max_tab_len = tui_tab_width;
> +
> +         for (j = column - ((column / max_tab_len) * max_tab_len);
> +              j < max_tab_len && column < first_col + line_width;
> +              column++, j++)
> +           result.push_back (' ');
> +       }
> +      else
> +       result.push_back (c);
> +    }
> +  while (c != '\0' && c != '\n' && c != '\r');
> +
> +  if (c == '\r' && *lineptr == '\n')
> +    ++lineptr;
> +  *ptr = lineptr;
> +
> +  return result;
> +}
> +
>  /* Function to display source in the source window.  */
>  enum tui_status
>  tui_set_source_content (struct symtab *s,
> @@ -46,8 +122,7 @@ tui_set_source_content (struct symtab *s,
>
>    if (s != (struct symtab *) NULL)
>      {
> -      int i, c, line_width, nlines;
> -      char *src_line = 0;
> +      int line_width, nlines;
>
>        if ((ret = tui_alloc_source_buffer (TUI_SRC_WIN)) == TUI_SUCCESS)
>         {
> @@ -55,8 +130,10 @@ tui_set_source_content (struct symtab *s,
>           /* Take hilite (window border) into account, when
>              calculating the number of lines.  */
>           nlines = (line_no + (TUI_SRC_WIN->generic.height - 2)) - line_no;
> -         scoped_fd desc = open_source_file (s);
> -         if (desc.get () < 0)
> +
> +         std::string srclines;
> +         if (!g_source_cache.get_source_lines (s, line_no, line_no + nlines,
> +                                               &srclines))
>             {
>               if (!noerror)
>                 {
> @@ -70,165 +147,68 @@ tui_set_source_content (struct symtab *s,
>             }
>           else
>             {
> -             if (s->line_charpos == 0)
> -               find_source_lines (s, desc.get ());
> -
> -             if (line_no < 1 || line_no > s->nlines)
> -               printf_unfiltered ("Line number %d out of range; "
> -                                  "%s has %d lines.\n",
> -                                  line_no,
> -                                  symtab_to_filename_for_display (s),
> -                                  s->nlines);
> -             else if (lseek (desc.get (), s->line_charpos[line_no - 1], 0)
> -                      < 0)
> -               perror_with_name (symtab_to_filename_for_display (s));
> -             else
> +             int cur_line_no, cur_line;
> +             struct tui_gen_win_info *locator
> +               = tui_locator_win_info_ptr ();
> +             struct tui_source_info *src
> +               = &TUI_SRC_WIN->detail.source_info;
> +             const char *s_filename = symtab_to_filename_for_display (s);
> +
> +             if (TUI_SRC_WIN->generic.title)
> +               xfree (TUI_SRC_WIN->generic.title);
> +             TUI_SRC_WIN->generic.title = xstrdup (s_filename);
> +
> +             xfree (src->fullname);
> +             src->fullname = xstrdup (symtab_to_fullname (s));
> +
> +             cur_line = 0;
> +             src->gdbarch = get_objfile_arch (SYMTAB_OBJFILE (s));
> +             src->start_line_or_addr.loa = LOA_LINE;
> +             cur_line_no = src->start_line_or_addr.u.line_no = line_no;
> +
> +             const char *iter = srclines.c_str ();
> +             while (cur_line < nlines)
>                 {
> -                 int offset, cur_line_no, cur_line, cur_len, threshold;
> -                 struct tui_gen_win_info *locator
> -                   = tui_locator_win_info_ptr ();
> -                  struct tui_source_info *src
> -                   = &TUI_SRC_WIN->detail.source_info;
> -                 const char *s_filename = symtab_to_filename_for_display (s);
> -
> -                  if (TUI_SRC_WIN->generic.title)
> -                    xfree (TUI_SRC_WIN->generic.title);
> -                  TUI_SRC_WIN->generic.title = xstrdup (s_filename);
> -
> -                 xfree (src->fullname);
> -                 src->fullname = xstrdup (symtab_to_fullname (s));
> -
> -                 /* Determine the threshold for the length of the
> -                     line and the offset to start the display.  */
> -                 offset = src->horizontal_offset;
> -                 threshold = (line_width - 1) + offset;
> -                 gdb_file_up stream = desc.to_file (FOPEN_RT);
> -                 clearerr (stream.get ());
> -                 cur_line = 0;
> -                 src->gdbarch = get_objfile_arch (SYMTAB_OBJFILE (s));
> -                 src->start_line_or_addr.loa = LOA_LINE;
> -                 cur_line_no = src->start_line_or_addr.u.line_no = line_no;
> -                 if (offset > 0)
> -                   src_line = (char *) xmalloc (
> -                                          (threshold + 1) * sizeof (char));
> -                 while (cur_line < nlines)
> -                   {
> -                     struct tui_win_element *element
> -                       = TUI_SRC_WIN->generic.content[cur_line];
> -
> -                     /* Get the first character in the line.  */
> -                     c = fgetc (stream.get ());
> -
> -                     if (offset == 0)
> -                       src_line = TUI_SRC_WIN->generic.content[cur_line]
> -                                    ->which_element.source.line;
> -                     /* Init the line with the line number.  */
> -                     sprintf (src_line, "%-6d", cur_line_no);
> -                     cur_len = strlen (src_line);
> -                     i = cur_len - ((cur_len / tui_tab_width)
> -                                    * tui_tab_width);
> -                     while (i < tui_tab_width)
> -                       {
> -                         src_line[cur_len] = ' ';
> -                         i++;
> -                         cur_len++;
> -                       }
> -                     src_line[cur_len] = (char) 0;
> -
> -                     /* Set whether element is the execution point
> -                        and whether there is a break point on it.  */
> -                     element->which_element.source.line_or_addr.loa =
> -                       LOA_LINE;
> -                     element->which_element.source.line_or_addr.u.line_no =
> -                       cur_line_no;
> -                     element->which_element.source.is_exec_point =
> -                       (filename_cmp (locator->content[0]
> -                                        ->which_element.locator.full_name,
> -                                      symtab_to_fullname (s)) == 0
> -                                        && cur_line_no
> -                                             == locator->content[0]
> -                                                  ->which_element.locator.line_no);
> -                     if (c != EOF)
> -                       {
> -                         i = strlen (src_line) - 1;
> -                         do
> -                           {
> -                             if ((c != '\n') && (c != '\r')
> -                                 && (++i < threshold))
> -                               {
> -                                 if (c < 040 && c != '\t')
> -                                   {
> -                                     src_line[i++] = '^';
> -                                     src_line[i] = c + 0100;
> -                                   }
> -                                 else if (c == 0177)
> -                                   {
> -                                     src_line[i++] = '^';
> -                                     src_line[i] = '?';
> -                                   }
> -                                 else
> -                                   { /* Store the charcter in the
> -                                        line buffer.  If it is a tab,
> -                                        then translate to the correct
> -                                        number of chars so we don't
> -                                        overwrite our buffer.  */
> -                                     if (c == '\t')
> -                                       {
> -                                         int j, max_tab_len
> -                                           = tui_tab_width;
> -
> -                                         for (j = i - ((i / max_tab_len)
> -                                                       * max_tab_len);
> -                                              j < max_tab_len
> -                                                && i < threshold;
> -                                              i++, j++)
> -                                           src_line[i] = ' ';
> -                                         i--;
> -                                       }
> -                                     else
> -                                       src_line[i] = c;
> -                                   }
> -                                 src_line[i + 1] = 0;
> -                               }
> -                             else
> -                               { /* If we have not reached EOL, then
> -                                    eat chars until we do.  */
> -                                 while (c != EOF && c != '\n' && c != '\r')
> -                                   c = fgetc (stream.get ());
> -                                 /* Handle non-'\n' end-of-line.  */
> -                                 if (c == '\r'
> -                                     && (c = fgetc (stream.get ())) != '\n'
> -                                     && c != EOF)
> -                                   {
> -                                     ungetc (c, stream.get ());
> -                                     c = '\r';
> -                                   }
> -
> -                               }
> -                           }
> -                         while (c != EOF && c != '\n' && c != '\r'
> -                                && i < threshold
> -                                && (c = fgetc (stream.get ())));
> -                       }
> -                     /* Now copy the line taking the offset into
> -                        account.  */
> -                     if (offset == 0)
> -                       ;
> -                     else if (strlen (src_line) > offset)
> -                       strcpy (TUI_SRC_WIN->generic.content[cur_line]
> -                                 ->which_element.source.line,
> -                               &src_line[offset]);
> -                     else
> -                       TUI_SRC_WIN->generic.content[cur_line]
> -                         ->which_element.source.line[0] = (char) 0;
> -                     cur_line++;
> -                     cur_line_no++;
> -                   }
> -                 if (offset > 0)
> -                   xfree (src_line);
> -                 TUI_SRC_WIN->generic.content_size = nlines;
> -                 ret = TUI_SUCCESS;
> +                 struct tui_win_element *element
> +                   = TUI_SRC_WIN->generic.content[cur_line];
> +
> +                 std::string text;
> +                 if (*iter != '\0')
> +                   text = copy_source_line (&iter, cur_line_no,
> +                                            src->horizontal_offset,
> +                                            line_width);
> +
> +                 /* Set whether element is the execution point
> +                    and whether there is a break point on it.  */
> +                 element->which_element.source.line_or_addr.loa =
> +                   LOA_LINE;
> +                 element->which_element.source.line_or_addr.u.line_no =
> +                   cur_line_no;
> +                 element->which_element.source.is_exec_point =
> +                   (filename_cmp (locator->content[0]
> +                                  ->which_element.locator.full_name,
> +                                  symtab_to_fullname (s)) == 0
> +                    && cur_line_no
> +                    == locator->content[0]
> +                    ->which_element.locator.line_no);
> +
> +                 xfree (TUI_SRC_WIN->generic.content[cur_line]
> +                        ->which_element.source.line);
> +                 int alloc_len = text.size ();
> +                 if (alloc_len < line_width)
> +                   alloc_len = line_width + 1;
> +                 TUI_SRC_WIN->generic.content[cur_line]
> +                   ->which_element.source.line
> +                   = (char *) xmalloc (alloc_len);
> +                 strcpy (TUI_SRC_WIN->generic.content[cur_line]
> +                         ->which_element.source.line,
> +                         text.c_str ());
> +
> +                 cur_line++;
> +                 cur_line_no++;
>                 }
> +             TUI_SRC_WIN->generic.content_size = nlines;
> +             ret = TUI_SUCCESS;
>             }
>         }
>      }
> diff --git a/gdb/tui/tui-winsource.c b/gdb/tui/tui-winsource.c
> index 0bf74383b1..00b4b9e4fa 100644
> --- a/gdb/tui/tui-winsource.c
> +++ b/gdb/tui/tui-winsource.c
> @@ -31,6 +31,7 @@
>
>  #include "tui/tui.h"
>  #include "tui/tui-data.h"
> +#include "tui/tui-io.h"
>  #include "tui/tui-stack.h"
>  #include "tui/tui-win.h"
>  #include "tui/tui-wingeneral.h"
> @@ -277,8 +278,9 @@ tui_show_source_line (struct tui_win_info *win_info, int lineno)
>    if (line->which_element.source.is_exec_point)
>      wattron (win_info->generic.handle, A_STANDOUT);
>
> -  mvwaddstr (win_info->generic.handle, lineno, 1,
> -            line->which_element.source.line);
> +  wmove (win_info->generic.handle, lineno, 1);
> +  tui_puts (line->which_element.source.line,
> +           win_info->generic.handle);
>    if (line->which_element.source.is_exec_point)
>      wattroff (win_info->generic.handle, A_STANDOUT);
>
> @@ -595,7 +597,6 @@ tui_update_exec_info (struct tui_win_info *win_info)
>  enum tui_status
>  tui_alloc_source_buffer (struct tui_win_info *win_info)
>  {
> -  char *src_line_buf;
>    int i, line_width, max_lines;
>
>    /* The window width/height includes the highlight box.  Determine actual
> @@ -603,20 +604,14 @@ tui_alloc_source_buffer (struct tui_win_info *win_info)
>    max_lines = win_info->generic.height - 2;
>    line_width = win_info->generic.width - 2 + 1;
>
> -  /*
> -   * Allocate the buffer for the source lines.  Do this only once
> -   * since they will be re-used for all source displays.  The only
> -   * other time this will be done is when a window's size changes.
> -   */
> +  /* Allocate the buffer for the source lines.  */
>    if (win_info->generic.content == NULL)
>      {
> -      src_line_buf = (char *)
> -       xmalloc ((max_lines * line_width) * sizeof (char));
>        /* Allocate the content list.  */
>        win_info->generic.content = tui_alloc_content (max_lines, SRC_WIN);
>        for (i = 0; i < max_lines; i++)
>         win_info->generic.content[i]->which_element.source.line
> -         = src_line_buf + (line_width * i);
> +         = (char *) xmalloc (line_width);
>      }
>
>    return TUI_SUCCESS;
> diff --git a/gdb/utils.c b/gdb/utils.c
> index 85b0fb14e3..00f524c52e 100644
> --- a/gdb/utils.c
> +++ b/gdb/utils.c
> @@ -1444,9 +1444,9 @@ emit_style_escape (const ui_file_style &style)
>    wrap_buffer.append (style.to_ansi ());
>  }
>
> -/* Return true if ANSI escapes can be used on STREAM.  */
> +/* See utils.h.  */
>
> -static bool
> +bool
>  can_emit_style_escape (struct ui_file *stream)
>  {
>    if (stream != gdb_stdout
> diff --git a/gdb/utils.h b/gdb/utils.h
> index 1f09ec2d47..16ee9fbd33 100644
> --- a/gdb/utils.h
> +++ b/gdb/utils.h
> @@ -443,6 +443,10 @@ extern void fputs_styled (const char *linebuffer,
>
>  extern void reset_terminal_style (struct ui_file *stream);
>
> +/* Return true if ANSI escapes can be used on STREAM.  */
> +
> +extern bool can_emit_style_escape (struct ui_file *stream);
> +
>  /* Display the host ADDR on STREAM formatted as ``0x%x''.  */
>  extern void gdb_print_host_address_1 (const void *addr, struct ui_file *stream);
>
> --
> 2.17.2
>
  

> From: Andrew Pinski <pinskia@gmail.com>
> Date: Fri, 15 Nov 2019 16:57:49 -0800
> Cc: GDB Patches <gdb-patches@sourceware.org>
> 
> If it cannot be compiled at a top-level, does it support as a static
> library?

Not sure I understand what you are asking about.  If you ask whether
GDB can be linked statically with Source Highlight, then the answer is
YES.  In fact, this is how I build my GDB, because the shared library
version of Source Highlight includes (on MS-Windows) dependencies on
libstdc++ and libgcc DLLs, which makes it impossible for me to
redistribute the GDB binaries without also providing the humongous GCC
source tarballs.

Apologies if this is not the question you were asking.
  

Andrew> I know this patch has been in for almost a year now.  But I have a
Andrew> question, is there a way to build GNU Source Highlight as a static
Andrew> library and have it built at the top level?  Like what is done for
Andrew> GMP/MPFR for GCC?  It would reduce how I built a gdb that is for a
Andrew> non-native target.

There's currently no support for dropping the sources into the tree.
However, I think it would be fine if you wanted to add such support.

Andrew> If it cannot be compiled at a top-level, does it support as a static
Andrew> library?

Eli answered this in the affirmative; I have never tried it myself.

I've considered also adding support for Pygments, in addition to Source
Highlight.  In some situations that might be simpler to distribute; and
also it would mean that one could just install Pygments and have it
suddenly start working.

Tom
  

On Sun, Nov 24, 2019 at 11:41 AM Tom Tromey <tom@tromey.com> wrote:
>
> Andrew> I know this patch has been in for almost a year now.  But I have a
> Andrew> question, is there a way to build GNU Source Highlight as a static
> Andrew> library and have it built at the top level?  Like what is done for
> Andrew> GMP/MPFR for GCC?  It would reduce how I built a gdb that is for a
> Andrew> non-native target.
>
> There's currently no support for dropping the sources into the tree.
> However, I think it would be fine if you wanted to add such support.

Note that the source highlight library depends on boost_regex and ICU,
according to ldd, so that might make this harder?

(I wonder why it doesn't use std::regex)

Christian
  

Christian> Note that the source highlight library depends on boost_regex and ICU,
Christian> according to ldd, so that might make this harder?

Yeah.  I forgot about that.

Christian> (I wonder why it doesn't use std::regex)

There's a bug open in the source highlight tracker about this (I can't
link right now since savannah seems to be down).  Basically, I think
std::regex wasn't available (maybe not implemented in libstdc++?) when
it was written.  Anyway, removing the boost dependency would be good.

Tom