[5/7] gdb/debuginfod: Support on-demand debuginfo downloading

Message ID 20230227194212.348003-5-amerey@redhat.com
State New
Headers
Series [1/7] gdb/debuginfod: Add debuginfod_section_query |

Commit Message

Aaron Merey Feb. 27, 2023, 7:42 p.m. UTC
  At the beginning of a session, gdb may attempt to download debuginfo
for all shared libraries associated with the process or core file
being debugged.  This can be a waste of time and storage space when much
of the debuginfo ends up not being used during the session.

To reduce the gdb's startup latency and to download only the debuginfo
that is really needed, this patch adds on-demand, a.k.a lazy, downloading.

When 'set debuginfo enabled lazy' is on, gdb will attempt to download
a .gdb_index for each shared library instead of its full debuginfo.
Each debuginfo download will be deferred until gdb needs to expand symtabs
associated with the debuginfo's index.

Because these indices are significantly smaller than their corresponding
debuginfo, this generally reduces the total amount of data gdb downloads
by a large margin.  Reductions of 80%-95% have been commonly observed when
debugging large GUI programs.

    (gdb) set debuginfod enabled lazy
    (gdb) start
    Downloading section .gdb_index for /lib64/libcurl.so.4
    [...]
    1826        client->server_mhandle = curl_multi_init ();
    (gdb) step
    Downloading separate debug info for /lib64/libcurl.so.4
    Downloading separate debug info for [libcurl dwz]
    curl_multi_init () at ../../lib/multi.c:457
    Downloading source file /usr/src/debug/curl-7.85.0-6.fc37.x86_64/build-full/lib/../../lib/multi.c
    457     {
    (gdb)

Some of the key functions below include dwarf2_has_separate_index which
downloads the separate .gdb_index.  If successful, it initializes a
separate debug objfile to own the index and act as a placeholder for
the full debuginfo that might be downloaded later.

read_full_dwarf_from_debuginfod downloads the full debuginfo and
reinitializes the separate debug objfile.  It is called by
dwarf2_gdb_index::expand_symtabs_matching and
dwarf2_base_index_functions::find_pc_sect_compunit_symtab when symtab
expansion is required.
---
 gdb/dwarf2/frame.c          |  13 +++
 gdb/dwarf2/frame.h          |   4 +
 gdb/dwarf2/index-cache.c    |  33 ++++++
 gdb/dwarf2/index-cache.h    |  13 +++
 gdb/dwarf2/public.h         |   7 ++
 gdb/dwarf2/read-gdb-index.c |  89 ++++++++++++----
 gdb/dwarf2/read.c           | 197 +++++++++++++++++++++++++++++++++++-
 gdb/dwarf2/read.h           |   9 ++
 gdb/dwarf2/section.c        |   4 +-
 gdb/elfread.c               |   2 +-
 gdb/objfiles.c              |  18 ++++
 gdb/objfiles.h              |  16 +++
 gdb/symfile.c               |  76 +++++++++++++-
 gdb/symfile.h               |   7 ++
 gdb/symtab.h                |   5 +
 15 files changed, 472 insertions(+), 21 deletions(-)
  

Comments

Tom Tromey March 7, 2023, 8:20 p.m. UTC | #1
>>>>> "Aaron" == Aaron Merey via Gdb-patches <gdb-patches@sourceware.org> writes:

Aaron> At the beginning of a session, gdb may attempt to download debuginfo
Aaron> for all shared libraries associated with the process or core file
Aaron> being debugged.  This can be a waste of time and storage space when much
Aaron> of the debuginfo ends up not being used during the session.

Thank you for the patch.

Aaron> +/* See frame.h.  */
Aaron> +
Aaron> +void
Aaron> +clear_comp_unit (struct objfile *objfile)

"comp unit" has a very different meaning in the DWARF code, it's
confusing here.

Maybe a name like "dwarf2_clear_frame_data" would be better.
Though I wonder how this gets populated if there's no frame info.

Aaron> +  bool do_expand_symtabs_matching

I tend to think this isn't needed, see below.

Aaron> +bool
Aaron> +dwarf2_gdb_index::expand_symtabs_matching
Aaron> +    (struct objfile *objfile,
Aaron> +     gdb::function_view<expand_symtabs_file_matcher_ftype> file_matcher,
Aaron> +     const lookup_name_info *lookup_name,
Aaron> +     gdb::function_view<expand_symtabs_symbol_matcher_ftype> symbol_matcher,
Aaron> +     gdb::function_view<expand_symtabs_exp_notify_ftype> expansion_notify,
Aaron> +     block_search_flags search_flags,
Aaron> +     domain_enum domain,
Aaron> +     enum search_domain kind)
Aaron> +{
Aaron> +  if (objfile->flags & OBJF_READNEVER)

Can this even happen here?  If 'readnever' is set, debuginfod simply
should never be queried in the first plae.

Aaron> +  try
Aaron> +    {
Aaron> +      return do_expand_symtabs_matching (objfile, file_matcher, lookup_name,
Aaron> +					 symbol_matcher, expansion_notify,
Aaron> +					 search_flags, domain, kind);
Aaron> +    }
Aaron> +  catch (gdb_exception e)
Aaron> +    {
Aaron> +      if (!objfile->is_separate_index_stub ())
Aaron> +	{
Aaron> +	  exception_print (gdb_stderr, e);
Aaron> +	  return false;
Aaron> +	}

I guess the idea here is to try not to download anything -- maybe the
search will fail and we can skip the downloading entirely.

However, it seems better in this case to integrate it into the inner
loop of expand_symtabs_matching.

Also other methods of dwarf2_gdb_index can result in CU expansion, and
so require all the DWARF.  So maybe some other refactoring is needed,
like some kind of virtual function that is called before any possible
call to dw2_instantiate_symtab along these code paths.  I see some of
this in read.c but it seems like some are missing.

E.g., it seems like calls to dw2_expand_symtabs_matching_one are
unaffected, but should be.

Aaron> +/* See public.h.  */
Aaron> +
Aaron> +bool
Aaron> +dwarf2_has_separate_index (struct objfile *objfile)
Aaron> +{
Aaron> +  if (objfile->flags & OBJF_MAINLINE

What's the reason for testing this flag?

Anyway, gdb style is: (objfile->flags & OBJF_MAINLINE) != 0

Aaron> +    return 0;

bool

Aaron> +      /* We found a separate .gdb_index file so a separate debuginfo file
Aaron> +	 should exist, but don't want to read it until we really have to.
Aaron> +	 Create an objfile to own the index information and act as a
Aaron> +	 placeholder for the debuginfo that we have the option of aquiring
Aaron> +	 later.  */
Aaron> +      gdb_bfd_ref_ptr abfd (gdb_bfd_open (objfile_filename (objfile), gnutarget));
Aaron> +      if (abfd == nullptr)
Aaron> +	return false;
Aaron> +
Aaron> +      dwarf2_per_bfd_objfile_data_key.clear (objfile);
Aaron> +      dwarf2_objfile_data_key.clear (objfile);

This seems suspicious.

Aaron> +      dwarf2_per_bfd *per_bfd;
Aaron> +      dwarf2_per_objfile *per_objfile = get_dwarf2_per_objfile (objfile);
Aaron> +
Aaron> +      /* Perform a limited initialization based to dwarf2_has_info and
Aaron> +	 dwarf2_initialize_objfile.  */
Aaron> +      if (per_objfile == nullptr)
Aaron> +	{
Aaron> +	  per_bfd = dwarf2_per_bfd_objfile_data_key.get (objfile);
Aaron> +	  if (per_bfd == nullptr)
Aaron> +	    {
Aaron> +	      per_bfd = new dwarf2_per_bfd (objfile->obfd.get (), nullptr, false);
Aaron> +	      dwarf2_per_bfd_objfile_data_key.set (objfile, per_bfd);
Aaron> +	    }
Aaron> +	  per_objfile = dwarf2_objfile_data_key.emplace (objfile, objfile, per_bfd);
Aaron> +	}

Changing the objfile seems wrong.
I don't understand why this is needed.

Aaron> +      struct objfile *stub = objfile->separate_debug_objfile;
Aaron> +      per_objfile = get_dwarf2_per_objfile (stub);
Aaron> +      if (per_objfile == nullptr)
Aaron> +	{
Aaron> +	  per_bfd = dwarf2_per_bfd_objfile_data_key.get (stub);
Aaron> +	  if (per_bfd == nullptr)
Aaron> +	    {
Aaron> +	      per_bfd = new dwarf2_per_bfd (stub->obfd.get (), nullptr, false);
Aaron> +	      dwarf2_per_bfd_objfile_data_key.set (stub, per_bfd);
Aaron> +	    }
Aaron> +	  per_objfile = dwarf2_objfile_data_key.emplace (stub, stub, per_bfd);
Aaron> +	}

This also seems suspect.  Shouldn't it have been done when the new stub
objfile was created, like by dwarf2_has_info or something like that?

Aaron> diff --git a/gdb/dwarf2/section.c b/gdb/dwarf2/section.c
Aaron> index c9ef41893ee..ff8ef527c29 100644
Aaron> --- a/gdb/dwarf2/section.c
Aaron> +++ b/gdb/dwarf2/section.c
Aaron> @@ -54,7 +54,9 @@ dwarf2_section_info::get_bfd_owner () const
Aaron>        section = get_containing_section ();
Aaron>        gdb_assert (!section->is_virtual);
Aaron>      }
Aaron> -  gdb_assert (section->s.section != nullptr);
Aaron> +  if (section->s.section == nullptr)
Aaron> +    throw_error (NOT_FOUND_ERROR,
Aaron> +		 _("Can't find owner of DWARF section."));

I think this can just use error, but when is this needed?

Aaron> +  /* Return true if this objfile holds .gdb_index data and represents
Aaron> +     a debuginfo file whose download has been deferred.  */
Aaron> +
Aaron> +  bool is_separate_index_stub ()
Aaron> +  {
Aaron> +    return strcmp (original_name, SEPARATE_INDEX_FILENAME) == 0;
Aaron> +  }

If this is needed then I think it's better to have a new OBJF_ flag
rather than a magic file name.

Aaron> +bool
Aaron> +objfile::reinit (struct bfd *abfd)
Aaron> +{
Aaron> +  try
Aaron> +  {
Aaron> +    deferred_read_symbols (this, abfd);
Aaron> +    return true;
Aaron> +  }
Aaron> +  catch (const gdb_exception &except)
Aaron> +  {
Aaron> +    exception_print (gdb_stderr, except);
Aaron> +    this->flags |= OBJF_READNEVER;

Ok, maybe this answers my readnever question.

Aaron> +void
Aaron> +symbol_file_add_from_index (const gdb_bfd_ref_ptr &bfd,
Aaron> +			    symfile_add_flags symfile_flags,
Aaron> +			    struct objfile *parent)
Aaron> +{
Aaron> +  section_addr_info sap = build_section_addr_info_from_objfile (parent);
Aaron> +
Aaron> +  symbol_file_add_with_addrs
Aaron> +    (bfd, SEPARATE_INDEX_FILENAME, symfile_flags, &sap,
Aaron> +     (parent->flags & (OBJF_REORDERED | OBJF_SHARED | OBJF_READNOW
Aaron> +		      | OBJF_USERLOADED | OBJF_MAINLINE | OBJF_PSYMTABS_READ))
Aaron> +		  | OBJF_NOT_FILENAME, parent, true);
Aaron> +
Aaron> +  objfile *result = parent->separate_debug_objfile;
Aaron> +  init_objfile_sect_indices (result);
Aaron> +
Aaron> +  return;

No need for this.

Aaron> +/* See symtab.h.  */
Aaron> +
Aaron> +void
Aaron> +deferred_read_symbols (struct objfile *objfile, struct bfd *abfd)
Aaron> +{
Aaron> +  const char *obfd_filename;
Aaron> +
Aaron> +  obfd_filename = bfd_get_filename (abfd);
Aaron> +  gdb_bfd_ref_ptr temp (gdb_bfd_open (obfd_filename, gnutarget));
Aaron> +
Aaron> +  if (temp == nullptr)
Aaron> +    error (_("Can't open %s to read symbols."), obfd_filename);
Aaron> +
Aaron> +  if (!bfd_check_format (temp.get (), bfd_object))
Aaron> +    error (_("Can't read symbols from %s: %s."), obfd_filename,
Aaron> +	   bfd_errmsg (bfd_get_error ()));
Aaron> +
Aaron> +  objfile->obfd = std::move (temp);
Aaron> +
Aaron> +  /* Nuke all the state that we will re-read.  */
Aaron> +  objfile->registry_fields.clear_registry ();
Aaron> +  objfile->sections = NULL;
Aaron> +  objfile->sect_index_bss = -1;
Aaron> +  objfile->sect_index_data = -1;
Aaron> +  objfile->sect_index_rodata = -1;
Aaron> +  objfile->sect_index_text = -1;
Aaron> +  objfile->compunit_symtabs = NULL;
Aaron> +  objfile->template_symbols = NULL;
Aaron> +
Aaron> +  objfile_set_sym_fns (objfile, find_sym_fns (objfile->obfd.get ()));
Aaron> +  build_objfile_section_table (objfile);
Aaron> +  (*objfile->sf->sym_init) (objfile);
Aaron> +  init_objfile_sect_indices (objfile);
Aaron> +
Aaron> +  objfile->section_offsets.resize (gdb_bfd_count_sections
Aaron> +				     (objfile->obfd.get ()));
Aaron> +  read_symbols (objfile, 0);
Aaron> +
Aaron> +  objfile->mtime = bfd_get_mtime (objfile->obfd.get ());
Aaron> +  objfile->original_name
Aaron> +    = obstack_strdup (&objfile->objfile_obstack, obfd_filename);

So, gdb already does this kind of thing in reread_symbols, and it's been
a source of bugs.  I wonder if it's possible to just create a new,
non-stub objfile here and splice it in; removing the stub one.

Alternatively, I also don't really understand why this is needed here.
It seems like the reading of the symbols can be done purely in the DWARF
reader.

The idea of the "quick" API is to isolate the core gdb from whatever the
readers do under the hood.  It seems to be that the stub objfile can be
created with the index, and then when the full data is needed, it can
just be read in without involving or notifying any other part of gdb.
This is what's currently done with section data after all, just in this
case it's being downloaded from some server.

One other thing to consider is the sharing of indices.  The DWARF reader
tries pretty hard to share data across inferiors.  However I think this
new code will download the index separately for each inferior.  The full
DWARF will be shared again, due to the BFD cache.

The way this works is that the DWARF reader puts all the shareable data
into an object that is attached to the BFD's registry.  Then when a new
objfile is initialized, it checks the BFD's registry to see if there's
already DWARF info attached.  If so, it's reused.

The index code could maybe attach its own object to the parent BFD to
make this sharing work.

Tom
  
Aaron Merey March 9, 2023, 12:22 a.m. UTC | #2
On Tue, Mar 7, 2023 at 3:20 PM Tom Tromey <tom@tromey.com> wrote:
> Aaron> +/* See frame.h.  */
> Aaron> +
> Aaron> +void
> Aaron> +clear_comp_unit (struct objfile *objfile)
>
> "comp unit" has a very different meaning in the DWARF code, it's
> confusing here.
>
> Maybe a name like "dwarf2_clear_frame_data" would be better.
> Though I wonder how this gets populated if there's no frame info.

Will change the name.  I don't remember the details but the comp_unit
could end up with an empty fde_table when DWARF wasn't present.  The
empty table would hang around and indicate a lack of DWARF frame data
unless cleared this way, even after DWARF was acquired.

> Aaron> +  try
> Aaron> +    {
> Aaron> +      return do_expand_symtabs_matching (objfile, file_matcher, lookup_name,
> Aaron> +                                         symbol_matcher, expansion_notify,
> Aaron> +                                         search_flags, domain, kind);
> Aaron> +    }
> Aaron> +  catch (gdb_exception e)
> Aaron> +    {
> Aaron> +      if (!objfile->is_separate_index_stub ())
> Aaron> +        {
> Aaron> +          exception_print (gdb_stderr, e);
> Aaron> +          return false;
> Aaron> +        }
>
> I guess the idea here is to try not to download anything -- maybe the
> search will fail and we can skip the downloading entirely.
>
> However, it seems better in this case to integrate it into the inner
> loop of expand_symtabs_matching.

The reason for downloading the full DWARF at a level above the inner
loop of expand_symtabs_matching is because the loop acts on the
per_objfile and per_bfd which are deleted and reinitialized when
full DWARF is downloaded.

> Also other methods of dwarf2_gdb_index can result in CU expansion, and
> so require all the DWARF.  So maybe some other refactoring is needed,
> like some kind of virtual function that is called before any possible
> call to dw2_instantiate_symtab along these code paths.  I see some of
> this in read.c but it seems like some are missing.
>
> E.g., it seems like calls to dw2_expand_symtabs_matching_one are
> unaffected, but should be.

Will fix this.

> Aaron> +/* See public.h.  */
> Aaron> +
> Aaron> +bool
> Aaron> +dwarf2_has_separate_index (struct objfile *objfile)
> Aaron> +{
> Aaron> +  if (objfile->flags & OBJF_MAINLINE
>
> What's the reason for testing this flag?
>
> Anyway, gdb style is: (objfile->flags & OBJF_MAINLINE) != 0

I wanted to avoid unnecessarily downloading a separate .gdb_index
since we'll always want the debuginfo of the main executable.

> Aaron> +    return 0;
>
> bool
>
> Aaron> +      /* We found a separate .gdb_index file so a separate debuginfo file
> Aaron> +         should exist, but don't want to read it until we really have to.
> Aaron> +         Create an objfile to own the index information and act as a
> Aaron> +         placeholder for the debuginfo that we have the option of aquiring
> Aaron> +         later.  */
> Aaron> +      gdb_bfd_ref_ptr abfd (gdb_bfd_open (objfile_filename (objfile), gnutarget));
> Aaron> +      if (abfd == nullptr)
> Aaron> +        return false;
> Aaron> +
> Aaron> +      dwarf2_per_bfd_objfile_data_key.clear (objfile);
> Aaron> +      dwarf2_objfile_data_key.clear (objfile);
>
> This seems suspicious.
>
> Aaron> +      dwarf2_per_bfd *per_bfd;
> Aaron> +      dwarf2_per_objfile *per_objfile = get_dwarf2_per_objfile (objfile);
> Aaron> +
> Aaron> +      /* Perform a limited initialization based to dwarf2_has_info and
> Aaron> +         dwarf2_initialize_objfile.  */
> Aaron> +      if (per_objfile == nullptr)
> Aaron> +        {
> Aaron> +          per_bfd = dwarf2_per_bfd_objfile_data_key.get (objfile);
> Aaron> +          if (per_bfd == nullptr)
> Aaron> +            {
> Aaron> +              per_bfd = new dwarf2_per_bfd (objfile->obfd.get (), nullptr, false);
> Aaron> +              dwarf2_per_bfd_objfile_data_key.set (objfile, per_bfd);
> Aaron> +            }
> Aaron> +          per_objfile = dwarf2_objfile_data_key.emplace (objfile, objfile, per_bfd);
> Aaron> +        }
>
> Changing the objfile seems wrong.
> I don't understand why this is needed.

You're right, this is unnecessary.

> Aaron> +      struct objfile *stub = objfile->separate_debug_objfile;
> Aaron> +      per_objfile = get_dwarf2_per_objfile (stub);
> Aaron> +      if (per_objfile == nullptr)
> Aaron> +        {
> Aaron> +          per_bfd = dwarf2_per_bfd_objfile_data_key.get (stub);
> Aaron> +          if (per_bfd == nullptr)
> Aaron> +            {
> Aaron> +              per_bfd = new dwarf2_per_bfd (stub->obfd.get (), nullptr, false);
> Aaron> +              dwarf2_per_bfd_objfile_data_key.set (stub, per_bfd);
> Aaron> +            }
> Aaron> +          per_objfile = dwarf2_objfile_data_key.emplace (stub, stub, per_bfd);
> Aaron> +        }
>
> This also seems suspect.  Shouldn't it have been done when the new stub
> objfile was created, like by dwarf2_has_info or something like that?

We avoid a call to dwarf2_has_info by returning early from
symbol_file_add_with_addrs.

I tried to contain the stub initialization to this function to avoid unintended
interactions with the normal objfile process while also minimizing changes
to existing code.

We don't want to interact with the stub's bfd (a bfd of the shared library)
any more than we need to.  It's only there because gdb requires some of
the section indices and offsets.

> Aaron> diff --git a/gdb/dwarf2/section.c b/gdb/dwarf2/section.c
> Aaron> index c9ef41893ee..ff8ef527c29 100644
> Aaron> --- a/gdb/dwarf2/section.c
> Aaron> +++ b/gdb/dwarf2/section.c
> Aaron> @@ -54,7 +54,9 @@ dwarf2_section_info::get_bfd_owner () const
> Aaron>        section = get_containing_section ();
> Aaron>        gdb_assert (!section->is_virtual);
> Aaron>      }
> Aaron> -  gdb_assert (section->s.section != nullptr);
> Aaron> +  if (section->s.section == nullptr)
> Aaron> +    throw_error (NOT_FOUND_ERROR,
> Aaron> +                 _("Can't find owner of DWARF section."));
>
> I think this can just use error, but when is this needed?

section->s.section == nullptr when attempting to expand symtabs for a
stub objfile.  This error is caught in expand_symtabs_matching and
find_pc_sect_compunit_symtab which triggers the full debuginfo download.

> Aaron> +  /* Return true if this objfile holds .gdb_index data and represents
> Aaron> +     a debuginfo file whose download has been deferred.  */
> Aaron> +
> Aaron> +  bool is_separate_index_stub ()
> Aaron> +  {
> Aaron> +    return strcmp (original_name, SEPARATE_INDEX_FILENAME) == 0;
> Aaron> +  }
>
> If this is needed then I think it's better to have a new OBJF_ flag
> rather than a magic file name.

Agreed.

> Aaron> +bool
> Aaron> +objfile::reinit (struct bfd *abfd)
> Aaron> +{
> Aaron> +  try
> Aaron> +  {
> Aaron> +    deferred_read_symbols (this, abfd);
> Aaron> +    return true;
> Aaron> +  }
> Aaron> +  catch (const gdb_exception &except)
> Aaron> +  {
> Aaron> +    exception_print (gdb_stderr, except);
> Aaron> +    this->flags |= OBJF_READNEVER;
>
> Ok, maybe this answers my readnever question.
>
> Aaron> +void
> Aaron> +symbol_file_add_from_index (const gdb_bfd_ref_ptr &bfd,
> Aaron> +                            symfile_add_flags symfile_flags,
> Aaron> +                            struct objfile *parent)
> Aaron> +{
> Aaron> +  section_addr_info sap = build_section_addr_info_from_objfile (parent);
> Aaron> +
> Aaron> +  symbol_file_add_with_addrs
> Aaron> +    (bfd, SEPARATE_INDEX_FILENAME, symfile_flags, &sap,
> Aaron> +     (parent->flags & (OBJF_REORDERED | OBJF_SHARED | OBJF_READNOW
> Aaron> +                      | OBJF_USERLOADED | OBJF_MAINLINE | OBJF_PSYMTABS_READ))
> Aaron> +                  | OBJF_NOT_FILENAME, parent, true);
> Aaron> +
> Aaron> +  objfile *result = parent->separate_debug_objfile;
> Aaron> +  init_objfile_sect_indices (result);
> Aaron> +
> Aaron> +  return;
>
> No need for this.

Ok.

> Aaron> +/* See symtab.h.  */
> Aaron> +
> Aaron> +void
> Aaron> +deferred_read_symbols (struct objfile *objfile, struct bfd *abfd)
> Aaron> +{
> Aaron> +  const char *obfd_filename;
> Aaron> +
> Aaron> +  obfd_filename = bfd_get_filename (abfd);
> Aaron> +  gdb_bfd_ref_ptr temp (gdb_bfd_open (obfd_filename, gnutarget));
> Aaron> +
> Aaron> +  if (temp == nullptr)
> Aaron> +    error (_("Can't open %s to read symbols."), obfd_filename);
> Aaron> +
> Aaron> +  if (!bfd_check_format (temp.get (), bfd_object))
> Aaron> +    error (_("Can't read symbols from %s: %s."), obfd_filename,
> Aaron> +           bfd_errmsg (bfd_get_error ()));
> Aaron> +
> Aaron> +  objfile->obfd = std::move (temp);
> Aaron> +
> Aaron> +  /* Nuke all the state that we will re-read.  */
> Aaron> +  objfile->registry_fields.clear_registry ();
> Aaron> +  objfile->sections = NULL;
> Aaron> +  objfile->sect_index_bss = -1;
> Aaron> +  objfile->sect_index_data = -1;
> Aaron> +  objfile->sect_index_rodata = -1;
> Aaron> +  objfile->sect_index_text = -1;
> Aaron> +  objfile->compunit_symtabs = NULL;
> Aaron> +  objfile->template_symbols = NULL;
> Aaron> +
> Aaron> +  objfile_set_sym_fns (objfile, find_sym_fns (objfile->obfd.get ()));
> Aaron> +  build_objfile_section_table (objfile);
> Aaron> +  (*objfile->sf->sym_init) (objfile);
> Aaron> +  init_objfile_sect_indices (objfile);
> Aaron> +
> Aaron> +  objfile->section_offsets.resize (gdb_bfd_count_sections
> Aaron> +                                     (objfile->obfd.get ()));
> Aaron> +  read_symbols (objfile, 0);
> Aaron> +
> Aaron> +  objfile->mtime = bfd_get_mtime (objfile->obfd.get ());
> Aaron> +  objfile->original_name
> Aaron> +    = obstack_strdup (&objfile->objfile_obstack, obfd_filename);
>
> So, gdb already does this kind of thing in reread_symbols, and it's been
> a source of bugs.  I wonder if it's possible to just create a new,
> non-stub objfile here and splice it in; removing the stub one.

Ok we could do something like std::swap so that we create the new objfile
the normal way while still using the pointer to the stub (which functions
down the callstack will still be using when deferred_read_symbols is
called).

> Alternatively, I also don't really understand why this is needed here.
> It seems like the reading of the symbols can be done purely in the DWARF
> reader.
>
> The idea of the "quick" API is to isolate the core gdb from whatever the
> readers do under the hood.  It seems to be that the stub objfile can be
> created with the index, and then when the full data is needed, it can
> just be read in without involving or notifying any other part of gdb.
> This is what's currently done with section data after all, just in this
> case it's being downloaded from some server.

I can try to incorporate this, but I think there still needs to be a step where
we remove everything in the stub related to its original bfd and update it
with a bfd of the debuginfo. Otherwise it will be in an inconsistent state
with the wrong bfd, per_bfd, etc.

> One other thing to consider is the sharing of indices.  The DWARF reader
> tries pretty hard to share data across inferiors.  However I think this
> new code will download the index separately for each inferior.  The full
> DWARF will be shared again, due to the BFD cache.
>
> The way this works is that the DWARF reader puts all the shareable data
> into an object that is attached to the BFD's registry.  Then when a new
> objfile is initialized, it checks the BFD's registry to see if there's
> already DWARF info attached.  If so, it's reused.
>
> The index code could maybe attach its own object to the parent BFD to
> make this sharing work.

Debuginfod caches section downloads but the contents will have to be
reread, I'll see if the read contents can be shared.

Aaron
  

Patch

diff --git a/gdb/dwarf2/frame.c b/gdb/dwarf2/frame.c
index a561aaf3100..f6d227d8b36 100644
--- a/gdb/dwarf2/frame.c
+++ b/gdb/dwarf2/frame.c
@@ -1609,6 +1609,19 @@  set_comp_unit (struct objfile *objfile, struct comp_unit *unit)
   return dwarf2_frame_bfd_data.set (abfd, unit);
 }
 
+/* See frame.h.  */
+
+void
+clear_comp_unit (struct objfile *objfile)
+{
+  bfd *abfd = objfile->obfd.get ();
+
+  if (gdb_bfd_requires_relocations (abfd))
+    dwarf2_frame_objfile_data.clear (objfile);
+  else
+    dwarf2_frame_bfd_data.clear (abfd);
+}
+
 /* Find the FDE for *PC.  Return a pointer to the FDE, and store the
    initial location associated with it into *PC.  */
 
diff --git a/gdb/dwarf2/frame.h b/gdb/dwarf2/frame.h
index 5643e557513..1851a14b67e 100644
--- a/gdb/dwarf2/frame.h
+++ b/gdb/dwarf2/frame.h
@@ -238,6 +238,10 @@  void dwarf2_append_unwinders (struct gdbarch *gdbarch);
 extern const struct frame_base *
   dwarf2_frame_base_sniffer (frame_info_ptr this_frame);
 
+/* Delete OBJFILEs comp_unit.  */
+
+extern void clear_comp_unit (struct objfile * objfile);
+
 /* Compute the DWARF CFA for a frame.  */
 
 CORE_ADDR dwarf2_frame_cfa (frame_info_ptr this_frame);
diff --git a/gdb/dwarf2/index-cache.c b/gdb/dwarf2/index-cache.c
index 79ab706ee9d..bbafcd321b2 100644
--- a/gdb/dwarf2/index-cache.c
+++ b/gdb/dwarf2/index-cache.c
@@ -216,6 +216,33 @@  index_cache::lookup_gdb_index (const bfd_build_id *build_id,
   return {};
 }
 
+/* See index-cache.h.  */
+
+gdb::array_view<const gdb_byte>
+index_cache::lookup_gdb_index_debuginfod (const char *index_path,
+					  std::unique_ptr<index_cache_resource> *resource)
+{
+  try
+    {
+      /* Try to map that file.  */
+      index_cache_resource_mmap *mmap_resource
+	= new index_cache_resource_mmap (index_path);
+
+      /* Hand the resource to the caller.  */
+      resource->reset (mmap_resource);
+
+      return gdb::array_view<const gdb_byte>
+	  ((const gdb_byte *) mmap_resource->mapping.get (),
+	   mmap_resource->mapping.size ());
+    }
+  catch (const gdb_exception_error &except)
+    {
+      warning (_("Unable to read %s: %s"), index_path, except.what ());
+    }
+
+  return {};
+}
+
 #else /* !HAVE_SYS_MMAN_H */
 
 /* See dwarf-index-cache.h.  This is a no-op on unsupported systems.  */
@@ -227,6 +254,12 @@  index_cache::lookup_gdb_index (const bfd_build_id *build_id,
   return {};
 }
 
+gdb::array_view<const gdb_byte>
+index_cache::lookup_gdb_index_debuginfod (const char *index_path,
+					  std::unique_ptr<index_cache_resource> *resource)
+{
+  return {};
+}
 #endif
 
 /* See dwarf-index-cache.h.  */
diff --git a/gdb/dwarf2/index-cache.h b/gdb/dwarf2/index-cache.h
index 1efff17049f..e400afd5123 100644
--- a/gdb/dwarf2/index-cache.h
+++ b/gdb/dwarf2/index-cache.h
@@ -67,6 +67,19 @@  class index_cache
   lookup_gdb_index (const bfd_build_id *build_id,
 		    std::unique_ptr<index_cache_resource> *resource);
 
+  /* Look for an index file located at INDEX_PATH in the debuginfod cache.
+     Unlike lookup_gdb_index, this function does not exit early if the
+     index cache has not been enabled.
+
+     If found, return the contents as an array_view and store the underlying
+     resources (allocated memory, mapped file, etc) in RESOURCE.  The returned
+     array_view is valid as long as RESOURCE is not destroyed.
+
+     If no matching index file is found, return an empty array view.  */
+  gdb::array_view<const gdb_byte>
+  lookup_gdb_index_debuginfod (const char *index_path,
+			       std::unique_ptr<index_cache_resource> *resource);
+
   /* Return the number of cache hits.  */
   unsigned int n_hits () const
   { return m_n_hits; }
diff --git a/gdb/dwarf2/public.h b/gdb/dwarf2/public.h
index 0e74857eb1a..4a44cdbc223 100644
--- a/gdb/dwarf2/public.h
+++ b/gdb/dwarf2/public.h
@@ -40,4 +40,11 @@  extern void dwarf2_initialize_objfile (struct objfile *objfile);
 
 extern void dwarf2_build_frame_info (struct objfile *);
 
+/* Query debuginfod for the .gdb_index associated with OBJFILE.  If
+   successful, create an objfile to hold the .gdb_index information
+   and act as a placeholder until the full debuginfo needs to be
+   downloaded.  */
+
+extern bool dwarf2_has_separate_index (struct objfile *);
+
 #endif /* DWARF2_PUBLIC_H */
diff --git a/gdb/dwarf2/read-gdb-index.c b/gdb/dwarf2/read-gdb-index.c
index 1006386cb2d..f066d55bd1a 100644
--- a/gdb/dwarf2/read-gdb-index.c
+++ b/gdb/dwarf2/read-gdb-index.c
@@ -143,6 +143,9 @@  struct dwarf2_gdb_index : public dwarf2_base_index_functions
      int global,
      symbol_compare_ftype *ordered_compare) override;
 
+  /* Calls do_expand_symtabs_matching.  If this index was initialized
+     from a file downloaded from debuginfod and there is a match, attempt
+     to download debuginfo.  */
   bool expand_symtabs_matching
     (struct objfile *objfile,
      gdb::function_view<expand_symtabs_file_matcher_ftype> file_matcher,
@@ -152,6 +155,16 @@  struct dwarf2_gdb_index : public dwarf2_base_index_functions
      block_search_flags search_flags,
      domain_enum domain,
      enum search_domain kind) override;
+
+  bool do_expand_symtabs_matching
+    (struct objfile *objfile,
+     gdb::function_view<expand_symtabs_file_matcher_ftype> file_matcher,
+     const lookup_name_info *lookup_name,
+     gdb::function_view<expand_symtabs_symbol_matcher_ftype> symbol_matcher,
+     gdb::function_view<expand_symtabs_exp_notify_ftype> expansion_notify,
+     block_search_flags search_flags,
+     domain_enum domain,
+     enum search_domain kind);
 };
 
 /* This dumps minimal information about the index.
@@ -455,7 +468,7 @@  dw2_expand_marked_cus
 }
 
 bool
-dwarf2_gdb_index::expand_symtabs_matching
+dwarf2_gdb_index::do_expand_symtabs_matching
     (struct objfile *objfile,
      gdb::function_view<expand_symtabs_file_matcher_ftype> file_matcher,
      const lookup_name_info *lookup_name,
@@ -504,6 +517,44 @@  dwarf2_gdb_index::expand_symtabs_matching
   return result;
 }
 
+bool
+dwarf2_gdb_index::expand_symtabs_matching
+    (struct objfile *objfile,
+     gdb::function_view<expand_symtabs_file_matcher_ftype> file_matcher,
+     const lookup_name_info *lookup_name,
+     gdb::function_view<expand_symtabs_symbol_matcher_ftype> symbol_matcher,
+     gdb::function_view<expand_symtabs_exp_notify_ftype> expansion_notify,
+     block_search_flags search_flags,
+     domain_enum domain,
+     enum search_domain kind)
+{
+  if (objfile->flags & OBJF_READNEVER)
+    return false;
+
+  try
+    {
+      return do_expand_symtabs_matching (objfile, file_matcher, lookup_name,
+					 symbol_matcher, expansion_notify,
+					 search_flags, domain, kind);
+    }
+  catch (gdb_exception e)
+    {
+      if (!objfile->is_separate_index_stub ())
+	{
+	  exception_print (gdb_stderr, e);
+	  return false;
+	}
+
+      /* Objfile is a stub holding only index information.  Try to reinitialize
+	 objfile with the full debuginfo.  */
+      if (!read_full_dwarf_from_debuginfod (objfile))
+	return false;
+      return do_expand_symtabs_matching (objfile, file_matcher, lookup_name,
+					 symbol_matcher, expansion_notify,
+					 search_flags, domain, kind);
+    }
+}
+
 quick_symbol_functions_up
 mapped_gdb_index::make_quick_functions () const
 {
@@ -797,28 +848,32 @@  dwarf2_read_gdb_index
 
   /* If there is a .dwz file, read it so we can get its CU list as
      well.  */
-  dwz = dwarf2_get_dwz_file (per_bfd);
-  if (dwz != NULL)
+  if (get_gdb_index_contents_dwz != nullptr)
     {
       mapped_gdb_index dwz_map;
       const gdb_byte *dwz_types_ignore;
       offset_type dwz_types_elements_ignore;
+      dwz = dwarf2_get_dwz_file (per_bfd);
 
-      gdb::array_view<const gdb_byte> dwz_index_content
-	= get_gdb_index_contents_dwz (objfile, dwz);
-
-      if (dwz_index_content.empty ())
-	return 0;
-
-      if (!read_gdb_index_from_buffer (bfd_get_filename (dwz->dwz_bfd.get ()),
-				       1, dwz_index_content, &dwz_map,
-				       &dwz_list, &dwz_list_elements,
-				       &dwz_types_ignore,
-				       &dwz_types_elements_ignore))
+      if (dwz != nullptr)
 	{
-	  warning (_("could not read '.gdb_index' section from %s; skipping"),
-		   bfd_get_filename (dwz->dwz_bfd.get ()));
-	  return 0;
+	  gdb::array_view<const gdb_byte> dwz_index_content
+	    = get_gdb_index_contents_dwz (objfile, dwz);
+
+	  if (dwz_index_content.empty ())
+	    return 0;
+
+	  if (!read_gdb_index_from_buffer (bfd_get_filename
+					     (dwz->dwz_bfd.get ()),
+					   1, dwz_index_content, &dwz_map,
+					   &dwz_list, &dwz_list_elements,
+					   &dwz_types_ignore,
+					   &dwz_types_elements_ignore))
+	    {
+	      warning (_("could not read '.gdb_index' section from %s; skipping"),
+		       bfd_get_filename (dwz->dwz_bfd.get ()));
+		return 0;
+	    }
 	}
     }
 
diff --git a/gdb/dwarf2/read.c b/gdb/dwarf2/read.c
index 69b2310be1f..12b0dbd1ee8 100644
--- a/gdb/dwarf2/read.c
+++ b/gdb/dwarf2/read.c
@@ -34,6 +34,7 @@ 
 #include "dwarf2/attribute.h"
 #include "dwarf2/comp-unit-head.h"
 #include "dwarf2/cu.h"
+#include "dwarf2/frame.h"
 #include "dwarf2/index-cache.h"
 #include "dwarf2/index-common.h"
 #include "dwarf2/leb.h"
@@ -95,6 +96,8 @@ 
 #include "split-name.h"
 #include "gdbsupport/parallel-for.h"
 #include "gdbsupport/thread-pool.h"
+#include "inferior.h"
+#include "debuginfod-support.h"
 
 /* When == 1, print basic high level tracing messages.
    When > 1, be more verbose.
@@ -3156,7 +3159,7 @@  dwarf2_base_index_functions::find_per_cu (dwarf2_per_bfd *per_bfd,
 }
 
 struct compunit_symtab *
-dwarf2_base_index_functions::find_pc_sect_compunit_symtab
+dwarf2_base_index_functions::do_find_pc_sect_compunit_symtab
      (struct objfile *objfile,
       struct bound_minimal_symbol msymbol,
       CORE_ADDR pc,
@@ -3187,6 +3190,39 @@  dwarf2_base_index_functions::find_pc_sect_compunit_symtab
   return result;
 }
 
+struct compunit_symtab *
+dwarf2_base_index_functions::find_pc_sect_compunit_symtab
+     (struct objfile *objfile,
+      struct bound_minimal_symbol msymbol,
+      CORE_ADDR pc,
+      struct obj_section *section,
+      int warn_if_readin)
+{
+  if (objfile->flags & OBJF_READNEVER)
+    return nullptr;
+
+  try
+    {
+      return do_find_pc_sect_compunit_symtab (objfile, msymbol, pc,
+					      section, warn_if_readin);
+    }
+  catch (gdb_exception e)
+    {
+      if (!objfile->is_separate_index_stub ())
+	{
+	  exception_print (gdb_stderr, e);
+	  return nullptr;
+	}
+
+      /* Objfile is a stub holding only index information.  Try to
+	 reinitialize objfile with the full debuginfo.  */
+      if (!read_full_dwarf_from_debuginfod (objfile))
+	return nullptr;
+      return do_find_pc_sect_compunit_symtab (objfile, msymbol, pc,
+					      section, warn_if_readin);
+    }
+}
+
 void
 dwarf2_base_index_functions::map_symbol_filenames
      (struct objfile *objfile,
@@ -3412,6 +3448,165 @@  dwarf2_initialize_objfile (struct objfile *objfile)
   objfile->qf.push_front (make_cooked_index_funcs ());
 }
 
+/* Query debuginfod for the .gdb_index matching OBJFILE's build-id.  Return the
+   contents if successful.  */
+
+static gdb::array_view<const gdb_byte>
+get_gdb_index_contents_from_debuginfod (objfile *objfile, dwarf2_per_bfd *per_bfd)
+{
+  const bfd_build_id *build_id = build_id_bfd_get (objfile->obfd.get ());
+  if (build_id == nullptr)
+    return {};
+
+  gdb::unique_xmalloc_ptr<char> index_path;
+  scoped_fd fd = debuginfod_section_query (build_id->data, build_id->size,
+					   bfd_get_filename
+					     (objfile->obfd.get ()),
+					   ".gdb_index",
+					   &index_path);
+  if (fd.get () < 0)
+    return {};
+
+  return global_index_cache.lookup_gdb_index_debuginfod
+    (index_path.get (), &per_bfd->index_cache_res);
+}
+
+/* See read.h.  */
+
+bool
+read_full_dwarf_from_debuginfod (struct objfile *objfile)
+{
+  gdb_assert (objfile->is_separate_index_stub ());
+
+  const struct bfd_build_id *build_id = build_id_bfd_get (objfile->obfd.get ());
+  if (build_id == nullptr)
+    return false;
+
+  const char *filename = bfd_get_filename (objfile->obfd.get ());
+  gdb_bfd_ref_ptr debug_bfd;
+  gdb::unique_xmalloc_ptr<char> symfile_path;
+
+  scoped_fd fd (debuginfod_debuginfo_query (build_id->data,
+					    build_id->size,
+					    filename,
+					    &symfile_path));
+
+  if (fd.get () < 0)
+    return false;
+
+  /* File successfully retrieved from server.  */
+  debug_bfd = symfile_bfd_open (symfile_path.get ());
+  if (debug_bfd == nullptr
+      || !build_id_verify (debug_bfd.get (), build_id->size, build_id->data))
+    {
+      warning (_("File \"%s\" from debuginfod cannot be opened as bfd"),
+	       filename);
+      return false;
+    }
+
+  /* Fill in objfile's missing information using the debuginfo.  This
+     may also download dwz, if available.  */
+  if (!objfile->reinit (debug_bfd.release ()))
+    return false;
+
+  struct objfile *parent = objfile->separate_debug_objfile_backlink;
+  gdb_assert (parent != nullptr);
+
+  /* Clear the parent objfile's frame comp_unit information so it can be
+     recalculated with DWARF.  */
+  clear_comp_unit (parent);
+
+  return true;
+}
+
+/* See public.h.  */
+
+bool
+dwarf2_has_separate_index (struct objfile *objfile)
+{
+  if (objfile->flags & OBJF_MAINLINE
+      || objfile->separate_debug_objfile_backlink != nullptr)
+    return 0;
+  if (objfile->is_separate_index_stub ())
+    return 1;
+  if (!IS_DIR_SEPARATOR (*objfile_filename (objfile)))
+    return 0;
+
+  gdb::unique_xmalloc_ptr<char> index_path;
+  const bfd_build_id *build_id = build_id_bfd_get (objfile->obfd.get ());
+  scoped_fd fd = debuginfod_section_query (build_id->data,
+					   build_id->size,
+					   bfd_get_filename
+					     (objfile->obfd.get ()),
+					   ".gdb_index",
+					   &index_path);
+  if (fd.get () >= 0)
+    {
+      /* We found a separate .gdb_index file so a separate debuginfo file
+	 should exist, but don't want to read it until we really have to.
+	 Create an objfile to own the index information and act as a
+	 placeholder for the debuginfo that we have the option of aquiring
+	 later.  */
+      gdb_bfd_ref_ptr abfd (gdb_bfd_open (objfile_filename (objfile), gnutarget));
+      if (abfd == nullptr)
+	return false;
+
+      dwarf2_per_bfd_objfile_data_key.clear (objfile);
+      dwarf2_objfile_data_key.clear (objfile);
+
+      /* Initialize the new objfile using a bfd opened from the parent's
+	 filename.  This objfile will be reinitialized if/when we download
+	 the full debuginfo.  */
+      symbol_file_add_from_index
+	(abfd, current_inferior ()->symfile_flags | SYMFILE_NO_READ, objfile);
+
+      dwarf2_per_bfd *per_bfd;
+      dwarf2_per_objfile *per_objfile = get_dwarf2_per_objfile (objfile);
+
+      /* Perform a limited initialization based to dwarf2_has_info and
+	 dwarf2_initialize_objfile.  */
+      if (per_objfile == nullptr)
+	{
+	  per_bfd = dwarf2_per_bfd_objfile_data_key.get (objfile);
+	  if (per_bfd == nullptr)
+	    {
+	      per_bfd = new dwarf2_per_bfd (objfile->obfd.get (), nullptr, false);
+	      dwarf2_per_bfd_objfile_data_key.set (objfile, per_bfd);
+	    }
+	  per_objfile = dwarf2_objfile_data_key.emplace (objfile, objfile, per_bfd);
+	}
+
+      struct objfile *stub = objfile->separate_debug_objfile;
+      per_objfile = get_dwarf2_per_objfile (stub);
+      if (per_objfile == nullptr)
+	{
+	  per_bfd = dwarf2_per_bfd_objfile_data_key.get (stub);
+	  if (per_bfd == nullptr)
+	    {
+	      per_bfd = new dwarf2_per_bfd (stub->obfd.get (), nullptr, false);
+	      dwarf2_per_bfd_objfile_data_key.set (stub, per_bfd);
+	    }
+	  per_objfile = dwarf2_objfile_data_key.emplace (stub, stub, per_bfd);
+	}
+
+      /* Do not include a dwz index callback, otherwise we could download a dwz
+	 file. We don't want to do this until the full debuginfo is needed.  */
+      if (dwarf2_read_gdb_index (per_objfile,
+				 get_gdb_index_contents_from_debuginfod,
+				 nullptr))
+	{
+	  dwarf_read_debug_printf ("found .gdb_index from debuginfod");
+	  stub->qf.push_front (per_bfd->index_table->make_quick_functions ());
+	  return 1;
+	}
+
+      /* Unable to read the index.  */
+      stub->unlink ();
+    }
+
+  return 0;
+}
+
 
 
 /* Build a partial symbol table.  */
diff --git a/gdb/dwarf2/read.h b/gdb/dwarf2/read.h
index 0e6853ea2f4..4e6c35cba5e 100644
--- a/gdb/dwarf2/read.h
+++ b/gdb/dwarf2/read.h
@@ -866,6 +866,10 @@  struct dwarf2_base_index_functions : public quick_symbol_functions
      CORE_ADDR pc, struct obj_section *section, int warn_if_readin)
        override final;
 
+  struct compunit_symtab *do_find_pc_sect_compunit_symtab
+    (struct objfile *objfile, struct bound_minimal_symbol msymbol,
+     CORE_ADDR pc, struct obj_section *section, int warn_if_readin);
+
   struct compunit_symtab *find_compunit_symtab_by_address
     (struct objfile *objfile, CORE_ADDR address) override
   {
@@ -942,4 +946,9 @@  extern bool read_addrmap_from_aranges (dwarf2_per_objfile *per_objfile,
 				       dwarf2_section_info *section,
 				       addrmap *mutable_map);
 
+/* If OBJFILE is a stub holding only information from a .gdb_index, then attempt
+   to download the full debuginfo and reinitialize OBJFILE with it.  */
+
+extern bool read_full_dwarf_from_debuginfod (struct objfile *objfile);
+
 #endif /* DWARF2READ_H */
diff --git a/gdb/dwarf2/section.c b/gdb/dwarf2/section.c
index c9ef41893ee..ff8ef527c29 100644
--- a/gdb/dwarf2/section.c
+++ b/gdb/dwarf2/section.c
@@ -54,7 +54,9 @@  dwarf2_section_info::get_bfd_owner () const
       section = get_containing_section ();
       gdb_assert (!section->is_virtual);
     }
-  gdb_assert (section->s.section != nullptr);
+  if (section->s.section == nullptr)
+    throw_error (NOT_FOUND_ERROR,
+		 _("Can't find owner of DWARF section."));
   return section->s.section->owner;
 }
 
diff --git a/gdb/elfread.c b/gdb/elfread.c
index ca684aab57e..6c3e384eb8d 100644
--- a/gdb/elfread.c
+++ b/gdb/elfread.c
@@ -1229,7 +1229,7 @@  elf_symfile_read_dwarf2 (struct objfile *objfile,
 	  symbol_file_add_separate (debug_bfd, debugfile.c_str (),
 				    symfile_flags, objfile);
 	}
-      else
+      else if (!dwarf2_has_separate_index (objfile))
 	{
 	  has_dwarf2 = false;
 	  const struct bfd_build_id *build_id
diff --git a/gdb/objfiles.c b/gdb/objfiles.c
index ed29131d528..fa8ab4a02b6 100644
--- a/gdb/objfiles.c
+++ b/gdb/objfiles.c
@@ -52,6 +52,7 @@ 
 #include "gdb_bfd.h"
 #include "btrace.h"
 #include "gdbsupport/pathstuff.h"
+#include "symfile.h"
 
 #include <algorithm>
 #include <vector>
@@ -298,6 +299,23 @@  build_objfile_section_table (struct objfile *objfile)
 			   objfile, 1);
 }
 
+bool
+objfile::reinit (struct bfd *abfd)
+{
+  try
+  {
+    deferred_read_symbols (this, abfd);
+    return true;
+  }
+  catch (const gdb_exception &except)
+  {
+    exception_print (gdb_stderr, except);
+    this->flags |= OBJF_READNEVER;
+  }
+
+  return false;
+}
+
 /* Given a pointer to an initialized bfd (ABFD) and some flag bits,
    initialize the new objfile as best we can and link it into the list
    of all known objfiles.
diff --git a/gdb/objfiles.h b/gdb/objfiles.h
index 342aa09ac6a..55c9ad1f987 100644
--- a/gdb/objfiles.h
+++ b/gdb/objfiles.h
@@ -375,6 +375,8 @@  class separate_debug_iterator
   struct objfile *m_parent;
 };
 
+#define SEPARATE_INDEX_FILENAME "<separate index stub>"
+
 /* A range adapter wrapping separate_debug_iterator.  */
 
 typedef iterator_range<separate_debug_iterator> separate_debug_range;
@@ -497,6 +499,12 @@  struct objfile
   /* See quick_symbol_functions.  */
   struct symtab *find_last_source_symtab ();
 
+  /* Reinitialize this objfile using ABFD.  Return true if successful.
+     Objfile should have been originally initialized using a separate
+     index from ABFD.  Updates this objfile with ABFD's symbols and
+     section information.  */
+  bool reinit (struct bfd *abfd);
+
   /* See quick_symbol_functions.  */
   void forget_cached_source_info ();
 
@@ -609,6 +617,14 @@  struct objfile
     this->section_offsets[idx] = offset;
   }
 
+  /* Return true if this objfile holds .gdb_index data and represents
+     a debuginfo file whose download has been deferred.  */
+
+  bool is_separate_index_stub ()
+  {
+    return strcmp (original_name, SEPARATE_INDEX_FILENAME) == 0;
+  }
+
 private:
 
   /* Ensure that partial symbols have been read and return the "quick" (aka
diff --git a/gdb/symfile.c b/gdb/symfile.c
index 373f5592107..1e05dd6af6e 100644
--- a/gdb/symfile.c
+++ b/gdb/symfile.c
@@ -1035,7 +1035,8 @@  static struct objfile *
 symbol_file_add_with_addrs (const gdb_bfd_ref_ptr &abfd, const char *name,
 			    symfile_add_flags add_flags,
 			    section_addr_info *addrs,
-			    objfile_flags flags, struct objfile *parent)
+			    objfile_flags flags, struct objfile *parent,
+			    bool skip_syms = false)
 {
   struct objfile *objfile;
   const int from_tty = add_flags & SYMFILE_VERBOSE;
@@ -1073,6 +1074,15 @@  symbol_file_add_with_addrs (const gdb_bfd_ref_ptr &abfd, const char *name,
     flags |= OBJF_MAINLINE;
   objfile = objfile::make (abfd, name, flags, parent);
 
+  if (skip_syms)
+    {
+      /* objfile was initialized only using a separate index so don't
+	 try to read symbols yet.  */
+      objfile->section_offsets = parent->section_offsets;
+      gdb::observers::new_objfile.notify (objfile);
+      return objfile;
+    }
+
   /* We either created a new mapped symbol table, mapped an existing
      symbol table file which has not had initial symbol reading
      performed, or need to read an unmapped symbol table.  */
@@ -1134,6 +1144,27 @@  symbol_file_add_with_addrs (const gdb_bfd_ref_ptr &abfd, const char *name,
   return (objfile);
 }
 
+/* See symfile.h.  */
+
+void
+symbol_file_add_from_index (const gdb_bfd_ref_ptr &bfd,
+			    symfile_add_flags symfile_flags,
+			    struct objfile *parent)
+{
+  section_addr_info sap = build_section_addr_info_from_objfile (parent);
+
+  symbol_file_add_with_addrs
+    (bfd, SEPARATE_INDEX_FILENAME, symfile_flags, &sap,
+     (parent->flags & (OBJF_REORDERED | OBJF_SHARED | OBJF_READNOW
+		      | OBJF_USERLOADED | OBJF_MAINLINE | OBJF_PSYMTABS_READ))
+		  | OBJF_NOT_FILENAME, parent, true);
+
+  objfile *result = parent->separate_debug_objfile;
+  init_objfile_sect_indices (result);
+
+  return;
+}
+
 /* Add BFD as a separate debug file for OBJFILE.  For NAME description
    see the objfile constructor.  */
 
@@ -2440,6 +2471,49 @@  remove_symbol_file_command (const char *args, int from_tty)
   clear_symtab_users (0);
 }
 
+/* See symtab.h.  */
+
+void
+deferred_read_symbols (struct objfile *objfile, struct bfd *abfd)
+{
+  const char *obfd_filename;
+
+  obfd_filename = bfd_get_filename (abfd);
+  gdb_bfd_ref_ptr temp (gdb_bfd_open (obfd_filename, gnutarget));
+
+  if (temp == nullptr)
+    error (_("Can't open %s to read symbols."), obfd_filename);
+
+  if (!bfd_check_format (temp.get (), bfd_object))
+    error (_("Can't read symbols from %s: %s."), obfd_filename,
+	   bfd_errmsg (bfd_get_error ()));
+
+  objfile->obfd = std::move (temp);
+
+  /* Nuke all the state that we will re-read.  */
+  objfile->registry_fields.clear_registry ();
+  objfile->sections = NULL;
+  objfile->sect_index_bss = -1;
+  objfile->sect_index_data = -1;
+  objfile->sect_index_rodata = -1;
+  objfile->sect_index_text = -1;
+  objfile->compunit_symtabs = NULL;
+  objfile->template_symbols = NULL;
+
+  objfile_set_sym_fns (objfile, find_sym_fns (objfile->obfd.get ()));
+  build_objfile_section_table (objfile);
+  (*objfile->sf->sym_init) (objfile);
+  init_objfile_sect_indices (objfile);
+
+  objfile->section_offsets.resize (gdb_bfd_count_sections
+				     (objfile->obfd.get ()));
+  read_symbols (objfile, 0);
+
+  objfile->mtime = bfd_get_mtime (objfile->obfd.get ());
+  objfile->original_name
+    = obstack_strdup (&objfile->objfile_obstack, obfd_filename);
+}
+
 /* Re-read symbols if a symbol-file has changed.  */
 
 void
diff --git a/gdb/symfile.h b/gdb/symfile.h
index b433e2be31a..353cf0cab42 100644
--- a/gdb/symfile.h
+++ b/gdb/symfile.h
@@ -241,6 +241,13 @@  extern struct objfile *symbol_file_add_from_bfd (const gdb_bfd_ref_ptr &,
 extern void symbol_file_add_separate (const gdb_bfd_ref_ptr &, const char *,
 				      symfile_add_flags, struct objfile *);
 
+/* Initialize an objfile acting as a placeholder for separate debuginfo
+   that has not yet been opened or read.  Infer some of its properties
+   from PARENT.  */
+
+extern void symbol_file_add_from_index (const gdb_bfd_ref_ptr &,
+					symfile_add_flags, struct objfile *);
+
 /* Find separate debuginfo for OBJFILE (using .gnu_debuglink section).
    Returns pathname, or an empty string.
 
diff --git a/gdb/symtab.h b/gdb/symtab.h
index 17d2746fd48..304482f81dd 100644
--- a/gdb/symtab.h
+++ b/gdb/symtab.h
@@ -2206,6 +2206,11 @@  extern bool find_pc_line_pc_range (CORE_ADDR, CORE_ADDR *, CORE_ADDR *);
 
 extern void reread_symbols (int from_tty);
 
+/* Reread a separate debug OBJFILE that was originally initialized
+   using an index.  ABFD will be used as OBJFILE's new bfd.  */
+
+extern void deferred_read_symbols (struct objfile *objfile, struct bfd *abfd);
+
 /* Look up a type named NAME in STRUCT_DOMAIN in the current language.
    The type returned must not be opaque -- i.e., must have at least one field
    defined.  */