CTF as a fallback when no DWARF debug info is present
Commit Message
Hello,
This patch implements the CTF fallback behaviour in libabigail tools
when DWARF debug info is not present.
Please let me know your thoughts, they will be really appreciated.
Thanks in advanced,
guillermo
--
By default, `abidw', `abidiff', `abipkgdiff' and `kmidiff' tools use
debug information in `DWARF` format, if present, otherwise now
automatically the tools try to extract and build the binaries IR using
debug information in `CTF` format without use of `--ctf' option, if
present, finally, if neither is found, they use only `ELF` symbols to
extract, build, compare and report which of them were added or removed.
In case of neither DWARF nor CTF debug information is present
(`STATUS_DEBUG_INFO_NOT_FOUND') and `--ctf' option was not passed in
the command line, those tools use the symbol object build by DWARF
reader (default) to completed theirs task.
Tools don't allow comparing corpora built with between different debug
information i.e DWARF vs CTF, the first corpus built dictate the
expected debug information in the second one.
This work for libraries and Linux kernel. The `--ctf' option is
preserved to explicitly indicate to those tools that we want to use
CTF support.
* doc/manuals/abidiff.rst: Adjust usage tool information
to indicates fallback for CTF debug info when DWARF info
is not present.
* doc/manuals/abidw.rst: Likewise.
* doc/manuals/abipkgdiff.rst: Likewise.
* doc/manuals/kmidiff.rst: Likewise.
* src/abg-ctf-reader.cc (slurp_elf_info): Report status
when debug information is not present.
(read_corpus): Add code to locate `vmlinux.ctfa' just when
`main corpus' is being processed.
* src/abg-tools-utils.cc (maybe_load_vmlinux_dwarf_corpus):
Replace by a reference `origin' argument to notify whether
CTF reader needs to be executed.
* tests/data/test-diff-pkg-ctf/dirpkg-3-report-2.txt: Adjust
test case.
* tests/test-diff-pkg.cc (in_out_specs): Add `--ctf' option
in test case.
* tools/abidiff.cc (main): Add `origin' depending of command
line argument by default it is `corpus::DWARF_ORIGIN'. Add
`dwarf_corpus' to be used as default corpus (containing ELF
symbols), when DWARF nor CTF debug information was found.
* tools/abidw.cc: Likewise.
* tools/abipkgdiff.cc: Likewise.
Signed-off-by: Guillermo E. Martinez <guillermo.e.martinez@oracle.com>
---
doc/manuals/abidiff.rst | 15 +--
doc/manuals/abidw.rst | 15 ++-
doc/manuals/abipkgdiff.rst | 13 +-
doc/manuals/kmidiff.rst | 9 +-
src/abg-ctf-reader.cc | 33 +++--
src/abg-tools-utils.cc | 22 +++-
.../test-diff-pkg-ctf/dirpkg-3-report-2.txt | 16 +++
tests/test-diff-pkg.cc | 2 +-
tools/abidiff.cc | 82 ++++++++-----
tools/abidw.cc | 54 +++++---
tools/abipkgdiff.cc | 116 +++++++++++-------
11 files changed, 257 insertions(+), 120 deletions(-)
Comments
Hello Guillermo,
Thank you for this patch.
Please find my comments below.
"Guillermo E. Martinez via Libabigail" <libabigail@sourceware.org> a
écrit:
> By default, `abidw', `abidiff', `abipkgdiff' and `kmidiff' tools use
> debug information in `DWARF` format, if present, otherwise now
> automatically the tools try to extract and build the binaries IR using
> debug information in `CTF` format without use of `--ctf' option, if
> present, finally, if neither is found, they use only `ELF` symbols to
> extract, build, compare and report which of them were added or removed.
>
> In case of neither DWARF nor CTF debug information is present
> (`STATUS_DEBUG_INFO_NOT_FOUND') and `--ctf' option was not passed in
> the command line, those tools use the symbol object build by DWARF
> reader (default) to completed theirs task.
>
> Tools don't allow comparing corpora built with between different debug
> information i.e DWARF vs CTF, the first corpus built dictate the
> expected debug information in the second one.
>
> This work for libraries and Linux kernel. The `--ctf' option is
> preserved to explicitly indicate to those tools that we want to use
> CTF support.
>
> * doc/manuals/abidiff.rst: Adjust usage tool information
> to indicates fallback for CTF debug info when DWARF info
> is not present.
> * doc/manuals/abidw.rst: Likewise.
> * doc/manuals/abipkgdiff.rst: Likewise.
> * doc/manuals/kmidiff.rst: Likewise.
I have made some slight fixes to changes to these doc files. Nothing
major you'll see the changes in the updated version of the patch that I
am posting at the end of this message.
[...]
> * src/abg-ctf-reader.cc (slurp_elf_info): Report status
> when debug information is not present.
> (read_corpus): Add code to locate `vmlinux.ctfa' just when
> `main corpus' is being processed.
> * src/abg-tools-utils.cc (maybe_load_vmlinux_dwarf_corpus):
> Replace by a reference `origin' argument to notify whether
> CTF reader needs to be executed.
> * tests/data/test-diff-pkg-ctf/dirpkg-3-report-2.txt: Adjust
> test case.
> * tests/test-diff-pkg.cc (in_out_specs): Add `--ctf' option
> in test case.
> * tools/abidiff.cc (main): Add `origin' depending of command
> line argument by default it is `corpus::DWARF_ORIGIN'. Add
> `dwarf_corpus' to be used as default corpus (containing ELF
> symbols), when DWARF nor CTF debug information was found.
> * tools/abidw.cc: Likewise.
> * tools/abipkgdiff.cc: Likewise.
>
> Signed-off-by: Guillermo E. Martinez <guillermo.e.martinez@oracle.com>
> ---
> doc/manuals/abidiff.rst | 15 +--
> doc/manuals/abidw.rst | 15 ++-
> doc/manuals/abipkgdiff.rst | 13 +-
> doc/manuals/kmidiff.rst | 9 +-
> src/abg-ctf-reader.cc | 33 +++--
> src/abg-tools-utils.cc | 22 +++-
> .../test-diff-pkg-ctf/dirpkg-3-report-2.txt | 16 +++
> tests/test-diff-pkg.cc | 2 +-
> tools/abidiff.cc | 82 ++++++++-----
> tools/abidw.cc | 54 +++++---
> tools/abipkgdiff.cc | 116 +++++++++++-------
> 11 files changed, 257 insertions(+), 120 deletions(-)
>
> diff --git a/doc/manuals/abidiff.rst b/doc/manuals/abidiff.rst
> index 0c711d9e..caeceded 100644
> --- a/doc/manuals/abidiff.rst
> +++ b/doc/manuals/abidiff.rst
> @@ -12,11 +12,12 @@ This tool can also compare the textual representations of the ABI of
> two ELF binaries (as emitted by ``abidw``) or an ELF binary against a
> textual representation of another ELF binary.
>
> -For a comprehensive ABI change report that includes changes about
> -function and variable sub-types, the two input shared libraries must
> -be accompanied with their debug information in `DWARF`_ format.
> -Otherwise, only `ELF`_ symbols that were added or removed are
> -reported.
> +For a comprehensive ABI change report between two input shared
> +libraries that includes changes about function and variable sub-types,
> +``abidiff`` uses by default, debug information in `DWARF`_ format, if
> +present, otherwise it compares interfaces using debug information in
> +`CTF`_ format, if present, finally, if neither is found, it uses only
> +`ELF`_ symbols to report which of them were added or removed.
>
> .. include:: tools-use-libabigail.txt
>
> @@ -558,7 +559,7 @@ Options
>
> * ``--ctf``
>
> - When comparing binaries, extract ABI information from CTF debug
> + When comparing binaries, extract ABI information from `CTF`_ debug
> information, if present.
>
> * ``--stats``
> @@ -785,4 +786,4 @@ Usage examples
>
> .. _ELF: http://en.wikipedia.org/wiki/Executable_and_Linkable_Format
> .. _DWARF: http://www.dwarfstd.org
> -
> +.. _CTF: https://raw.githubusercontent.com/wiki/oracle/binutils-gdb/files/ctf-spec.pdf
> diff --git a/doc/manuals/abidw.rst b/doc/manuals/abidw.rst
> index a3055c7e..20948805 100644
> --- a/doc/manuals/abidw.rst
> +++ b/doc/manuals/abidw.rst
> @@ -8,8 +8,7 @@ representation of its ABI to standard output. The emitted
> representation format, named ``ABIXML``, includes all the globally
> defined functions and variables, along with a complete representation
> of their types. It also includes a representation of the globally
> -defined ELF symbols of the file. The input shared library must
> -contain associated debug information in `DWARF`_ format.
> +defined ELF symbols of the file.
>
> When given the ``--linux-tree`` option, this program can also handle a
> `Linux kernel`_ tree. That is, a directory tree that contains both
> @@ -19,8 +18,13 @@ interface between the kernel and its module, to standard output. In
> this case, we don't call it an ABI, but a KMI (Kernel Module
> Interface). The emitted KMI includes all the globally defined
> functions and variables, along with a complete representation of their
> -types. The input binaries must contain associated debug information
> -in `DWARF`_ format.
> +types.
> +
> +To generate either ABI or KMI representation, by default ``abidw``
> +uses debug information in `DWARF`_ format, if present, otherwise it
> +looks for debug information in `CTF`_ format, if present, finally, if
> +neither is found, it uses only `ELF`_ symbols to report which of them
> +were added or removed.
>
> .. include:: tools-use-libabigail.txt
>
> @@ -326,7 +330,7 @@ Options
>
> * ``--ctf``
>
> - Extract ABI information from CTF debug information, if present in
> + Extract ABI information from `CTF`_ debug information, if present in
> the given object.
>
> * ``--annotate``
> @@ -365,3 +369,4 @@ standard `here
> .. _DWARF: http://www.dwarfstd.org
> .. _GNU: http://www.gnu.org
> .. _Linux Kernel: https://kernel.org/
> +.. _CTF: https://raw.githubusercontent.com/wiki/oracle/binutils-gdb/files/ctf-spec.pdf
> diff --git a/doc/manuals/abipkgdiff.rst b/doc/manuals/abipkgdiff.rst
> index 9114775a..771bb034 100644
> --- a/doc/manuals/abipkgdiff.rst
> +++ b/doc/manuals/abipkgdiff.rst
> @@ -13,12 +13,18 @@ binaries.
> For a comprehensive ABI change report that includes changes about
> function and variable sub-types, the two input packages must be
> accompanied with their debug information packages that contain debug
> -information either in `DWARF`_ or in `CTF` formats. Please note
> +information either in `DWARF`_ or in `CTF`_ formats. Please note
> however that some packages contain binaries that embed the debug
> information directly in a section of said binaries. In those cases,
> obviously, no separate debug information package is needed as the tool
> will find the debug information inside the binaries.
>
> +By default, ``abipkgdiff`` uses debug information in `DWARF`_ format,
> +if present, otherwise it compares binaries interfaces using debug
> +information in `CTF`_ format, if present, finally, if neither is
> +found, it uses only `ELF`_ symbols to report which of them were added
> +or removed.
> +
> .. include:: tools-use-libabigail.txt
>
> .. _abipkgdiff_invocation_label:
> @@ -525,8 +531,8 @@ Options
>
> * ``--ctf``
>
> - This is used to compare packages with CTF debug information, if
> - present.
> + This is used to compare packages with `CTF`_ debug information,
> + if present.
>
> .. _abipkgdiff_return_value_label:
>
> @@ -546,4 +552,5 @@ In the later case, the value of the exit code is the same as for the
> .. _Deb: https://en.wikipedia.org/wiki/Deb_%28file_format%29
> .. _tar: https://en.wikipedia.org/wiki/Tar_%28computing%29
> .. _DWARF: http://www.dwarfstd.org
> +.. _CTF: https://raw.githubusercontent.com/wiki/oracle/binutils-gdb/files/ctf-spec.pdf
> .. _Development Package: https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Devel_Packages
> diff --git a/doc/manuals/kmidiff.rst b/doc/manuals/kmidiff.rst
> index 53010189..a27d2456 100644
> --- a/doc/manuals/kmidiff.rst
> +++ b/doc/manuals/kmidiff.rst
> @@ -74,6 +74,11 @@ functions and variables) between the Kernel and its modules. In
> practice, though, some users might want to compare a subset of the
> those interfaces.
>
> +By default, ``kmidiff`` uses debug information in `DWARF`_ format,
> +if present, otherwise it compares interfaces using debug information
> +in `CTF`_ format, if present, finally, if neither is found, it uses
> +only `ELF`_ symbols to report which were added or removed.
> +
> Users can then define a "white list" of the interfaces to compare.
> Such a white list is a just a file in the "INI" format that looks
> like: ::
> @@ -174,7 +179,7 @@ Options
>
> * ``--ctf``
>
> - Extract ABI information from CTF debug information, if present in
> + Extract ABI information from `CTF`_ debug information, if present in
> the Kernel and Modules.
>
> * ``--impacted-interfaces | -i``
> @@ -242,3 +247,5 @@ Options
> .. _ELF: http://en.wikipedia.org/wiki/Executable_and_Linkable_Format
> .. _ksymtab: http://en.wikipedia.org/wiki/Executable_and_Linkable_Format
> .. _Linux Kernel: https://kernel.org
> +.. _DWARF: http://www.dwarfstd.org
> +.. _CTF: https://raw.githubusercontent.com/wiki/oracle/binutils-gdb/files/ctf-spec.pdf
> diff --git a/src/abg-ctf-reader.cc b/src/abg-ctf-reader.cc
> index 9148a646..b3cde365 100644
> --- a/src/abg-ctf-reader.cc
> +++ b/src/abg-ctf-reader.cc
> @@ -45,6 +45,7 @@ namespace ctf_reader
> using std::dynamic_pointer_cast;
> using abigail::tools_utils::dir_name;
> using abigail::tools_utils::file_exists;
> +using abigail::tools_utils::base_name;
>
> class read_context
> {
> @@ -1545,7 +1546,12 @@ slurp_elf_info(read_context *ctxt,
> corp->set_architecture_name(elf_helpers::e_machine_to_string(ehdr->e_machine));
>
> find_alt_debuginfo(ctxt, &ctf_dbg_scn);
> - ABG_ASSERT(ctxt->symtab);
> + if (!ctxt->symtab)
> + {
> + status |= elf_reader::STATUS_NO_SYMBOLS_FOUND;
> + return;
> + }
> +
> corp->set_symtab(ctxt->symtab);
>
> if ((corp->get_origin() & corpus::LINUX_KERNEL_BINARY_ORIGIN)
> @@ -1564,7 +1570,8 @@ slurp_elf_info(read_context *ctxt,
> ctf_scn = ctf_dbg_scn;
> else
> {
> - status |= elf_reader::STATUS_DEBUG_INFO_NOT_FOUND;
> + status |= (elf_reader::STATUS_OK |
> + elf_reader::STATUS_DEBUG_INFO_NOT_FOUND);
> return;
> }
> }
> @@ -1676,15 +1683,15 @@ read_corpus(read_context *ctxt, elf_reader::status &status)
> origin |= corpus::LINUX_KERNEL_BINARY_ORIGIN;
> corp->set_origin(origin);
>
> - if (ctxt->cur_corpus_group_)
> - ctxt->cur_corpus_group_->add_corpus(ctxt->cur_corpus_);
> -
> slurp_elf_info(ctxt, corp, status);
> - if (!is_linux_kernel
> - && ((status & elf_reader::STATUS_DEBUG_INFO_NOT_FOUND) |
> - (status & elf_reader::STATUS_NO_SYMBOLS_FOUND)))
> + if ((status & elf_reader::STATUS_NO_SYMBOLS_FOUND) ||
> + (!is_linux_kernel &&
> + (status & elf_reader::STATUS_DEBUG_INFO_NOT_FOUND)))
> return corp;
>
> + if (ctxt->cur_corpus_group_)
> + ctxt->cur_corpus_group_->add_corpus(ctxt->cur_corpus_);
> +
> // Set the set of exported declaration that are defined.
> ctxt->exported_decls_builder
> (ctxt->cur_corpus_->get_exported_decls_builder().get());
> @@ -1695,9 +1702,13 @@ read_corpus(read_context *ctxt, elf_reader::status &status)
> {
> if (ctxt->ctfa == NULL)
> {
> - std::string ctfa_filename;
> - if (find_ctfa_file(ctxt, ctfa_filename))
> - ctxt->ctfa = ctf_arc_open(ctfa_filename.c_str(), &errp);
> + std::string filename;
> + base_name(ctxt->filename, filename);
> +
> + // locate vmlinux.ctfa only when reader is processing
> + // vmlinux file, i.e the main corpus in the group.
> + if (filename == "vmlinux" && find_ctfa_file(ctxt, filename))
> + ctxt->ctfa = ctf_arc_open(filename.c_str(), &errp);
> }
> }
> else
> diff --git a/src/abg-tools-utils.cc b/src/abg-tools-utils.cc
> index fe9ebc72..7d00f726 100644
> --- a/src/abg-tools-utils.cc
> +++ b/src/abg-tools-utils.cc
> @@ -2493,6 +2493,11 @@ get_binary_paths_from_kernel_dist(const string& dist_root,
> /// made of vmlinux kernel file and the linux kernel modules found
> /// under @p root directory and under its sub-directories, recursively.
> ///
> +/// If the vmlinux file doens't have DWARF info, it looks for
> +/// vmlinux.ctfa, if it's present, it assumes that kernel was build
> +/// with CTF support, then it updates @ref origin, given chance to
> +/// CTF reader to build the IR for kernel build directory.
> +///
> /// @param origin the debug type information in vmlinux kernel and
> /// the linux kernel modules to be used to build the corpora @p group.
> ///
> @@ -2526,7 +2531,7 @@ get_binary_paths_from_kernel_dist(const string& dist_root,
> ///
> /// @param env the environment to create the corpus_group in.
> static void
> -maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
> +maybe_load_vmlinux_dwarf_corpus(corpus::origin &origin,
> corpus_group_sptr& group,
> const string& vmlinux,
> vector<string>& modules,
> @@ -2578,6 +2583,21 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
> << " reading DONE:"
> << t << "\n";
>
> + if (status & elf_reader::STATUS_DEBUG_INFO_NOT_FOUND)
> + {
> + // vmlinux doesn't have DWARF debug info, so it might contain
> + // CTF debug info, this means vmlinux.ctfa is under kernel
> + // build directory.
> + string ctfa_file = root + "/vmlinux.ctfa";
> + if (file_exists(ctfa_file))
> + {
> + // OK. Likely CTF could build a better IR, then let's
> + // notify the caller to try with CTF reader.
> + origin = corpus::CTF_ORIGIN;
> + return;
> + }
> + }
> +
Hmmh, I think this change is done at logical level that is too "low",
compared to the level at which I believe this decision should be made.
Namely, maybe_load_vmlinux_dwarf_corpus is meant to try to load DWARF
debug information. It should not deal with CTF debug information
(looking for CTF archive files etc). Rather, I think that if the
loading of DWARF failed, that function should tell its caller about it.
It would thus be up to the caller to decide what to do with that
information. For instance, the caller would then be able to try and load
CTF debug information instead.
So I would rather modify maybe_load_vmlinux_dwarf_corpus to make it
return the status of the loading, so that the caller can know if the
loading succeeded or not.
So here is the change I would propose for the
maybe_load_vmlinux_dwarf_corpus and maybe_load_vmlinux_ctf_corpus
functions instead (it's part of the whole patch that I have posted at
the end of this message):
@@ -2525,8 +2542,12 @@ get_binary_paths_from_kernel_dist(const string& dist_root,
/// @param t time to trace time spent in each step.
///
/// @param env the environment to create the corpus_group in.
-static void
-maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
+///
+/// @return the status of the loading. If it's
+/// abigail::elf_reader::STATUS_UNKNOWN, then it means nothing was
+/// done, meaning the function got out early.
+static abigail::elf_reader::status
+maybe_load_vmlinux_dwarf_corpus(corpus::origin& origin,
corpus_group_sptr& group,
const string& vmlinux,
vector<string>& modules,
@@ -2539,10 +2560,11 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
timer& t,
environment_sptr& env)
{
+ abigail::elf_reader::status status = abigail::elf_reader::STATUS_UNKNOWN;
+
if (!(origin & corpus::DWARF_ORIGIN))
- return;
+ return status;
- abigail::elf_reader::status status = abigail::elf_reader::STATUS_OK;
dwarf_reader::read_context_sptr ctxt;
ctxt =
dwarf_reader::create_read_context(vmlinux, di_roots, env.get(),
@@ -2569,6 +2591,7 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
<< vmlinux << "' ...\n" << std::flush;
// Read the vmlinux corpus and add it to the group.
+ status = abigail::elf_reader::STATUS_OK;
t.start();
read_and_add_corpus_to_group_from_elf(*ctxt, *group, status);
t.stop();
@@ -2579,7 +2602,7 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
<< t << "\n";
if (group->is_empty())
- return;
+ return status;
// Now add the corpora of the modules to the corpus group.
int total_nb_modules = modules.size();
@@ -2614,6 +2637,7 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
<< "' reading DONE: "
<< t << "\n";
}
+ return status;
}
/// If the @ref origin is CTF_ORIGIN it build a @ref corpus_group
@@ -2642,8 +2666,12 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
/// @param t time to trace time spent in each step.
///
/// @param env the environment to create the corpus_group in.
+///
+/// @return the status of the loading. If it's
+/// abigail::elf_reader::STATUS_UNKNOWN, then it means nothing was
+/// done, meaning the function got out early.
#ifdef WITH_CTF
-static void
+static abigail::elf_reader::status
maybe_load_vmlinux_ctf_corpus(corpus::origin origin,
corpus_group_sptr& group,
const string& vmlinux,
@@ -2654,10 +2682,11 @@ maybe_load_vmlinux_ctf_corpus(corpus::origin origin,
timer& t,
environment_sptr& env)
{
+ abigail::elf_reader::status status = abigail::elf_reader::STATUS_UNKNOWN;
+
if (!(origin & corpus::CTF_ORIGIN))
- return;
+ return status;
- abigail::elf_reader::status status = abigail::elf_reader::STATUS_OK;
ctf_reader::read_context_sptr ctxt;
ctxt = ctf_reader::create_read_context(vmlinux, di_roots, env.get());
group.reset(new corpus_group(env.get(), root));
@@ -2668,6 +2697,7 @@ maybe_load_vmlinux_ctf_corpus(corpus::origin origin,
<< vmlinux << "' ...\n" << std::flush;
// Read the vmlinux corpus and add it to the group.
+ status = abigail::elf_reader::STATUS_OK;
t.start();
read_and_add_corpus_to_group_from_elf(ctxt.get(), *group, status);
t.stop();
@@ -2678,7 +2708,7 @@ maybe_load_vmlinux_ctf_corpus(corpus::origin origin,
<< t << "\n";
if (group->is_empty())
- return;
+ return status;
// Now add the corpora of the modules to the corpus group.
int total_nb_modules = modules.size();
@@ -2707,6 +2737,7 @@ maybe_load_vmlinux_ctf_corpus(corpus::origin origin,
<< "' reading DONE: "
<< t << "\n";
}
+ return status;
}
#endif
I have also introduced a new function called
tools_utils::dir_contains_ctf_archive to look for a file that ends with
".ctfa". This abstracts away the search for "vmlinux.ctfa" as I wasn't
sure if those archives could exist for normal (non-kernel) binaries as
well:
diff --git a/src/abg-tools-utils.cc b/src/abg-tools-utils.cc
index fe9ebc72..a23f08ea 100644
--- a/src/abg-tools-utils.cc
+++ b/src/abg-tools-utils.cc
@@ -1694,6 +1694,23 @@ file_is_kernel_debuginfo_package(const string& file_name, file_type file_type)
return result;
}
+/// Test if a directory contains a CTF archive.
+///
+/// @param directory the directory to consider.
+///
+/// @param archive_prefix the prefix of the archive file.
+///
+/// @return true iff @p directory contains a CTF archive file.
+bool
+dir_contains_ctf_archive(const string& directory,
+ const string& archive_prefix)
+{
+ string ctf_archive = directory + "/" + archive_prefix + ".ctfa";
+ if (file_exists(ctf_archive))
+ return true;
+ return false;
+}
+
Then, now that the caller of these functions can know if loading the
vmlinux binary succeeded or not, we can update that caller, which is
build_corpus_group_from_kernel_dist_under:
@@ -2786,14 +2817,23 @@ build_corpus_group_from_kernel_dist_under(const string& root,
vector<char**> di_roots;
di_roots.push_back(&di_root_ptr);
- maybe_load_vmlinux_dwarf_corpus(origin, group, vmlinux,
- modules, root, di_roots,
- suppr_paths, kabi_wl_paths,
- supprs, verbose, t, env);
+ abigail::elf_reader::status status =
+ maybe_load_vmlinux_dwarf_corpus(origin, group, vmlinux,
+ modules, root, di_roots,
+ suppr_paths, kabi_wl_paths,
+ supprs, verbose, t, env);
#ifdef WITH_CTF
+ string vmlinux_basename;
+ base_name(vmlinux, vmlinux_basename);
+ if (origin == corpus::DWARF_ORIGIN
+ && (status & abigail::elf_reader::STATUS_DEBUG_INFO_NOT_FOUND)
+ && dir_contains_ctf_archive(root, vmlinux_basename))
+ // Let's try to load the binary using the CTF archive!
+ origin |= corpus::CTF_ORIGIN;
+
maybe_load_vmlinux_ctf_corpus(origin, group, vmlinux,
- modules, root, di_roots,
- verbose, t, env);
+ modules, root, di_roots,
+ verbose, t, env);
#endif
}
If you feel like we shouldn't test if the directory 'root' contains a
ctf archive before trying to load the CTF archive, then please remove
the call to dir_contains_ctf_archive above. I just meant to show the
general idea of where the decision should be made to try to load the ctf
debug information, in my opinion.
The general idea, however, is to make the reader of
build_corpus_group_from_kernel_dist_under understand what is going on
(that is, we try loading vmlinux with DWARF, and if it fails, then we
try CTF) without having to go dig deep into how
maybe_load_vmlinux_ctf_corpus works.
Similarly, I amended the changes to the tools (abi{diff,dw,pkgdiff}) to
make them follow the same path.
[...]
> diff --git a/tools/abidiff.cc b/tools/abidiff.cc
> index e0bb35ac..815c68df 100644
> --- a/tools/abidiff.cc
> +++ b/tools/abidiff.cc
> @@ -1118,6 +1118,13 @@ main(int argc, char* argv[])
> return 0;
> }
>
> + corpus::origin first_input_origin;
> + corpus::origin origin =
> +#ifdef WITH_CTF
> + opts.use_ctf ? corpus::CTF_ORIGIN :
> +#endif
> + corpus::DWARF_ORIGIN;
> +
I removed these changes above. This is because we don't need to perform
so many changes. We can just let the code as it was and add new code to
just one place to test if loading DWARF failed. If it failed then we
try to load CTF.
See below.
> prepare_di_root_paths(opts);
>
> if (!maybe_check_suppression_files(opts))
> @@ -1150,7 +1157,7 @@ main(int argc, char* argv[])
> abigail::elf_reader::status c1_status =
> abigail::elf_reader::STATUS_OK,
> c2_status = abigail::elf_reader::STATUS_OK;
> - corpus_sptr c1, c2;
> + corpus_sptr c1, c2, dwarf_corpus;
Likewise, removed.
> corpus_group_sptr g1, g2;
> bool files_suppressed = false;
>
> @@ -1180,20 +1187,9 @@ main(int argc, char* argv[])
> case abigail::tools_utils::FILE_TYPE_ELF: // fall through
> case abigail::tools_utils::FILE_TYPE_AR:
> {
> -#ifdef WITH_CTF
> - if (opts.use_ctf)
> - {
> - abigail::ctf_reader::read_context_sptr ctxt
> - = abigail::ctf_reader::create_read_context(opts.file1,
> - opts.prepared_di_root_paths1,
> - env.get());
> - ABG_ASSERT(ctxt);
> - c1 = abigail::ctf_reader::read_corpus(ctxt.get(),
> - c1_status);
> - }
> - else
> -#endif
I left this code in. So no change here either.
> + if (origin & corpus::DWARF_ORIGIN)
> {
> + first_input_origin = corpus::DWARF_ORIGIN;
I remove this change as well. So, still no change.
> abigail::dwarf_reader::read_context_sptr ctxt =
> abigail::dwarf_reader::create_read_context
> (opts.file1, opts.prepared_di_root_paths1,
> @@ -1205,6 +1201,7 @@ main(int argc, char* argv[])
> set_suppressions(*ctxt, opts);
> abigail::dwarf_reader::set_do_log(*ctxt, opts.do_log);
> c1 = abigail::dwarf_reader::read_corpus_from_elf(*ctxt, c1_status);
> + dwarf_corpus = c1;
Likewise.
But then, it's here that we are going to inspect c1_status to see if
loading DWARF failed. If it failed, then we'll try to load CTF. So,
here is the change I am adding to the process of loading the corpus c1:
@@ -1205,6 +1208,36 @@ main(int argc, char* argv[])
set_suppressions(*ctxt, opts);
abigail::dwarf_reader::set_do_log(*ctxt, opts.do_log);
c1 = abigail::dwarf_reader::read_corpus_from_elf(*ctxt, c1_status);
+
+#ifdef WITH_CTF
+ if (// We were not instructed to use CTF ...
+ !opts.use_ctf
+ // ... and yet, no debug info was found ...
+ && (c1_status & STATUS_DEBUG_INFO_NOT_FOUND)
+ // ... but we found ELF symbols ...
+ && !(c1_status & STATUS_NO_SYMBOLS_FOUND))
+ {
+ string basenam, dirnam;
+ base_name(opts.file1, basenam);
+ dir_name(opts.file1, dirnam);
+ // ... the input file seems to contain CTF
+ // archive, so let's try to see if the file
+ // contains a CTF archive, who knows ...
+ if (dir_contains_ctf_archive(dirnam, basenam))
+ {
+ // The file does contain CTF debug information finally!
+ abigail::ctf_reader::read_context_sptr ctxt =
+ abigail::ctf_reader::create_read_context
+ (opts.file1,
+ opts.prepared_di_root_paths1,
+ env.get());
+ ABG_ASSERT(ctxt);
+
+ c1 = abigail::ctf_reader::read_corpus(ctxt.get(),
+ c1_status);
+ }
+ }
+#endif
OK, maybe the statement
"if (dir_contains_ctf_archive(dirnam, basenam))"
above is not necessary as I am not sure if the CTF archive file is
supposed to be present for normal binaries or not. If it's not
necessary, then please remove that line as well as the use of the
basenam/dirnam variables.
But you get the general idea. We just test if loading DWARF failed
(even if we were not instructed to use CTF) and then we try to load CTF.
The change is at one place right after trying to load the DWARF,
logically.
I do something similar to the loading of the second corpus c2.
Here is the entire change to the abidiff file:
diff --git a/tools/abidiff.cc b/tools/abidiff.cc
index e0bb35ac..0d9e59c2 100644
--- a/tools/abidiff.cc
+++ b/tools/abidiff.cc
@@ -47,6 +47,9 @@ using abigail::suppr::suppressions_type;
using abigail::suppr::read_suppressions;
using namespace abigail::dwarf_reader;
using namespace abigail::elf_reader;
+using abigail::tools_utils::base_name;
+using abigail::tools_utils::dir_name;
+using abigail::tools_utils::dir_contains_ctf_archive;
using abigail::tools_utils::emit_prefix;
using abigail::tools_utils::check_file;
using abigail::tools_utils::guess_file_type;
@@ -1205,6 +1208,36 @@ main(int argc, char* argv[])
set_suppressions(*ctxt, opts);
abigail::dwarf_reader::set_do_log(*ctxt, opts.do_log);
c1 = abigail::dwarf_reader::read_corpus_from_elf(*ctxt, c1_status);
+
+#ifdef WITH_CTF
+ if (// We were not instructed to use CTF ...
+ !opts.use_ctf
+ // ... and yet, no debug info was found ...
+ && (c1_status & STATUS_DEBUG_INFO_NOT_FOUND)
+ // ... but we found ELF symbols ...
+ && !(c1_status & STATUS_NO_SYMBOLS_FOUND))
+ {
+ string basenam, dirnam;
+ base_name(opts.file1, basenam);
+ dir_name(opts.file1, dirnam);
+ // ... the input file seems to contain CTF
+ // archive, so let's try to see if the file
+ // contains a CTF archive, who knows ...
+ if (dir_contains_ctf_archive(dirnam, basenam))
+ {
+ // The file does contain CTF debug information finally!
+ abigail::ctf_reader::read_context_sptr ctxt =
+ abigail::ctf_reader::create_read_context
+ (opts.file1,
+ opts.prepared_di_root_paths1,
+ env.get());
+ ABG_ASSERT(ctxt);
+
+ c1 = abigail::ctf_reader::read_corpus(ctxt.get(),
+ c1_status);
+ }
+ }
+#endif
if (!c1
|| (opts.fail_no_debug_info
&& (c1_status & STATUS_ALT_DEBUG_INFO_NOT_FOUND)
@@ -1212,7 +1245,7 @@ main(int argc, char* argv[])
return handle_error(c1_status, ctxt.get(),
argv[0], opts);
}
- }
+ }
break;
case abigail::tools_utils::FILE_TYPE_XML_CORPUS:
{
@@ -1289,13 +1322,44 @@ main(int argc, char* argv[])
set_suppressions(*ctxt, opts);
c2 = abigail::dwarf_reader::read_corpus_from_elf(*ctxt, c2_status);
+
+#ifdef WITH_CTF
+ if (// We were not instructed to use CTF ...
+ !opts.use_ctf
+ // ... and yet, no debug info was found ...
+ && (c2_status & STATUS_DEBUG_INFO_NOT_FOUND)
+ // ... but we found ELF symbols ...
+ && !(c2_status & STATUS_NO_SYMBOLS_FOUND))
+ {
+ string basenam, dirnam;
+ base_name(opts.file2, basenam);
+ dir_name(opts.file2, dirnam);
+ // ... the input file seems to contain CTF
+ // archive, so let's try to see if the file
+ // contains a CTF archive, who knows ...
+ if (dir_contains_ctf_archive(dirnam, basenam))
+ {
+ // The file does contain CTF debug information finally!
+ abigail::ctf_reader::read_context_sptr ctxt =
+ abigail::ctf_reader::create_read_context
+ (opts.file2,
+ opts.prepared_di_root_paths2,
+ env.get());
+ ABG_ASSERT(ctxt);
+
+ c2 = abigail::ctf_reader::read_corpus(ctxt.get(),
+ c2_status);
+ }
+ }
+#endif
+
if (!c2
|| (opts.fail_no_debug_info
&& (c2_status & STATUS_ALT_DEBUG_INFO_NOT_FOUND)
&& (c2_status & STATUS_DEBUG_INFO_NOT_FOUND)))
return handle_error(c2_status, ctxt.get(), argv[0], opts);
}
- }
+ }
break;
case abigail::tools_utils::FILE_TYPE_XML_CORPUS:
{
By the way, I think there should be at least one test that exercises the
workflow "try to load DWARF from the binary, fail, try to load CTF,
succeed", without having the --ctf provided. And we should have that
test for all the tools that were modified. We don't need to have that
for a Linux Kernel, obviously, as that would be too big to have in the
tarball.
Below is the entire amended patch that I came up with. I suspect the
use of the dir_contains_ctf_archive function everywhere is wrong, but
I'll let you be the judge of that. I think the documentation of when
that archive is present/necessary would be good to improve/add.
Thanks again for the patch!
From 481b547d7871a544b690943b16e6a173a729932f Mon Sep 17 00:00:00 2001
From: "Guillermo E. Martinez via Libabigail" <libabigail@sourceware.org>
Date: Fri, 30 Sep 2022 19:15:44 -0500
Subject: [PATCH] use CTF as a fallback when no DWARF debug info is present
By default, `abidw', `abidiff', `abipkgdiff' and `kmidiff' tools use
debug information in `DWARF` format, if present.
With this patch however, if no DWARF debug info was found, these tools
automatically try to use the CTF debug information without needing the
`--ctf' option flag. If no DWARF or CTF debug information were found,
the tools rely only on `ELF` symbols.
As the tools don't allow comparing corpora constructed using different
debug information i.e DWARF vs CTF, the first corpus built dictates
the expected debug information in the second one.
This works for libraries and Linux kernel. The `--ctf' option is
preserved to explicitly indicate to those tools that we want to use
CTF support.
* doc/manuals/abidiff.rst: Adjust usage tool information
to indicates fallback for CTF debug info when DWARF info
is not present.
* doc/manuals/abidw.rst: Likewise.
* doc/manuals/abipkgdiff.rst: Likewise.
* doc/manuals/kmidiff.rst: Likewise.
* src/abg-ctf-reader.cc (slurp_elf_info): Report status
when debug information is not present.
(read_corpus): Add code to locate `vmlinux.ctfa' just when
`main corpus' is being processed.
* src/abg-tools-utils.cc (maybe_load_vmlinux_dwarf_corpus)
(maybe_load_vmlinux_ctf_corpus): Return the status of the loading.
(build_corpus_group_from_kernel_dist_under): If loading the DWARF
debug info was not successful, try loading the CTF debug info.
* tests/data/test-diff-pkg-ctf/dirpkg-3-report-2.txt: Adjust
test case.
* tests/test-diff-pkg.cc (in_out_specs): Add `--ctf' option
in test case.
* tools/abidiff.cc (main): If loading the debug info assuming
DWARF failed, then try with CTF.
* tools/abidw.cc: Likewise.
* tools/abipkgdiff.cc: Likewise.
Signed-off-by: Guillermo E. Martinez <guillermo.e.martinez@oracle.com>
Signed-off-by: Dodji Seketeli <dodji@redhat.com>
---
doc/manuals/abidiff.rst | 15 +--
doc/manuals/abidw.rst | 15 ++-
doc/manuals/abipkgdiff.rst | 13 ++-
doc/manuals/kmidiff.rst | 9 +-
include/abg-tools-utils.h | 4 +
src/abg-ctf-reader.cc | 33 ++++---
src/abg-tools-utils.cc | 70 +++++++++++---
.../test-diff-pkg-ctf/dirpkg-3-report-2.txt | 16 ++++
tests/test-diff-pkg.cc | 2 +-
tools/abidiff.cc | 68 +++++++++++++-
tools/abidw.cc | 49 ++++++++--
tools/abipkgdiff.cc | 92 ++++++++++++++++++-
12 files changed, 330 insertions(+), 56 deletions(-)
diff --git a/doc/manuals/abidiff.rst b/doc/manuals/abidiff.rst
index 0c711d9e..5d0834cd 100644
--- a/doc/manuals/abidiff.rst
+++ b/doc/manuals/abidiff.rst
@@ -12,11 +12,12 @@ This tool can also compare the textual representations of the ABI of
two ELF binaries (as emitted by ``abidw``) or an ELF binary against a
textual representation of another ELF binary.
-For a comprehensive ABI change report that includes changes about
-function and variable sub-types, the two input shared libraries must
-be accompanied with their debug information in `DWARF`_ format.
-Otherwise, only `ELF`_ symbols that were added or removed are
-reported.
+For a comprehensive ABI change report between two input shared
+libraries that includes changes about function and variable sub-types,
+``abidiff`` uses by default, debug information in `DWARF`_ format, if
+present, otherwise it compares interfaces using debug information in
+`CTF`_ format, if present. Finally, if neither is found, only `ELF`_
+symbols that were added or removed are reported.
.. include:: tools-use-libabigail.txt
@@ -558,7 +559,7 @@ Options
* ``--ctf``
- When comparing binaries, extract ABI information from CTF debug
+ When comparing binaries, extract ABI information from `CTF`_ debug
information, if present.
* ``--stats``
@@ -785,4 +786,4 @@ Usage examples
.. _ELF: http://en.wikipedia.org/wiki/Executable_and_Linkable_Format
.. _DWARF: http://www.dwarfstd.org
-
+.. _CTF: https://raw.githubusercontent.com/wiki/oracle/binutils-gdb/files/ctf-spec.pdf
diff --git a/doc/manuals/abidw.rst b/doc/manuals/abidw.rst
index a3055c7e..c162ef25 100644
--- a/doc/manuals/abidw.rst
+++ b/doc/manuals/abidw.rst
@@ -8,8 +8,7 @@ representation of its ABI to standard output. The emitted
representation format, named ``ABIXML``, includes all the globally
defined functions and variables, along with a complete representation
of their types. It also includes a representation of the globally
-defined ELF symbols of the file. The input shared library must
-contain associated debug information in `DWARF`_ format.
+defined ELF symbols of the file.
When given the ``--linux-tree`` option, this program can also handle a
`Linux kernel`_ tree. That is, a directory tree that contains both
@@ -19,8 +18,13 @@ interface between the kernel and its module, to standard output. In
this case, we don't call it an ABI, but a KMI (Kernel Module
Interface). The emitted KMI includes all the globally defined
functions and variables, along with a complete representation of their
-types. The input binaries must contain associated debug information
-in `DWARF`_ format.
+types.
+
+To generate the ABI or KMI representation, by default ``abidw`` uses
+debug information in the `DWARF`_ format, if present, otherwise it
+looks for the debug information in the `CTF`_ format. Finally, if
+neither is found, it only reports which `ELF`_ symbols were added or
+removed.
.. include:: tools-use-libabigail.txt
@@ -326,7 +330,7 @@ Options
* ``--ctf``
- Extract ABI information from CTF debug information, if present in
+ Extract ABI information from `CTF`_ debug information, if present in
the given object.
* ``--annotate``
@@ -365,3 +369,4 @@ standard `here
.. _DWARF: http://www.dwarfstd.org
.. _GNU: http://www.gnu.org
.. _Linux Kernel: https://kernel.org/
+.. _CTF: https://raw.githubusercontent.com/wiki/oracle/binutils-gdb/files/ctf-spec.pdf
diff --git a/doc/manuals/abipkgdiff.rst b/doc/manuals/abipkgdiff.rst
index 9114775a..086762d5 100644
--- a/doc/manuals/abipkgdiff.rst
+++ b/doc/manuals/abipkgdiff.rst
@@ -13,12 +13,18 @@ binaries.
For a comprehensive ABI change report that includes changes about
function and variable sub-types, the two input packages must be
accompanied with their debug information packages that contain debug
-information either in `DWARF`_ or in `CTF` formats. Please note
+information either in `DWARF`_ or in `CTF`_ formats. Please note
however that some packages contain binaries that embed the debug
information directly in a section of said binaries. In those cases,
obviously, no separate debug information package is needed as the tool
will find the debug information inside the binaries.
+By default, ``abipkgdiff`` uses the debug information in `DWARF`_
+format, if present, otherwise it compares binaries interfaces using
+debug information in `CTF`_ format, if present. Finally, if neither
+is found, it uses only reports about `ELF`_ symbols that were added or
+removed.
+
.. include:: tools-use-libabigail.txt
.. _abipkgdiff_invocation_label:
@@ -525,8 +531,8 @@ Options
* ``--ctf``
- This is used to compare packages with CTF debug information, if
- present.
+ This is used to compare packages with `CTF`_ debug information,
+ if present.
.. _abipkgdiff_return_value_label:
@@ -546,4 +552,5 @@ In the later case, the value of the exit code is the same as for the
.. _Deb: https://en.wikipedia.org/wiki/Deb_%28file_format%29
.. _tar: https://en.wikipedia.org/wiki/Tar_%28computing%29
.. _DWARF: http://www.dwarfstd.org
+.. _CTF: https://raw.githubusercontent.com/wiki/oracle/binutils-gdb/files/ctf-spec.pdf
.. _Development Package: https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Devel_Packages
diff --git a/doc/manuals/kmidiff.rst b/doc/manuals/kmidiff.rst
index 53010189..4059c71f 100644
--- a/doc/manuals/kmidiff.rst
+++ b/doc/manuals/kmidiff.rst
@@ -74,6 +74,11 @@ functions and variables) between the Kernel and its modules. In
practice, though, some users might want to compare a subset of the
those interfaces.
+By default, ``kmidiff`` uses the debug information in `DWARF`_ format,
+if present, otherwise it compares interfaces using debug information
+in `CTF`_ format, if present. Finally, if neither is found, it only
+reports about `ELF`_ symbols that were added or removed.
+
Users can then define a "white list" of the interfaces to compare.
Such a white list is a just a file in the "INI" format that looks
like: ::
@@ -174,7 +179,7 @@ Options
* ``--ctf``
- Extract ABI information from CTF debug information, if present in
+ Extract ABI information from `CTF`_ debug information, if present in
the Kernel and Modules.
* ``--impacted-interfaces | -i``
@@ -242,3 +247,5 @@ Options
.. _ELF: http://en.wikipedia.org/wiki/Executable_and_Linkable_Format
.. _ksymtab: http://en.wikipedia.org/wiki/Executable_and_Linkable_Format
.. _Linux Kernel: https://kernel.org
+.. _DWARF: http://www.dwarfstd.org
+.. _CTF: https://raw.githubusercontent.com/wiki/oracle/binutils-gdb/files/ctf-spec.pdf
diff --git a/include/abg-tools-utils.h b/include/abg-tools-utils.h
index 27bbfefe..391a389a 100644
--- a/include/abg-tools-utils.h
+++ b/include/abg-tools-utils.h
@@ -297,6 +297,10 @@ bool
file_is_kernel_debuginfo_package(const string& file_path,
file_type file_type);
+bool
+dir_contains_ctf_archive(const string& directory,
+ const string& archive_prefix);
+
std::shared_ptr<char>
make_path_absolute(const char*p);
diff --git a/src/abg-ctf-reader.cc b/src/abg-ctf-reader.cc
index e307fcd7..bcd011d3 100644
--- a/src/abg-ctf-reader.cc
+++ b/src/abg-ctf-reader.cc
@@ -45,6 +45,7 @@ namespace ctf_reader
using std::dynamic_pointer_cast;
using abigail::tools_utils::dir_name;
using abigail::tools_utils::file_exists;
+using abigail::tools_utils::base_name;
class read_context
{
@@ -1524,7 +1525,12 @@ slurp_elf_info(read_context *ctxt,
corp->set_architecture_name(elf_helpers::e_machine_to_string(ehdr->e_machine));
find_alt_debuginfo(ctxt, &ctf_dbg_scn);
- ABG_ASSERT(ctxt->symtab);
+ if (!ctxt->symtab)
+ {
+ status |= elf_reader::STATUS_NO_SYMBOLS_FOUND;
+ return;
+ }
+
corp->set_symtab(ctxt->symtab);
if ((corp->get_origin() & corpus::LINUX_KERNEL_BINARY_ORIGIN)
@@ -1543,7 +1549,8 @@ slurp_elf_info(read_context *ctxt,
ctf_scn = ctf_dbg_scn;
else
{
- status |= elf_reader::STATUS_DEBUG_INFO_NOT_FOUND;
+ status |= (elf_reader::STATUS_OK |
+ elf_reader::STATUS_DEBUG_INFO_NOT_FOUND);
return;
}
}
@@ -1655,15 +1662,15 @@ read_corpus(read_context *ctxt, elf_reader::status &status)
origin |= corpus::LINUX_KERNEL_BINARY_ORIGIN;
corp->set_origin(origin);
- if (ctxt->cur_corpus_group_)
- ctxt->cur_corpus_group_->add_corpus(ctxt->cur_corpus_);
-
slurp_elf_info(ctxt, corp, status);
- if (!is_linux_kernel
- && ((status & elf_reader::STATUS_DEBUG_INFO_NOT_FOUND) |
- (status & elf_reader::STATUS_NO_SYMBOLS_FOUND)))
+ if ((status & elf_reader::STATUS_NO_SYMBOLS_FOUND) ||
+ (!is_linux_kernel &&
+ (status & elf_reader::STATUS_DEBUG_INFO_NOT_FOUND)))
return corp;
+ if (ctxt->cur_corpus_group_)
+ ctxt->cur_corpus_group_->add_corpus(ctxt->cur_corpus_);
+
// Set the set of exported declaration that are defined.
ctxt->exported_decls_builder
(ctxt->cur_corpus_->get_exported_decls_builder().get());
@@ -1674,9 +1681,13 @@ read_corpus(read_context *ctxt, elf_reader::status &status)
{
if (ctxt->ctfa == NULL)
{
- std::string ctfa_filename;
- if (find_ctfa_file(ctxt, ctfa_filename))
- ctxt->ctfa = ctf_arc_open(ctfa_filename.c_str(), &errp);
+ std::string filename;
+ base_name(ctxt->filename, filename);
+
+ // locate vmlinux.ctfa only when reader is processing
+ // vmlinux file, i.e the main corpus in the group.
+ if (filename == "vmlinux" && find_ctfa_file(ctxt, filename))
+ ctxt->ctfa = ctf_arc_open(filename.c_str(), &errp);
}
}
else
diff --git a/src/abg-tools-utils.cc b/src/abg-tools-utils.cc
index fe9ebc72..a23f08ea 100644
--- a/src/abg-tools-utils.cc
+++ b/src/abg-tools-utils.cc
@@ -1694,6 +1694,23 @@ file_is_kernel_debuginfo_package(const string& file_name, file_type file_type)
return result;
}
+/// Test if a directory contains a CTF archive.
+///
+/// @param directory the directory to consider.
+///
+/// @param archive_prefix the prefix of the archive file.
+///
+/// @return true iff @p directory contains a CTF archive file.
+bool
+dir_contains_ctf_archive(const string& directory,
+ const string& archive_prefix)
+{
+ string ctf_archive = directory + "/" + archive_prefix + ".ctfa";
+ if (file_exists(ctf_archive))
+ return true;
+ return false;
+}
+
/// The delete functor of a char buffer that has been created using
/// malloc.
struct malloced_char_star_deleter
@@ -2525,8 +2542,12 @@ get_binary_paths_from_kernel_dist(const string& dist_root,
/// @param t time to trace time spent in each step.
///
/// @param env the environment to create the corpus_group in.
-static void
-maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
+///
+/// @return the status of the loading. If it's
+/// abigail::elf_reader::STATUS_UNKNOWN, then it means nothing was
+/// done, meaning the function got out early.
+static abigail::elf_reader::status
+maybe_load_vmlinux_dwarf_corpus(corpus::origin& origin,
corpus_group_sptr& group,
const string& vmlinux,
vector<string>& modules,
@@ -2539,10 +2560,11 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
timer& t,
environment_sptr& env)
{
+ abigail::elf_reader::status status = abigail::elf_reader::STATUS_UNKNOWN;
+
if (!(origin & corpus::DWARF_ORIGIN))
- return;
+ return status;
- abigail::elf_reader::status status = abigail::elf_reader::STATUS_OK;
dwarf_reader::read_context_sptr ctxt;
ctxt =
dwarf_reader::create_read_context(vmlinux, di_roots, env.get(),
@@ -2569,6 +2591,7 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
<< vmlinux << "' ...\n" << std::flush;
// Read the vmlinux corpus and add it to the group.
+ status = abigail::elf_reader::STATUS_OK;
t.start();
read_and_add_corpus_to_group_from_elf(*ctxt, *group, status);
t.stop();
@@ -2579,7 +2602,7 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
<< t << "\n";
if (group->is_empty())
- return;
+ return status;
// Now add the corpora of the modules to the corpus group.
int total_nb_modules = modules.size();
@@ -2614,6 +2637,7 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
<< "' reading DONE: "
<< t << "\n";
}
+ return status;
}
/// If the @ref origin is CTF_ORIGIN it build a @ref corpus_group
@@ -2642,8 +2666,12 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
/// @param t time to trace time spent in each step.
///
/// @param env the environment to create the corpus_group in.
+///
+/// @return the status of the loading. If it's
+/// abigail::elf_reader::STATUS_UNKNOWN, then it means nothing was
+/// done, meaning the function got out early.
#ifdef WITH_CTF
-static void
+static abigail::elf_reader::status
maybe_load_vmlinux_ctf_corpus(corpus::origin origin,
corpus_group_sptr& group,
const string& vmlinux,
@@ -2654,10 +2682,11 @@ maybe_load_vmlinux_ctf_corpus(corpus::origin origin,
timer& t,
environment_sptr& env)
{
+ abigail::elf_reader::status status = abigail::elf_reader::STATUS_UNKNOWN;
+
if (!(origin & corpus::CTF_ORIGIN))
- return;
+ return status;
- abigail::elf_reader::status status = abigail::elf_reader::STATUS_OK;
ctf_reader::read_context_sptr ctxt;
ctxt = ctf_reader::create_read_context(vmlinux, di_roots, env.get());
group.reset(new corpus_group(env.get(), root));
@@ -2668,6 +2697,7 @@ maybe_load_vmlinux_ctf_corpus(corpus::origin origin,
<< vmlinux << "' ...\n" << std::flush;
// Read the vmlinux corpus and add it to the group.
+ status = abigail::elf_reader::STATUS_OK;
t.start();
read_and_add_corpus_to_group_from_elf(ctxt.get(), *group, status);
t.stop();
@@ -2678,7 +2708,7 @@ maybe_load_vmlinux_ctf_corpus(corpus::origin origin,
<< t << "\n";
if (group->is_empty())
- return;
+ return status;
// Now add the corpora of the modules to the corpus group.
int total_nb_modules = modules.size();
@@ -2707,6 +2737,7 @@ maybe_load_vmlinux_ctf_corpus(corpus::origin origin,
<< "' reading DONE: "
<< t << "\n";
}
+ return status;
}
#endif
@@ -2786,14 +2817,23 @@ build_corpus_group_from_kernel_dist_under(const string& root,
vector<char**> di_roots;
di_roots.push_back(&di_root_ptr);
- maybe_load_vmlinux_dwarf_corpus(origin, group, vmlinux,
- modules, root, di_roots,
- suppr_paths, kabi_wl_paths,
- supprs, verbose, t, env);
+ abigail::elf_reader::status status =
+ maybe_load_vmlinux_dwarf_corpus(origin, group, vmlinux,
+ modules, root, di_roots,
+ suppr_paths, kabi_wl_paths,
+ supprs, verbose, t, env);
#ifdef WITH_CTF
+ string vmlinux_basename;
+ base_name(vmlinux, vmlinux_basename);
+ if (origin == corpus::DWARF_ORIGIN
+ && (status & abigail::elf_reader::STATUS_DEBUG_INFO_NOT_FOUND)
+ && dir_contains_ctf_archive(root, vmlinux_basename))
+ // Let's try to load the binary using the CTF archive!
+ origin |= corpus::CTF_ORIGIN;
+
maybe_load_vmlinux_ctf_corpus(origin, group, vmlinux,
- modules, root, di_roots,
- verbose, t, env);
+ modules, root, di_roots,
+ verbose, t, env);
#endif
}
diff --git a/tests/data/test-diff-pkg-ctf/dirpkg-3-report-2.txt b/tests/data/test-diff-pkg-ctf/dirpkg-3-report-2.txt
index e69de29b..4938d221 100644
--- a/tests/data/test-diff-pkg-ctf/dirpkg-3-report-2.txt
+++ b/tests/data/test-diff-pkg-ctf/dirpkg-3-report-2.txt
@@ -0,0 +1,16 @@
+================ changes of 'libobj-v0.so'===============
+ Functions changes summary: 0 Removed, 1 Changed (1 filtered out), 0 Added functions
+ Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
+
+ 1 function with some indirect sub-type change:
+
+ [C] 'function void foo(S1*)' has some indirect sub-type changes:
+ parameter 1 of type 'S1*' has sub-type changes:
+ in pointed to type 'struct S1':
+ type size changed from 0 to 32 (in bits)
+ type alignment changed from 0 to 32
+ 1 data member insertion:
+ 'int mem2', at offset 0 (in bits)
+
+================ end of changes of 'libobj-v0.so'===============
+
diff --git a/tests/test-diff-pkg.cc b/tests/test-diff-pkg.cc
index e128ff63..8e510ca2 100644
--- a/tests/test-diff-pkg.cc
+++ b/tests/test-diff-pkg.cc
@@ -855,7 +855,7 @@ static InOutSpec in_out_specs[] =
{ // Just like the previous tests, but loc info is emitted.
"data/test-diff-pkg-ctf/dirpkg-3-dir1",
"data/test-diff-pkg-ctf/dirpkg-3-dir2",
- "--no-default-suppression --no-abignore",
+ "--ctf --no-default-suppression --no-abignore",
"data/test-diff-pkg-ctf/dirpkg-3.suppr",
"",
"",
diff --git a/tools/abidiff.cc b/tools/abidiff.cc
index e0bb35ac..0d9e59c2 100644
--- a/tools/abidiff.cc
+++ b/tools/abidiff.cc
@@ -47,6 +47,9 @@ using abigail::suppr::suppressions_type;
using abigail::suppr::read_suppressions;
using namespace abigail::dwarf_reader;
using namespace abigail::elf_reader;
+using abigail::tools_utils::base_name;
+using abigail::tools_utils::dir_name;
+using abigail::tools_utils::dir_contains_ctf_archive;
using abigail::tools_utils::emit_prefix;
using abigail::tools_utils::check_file;
using abigail::tools_utils::guess_file_type;
@@ -1205,6 +1208,36 @@ main(int argc, char* argv[])
set_suppressions(*ctxt, opts);
abigail::dwarf_reader::set_do_log(*ctxt, opts.do_log);
c1 = abigail::dwarf_reader::read_corpus_from_elf(*ctxt, c1_status);
+
+#ifdef WITH_CTF
+ if (// We were not instructed to use CTF ...
+ !opts.use_ctf
+ // ... and yet, no debug info was found ...
+ && (c1_status & STATUS_DEBUG_INFO_NOT_FOUND)
+ // ... but we found ELF symbols ...
+ && !(c1_status & STATUS_NO_SYMBOLS_FOUND))
+ {
+ string basenam, dirnam;
+ base_name(opts.file1, basenam);
+ dir_name(opts.file1, dirnam);
+ // ... the input file seems to contain CTF
+ // archive, so let's try to see if the file
+ // contains a CTF archive, who knows ...
+ if (dir_contains_ctf_archive(dirnam, basenam))
+ {
+ // The file does contain CTF debug information finally!
+ abigail::ctf_reader::read_context_sptr ctxt =
+ abigail::ctf_reader::create_read_context
+ (opts.file1,
+ opts.prepared_di_root_paths1,
+ env.get());
+ ABG_ASSERT(ctxt);
+
+ c1 = abigail::ctf_reader::read_corpus(ctxt.get(),
+ c1_status);
+ }
+ }
+#endif
if (!c1
|| (opts.fail_no_debug_info
&& (c1_status & STATUS_ALT_DEBUG_INFO_NOT_FOUND)
@@ -1212,7 +1245,7 @@ main(int argc, char* argv[])
return handle_error(c1_status, ctxt.get(),
argv[0], opts);
}
- }
+ }
break;
case abigail::tools_utils::FILE_TYPE_XML_CORPUS:
{
@@ -1289,13 +1322,44 @@ main(int argc, char* argv[])
set_suppressions(*ctxt, opts);
c2 = abigail::dwarf_reader::read_corpus_from_elf(*ctxt, c2_status);
+
+#ifdef WITH_CTF
+ if (// We were not instructed to use CTF ...
+ !opts.use_ctf
+ // ... and yet, no debug info was found ...
+ && (c2_status & STATUS_DEBUG_INFO_NOT_FOUND)
+ // ... but we found ELF symbols ...
+ && !(c2_status & STATUS_NO_SYMBOLS_FOUND))
+ {
+ string basenam, dirnam;
+ base_name(opts.file2, basenam);
+ dir_name(opts.file2, dirnam);
+ // ... the input file seems to contain CTF
+ // archive, so let's try to see if the file
+ // contains a CTF archive, who knows ...
+ if (dir_contains_ctf_archive(dirnam, basenam))
+ {
+ // The file does contain CTF debug information finally!
+ abigail::ctf_reader::read_context_sptr ctxt =
+ abigail::ctf_reader::create_read_context
+ (opts.file2,
+ opts.prepared_di_root_paths2,
+ env.get());
+ ABG_ASSERT(ctxt);
+
+ c2 = abigail::ctf_reader::read_corpus(ctxt.get(),
+ c2_status);
+ }
+ }
+#endif
+
if (!c2
|| (opts.fail_no_debug_info
&& (c2_status & STATUS_ALT_DEBUG_INFO_NOT_FOUND)
&& (c2_status & STATUS_DEBUG_INFO_NOT_FOUND)))
return handle_error(c2_status, ctxt.get(), argv[0], opts);
}
- }
+ }
break;
case abigail::tools_utils::FILE_TYPE_XML_CORPUS:
{
diff --git a/tools/abidw.cc b/tools/abidw.cc
index f38d6048..c68a30f4 100644
--- a/tools/abidw.cc
+++ b/tools/abidw.cc
@@ -41,6 +41,9 @@ using std::ofstream;
using std::vector;
using std::shared_ptr;
using abg_compat::optional;
+using abigail::tools_utils::base_name;
+using abigail::tools_utils::dir_name;
+using abigail::tools_utils::dir_contains_ctf_archive;
using abigail::tools_utils::emit_prefix;
using abigail::tools_utils::temp_file;
using abigail::tools_utils::temp_file_sptr;
@@ -557,16 +560,16 @@ load_corpus_and_write_abixml(char* argv[],
if (opts.use_ctf)
{
abigail::ctf_reader::read_context_sptr ctxt
- = abigail::ctf_reader::create_read_context(opts.in_file_path,
- opts.prepared_di_root_paths,
- env.get());
+ = abigail::ctf_reader::create_read_context(opts.in_file_path,
+ opts.prepared_di_root_paths,
+ env.get());
assert (ctxt);
t.start();
corp = abigail::ctf_reader::read_corpus (ctxt, s);
t.stop();
if (opts.do_log)
- emit_prefix(argv[0], cerr)
- << "read corpus from elf file in: " << t << "\n";
+ emit_prefix(argv[0], cerr)
+ << "read corpus from elf file in: " << t << "\n";
}
else
#endif
@@ -616,7 +619,7 @@ load_corpus_and_write_abixml(char* argv[],
}
if (opts.exported_interfaces_only.has_value())
- env->analyze_exported_interfaces_only(*opts.exported_interfaces_only);
+ env->analyze_exported_interfaces_only(*opts.exported_interfaces_only);
t.start();
corp = dwarf_reader::read_corpus_from_elf(ctxt, s);
@@ -632,6 +635,38 @@ load_corpus_and_write_abixml(char* argv[],
if (opts.do_log)
emit_prefix(argv[0], cerr)
<< "reset read context in: " << t << "\n";
+
+#ifdef WITH_CTF
+ if (// We were not instructed to use CTF ...
+ !opts.use_ctf
+ // ... and yet, no debug info was found ...
+ && (s & abigail::elf_reader::STATUS_DEBUG_INFO_NOT_FOUND)
+ // ... but we found ELF symbols ...
+ && !(s & abigail::elf_reader::STATUS_NO_SYMBOLS_FOUND))
+ {
+ string basenam, dirnam;
+ base_name(opts.in_file_path, basenam);
+ dir_name(opts.in_file_path, dirnam);
+ // ... the input file seems to contain CTF
+ // archive, so let's try to see if the file
+ // contains a CTF archive, who knows ...
+ if (dir_contains_ctf_archive(dirnam, basenam))
+ {
+ abigail::ctf_reader::read_context_sptr ctxt
+ = abigail::ctf_reader::create_read_context(opts.in_file_path,
+ opts.prepared_di_root_paths,
+ env.get());
+ assert (ctxt);
+ t.start();
+ corp = abigail::ctf_reader::read_corpus (ctxt, s);
+ t.stop();
+
+ if (opts.do_log)
+ emit_prefix(argv[0], cerr)
+ << "read CTF corpus from elf file in:" << t << "\n";
+ }
+ }
+#endif
}
// If we couldn't create a corpus, emit some (hopefully) useful
@@ -834,7 +869,7 @@ load_kernel_corpus_group_and_write_abixml(char* argv[],
global_timer.start();
t.start();
-corpus::origin origin =
+ corpus::origin origin =
#ifdef WITH_CTF
opts.use_ctf ? corpus::CTF_ORIGIN :
#endif
diff --git a/tools/abipkgdiff.cc b/tools/abipkgdiff.cc
index 656d5882..e7bb5c37 100644
--- a/tools/abipkgdiff.cc
+++ b/tools/abipkgdiff.cc
@@ -127,6 +127,7 @@ using abigail::tools_utils::make_path_absolute;
using abigail::tools_utils::base_name;
using abigail::tools_utils::get_rpm_arch;
using abigail::tools_utils::file_is_kernel_package;
+using abigail::tools_utils::dir_contains_ctf_archive;
using abigail::tools_utils::gen_suppr_spec_from_headers;
using abigail::tools_utils::get_default_system_suppression_file_path;
using abigail::tools_utils::get_default_user_suppression_file_path;
@@ -1346,6 +1347,34 @@ compare(const elf_file& elf1,
add_read_context_suppressions(*ctxt_dwarf, opts.kabi_suppressions);
corpus1 = read_corpus_from_elf(*ctxt_dwarf, c1_status);
+
+#ifdef WITH_CTF
+ if (// We were not instructed to use CTF ...
+ !opts.use_ctf
+ // ... and yet, no debug info was found ...
+ && (c1_status & abigail::elf_reader::STATUS_DEBUG_INFO_NOT_FOUND)
+ // ... but we found ELF symbols ...
+ && !(c1_status & abigail::elf_reader::STATUS_NO_SYMBOLS_FOUND))
+ {
+ string basenam, dirnam;
+ base_name(elf1.path, basenam);
+ dir_name(elf1.path, dirnam);
+ // ... the input file seems to contain CTF
+ // archive, so let's try to see if the file
+ // contains a CTF archive, who knows ...
+ if (dir_contains_ctf_archive(dirnam, basenam))
+ {
+ // The file does contain CTF debug information
+ // finally!
+ ctxt_ctf = abigail::ctf_reader::create_read_context(elf1.path,
+ di_dirs1,
+ env.get());
+ ABG_ASSERT(ctxt_ctf);
+ corpus1 = abigail::ctf_reader::read_corpus(ctxt_ctf.get(),
+ c1_status);
+ }
+ }
+#endif
}
bool bail_out = false;
@@ -1451,6 +1480,34 @@ compare(const elf_file& elf1,
add_read_context_suppressions(*ctxt_dwarf, opts.kabi_suppressions);
corpus2 = read_corpus_from_elf(*ctxt_dwarf, c2_status);
+
+#ifdef WITH_CTF
+ if (// We were not instructed to use CTF ...
+ !opts.use_ctf
+ // ... and yet, no debug info was found ...
+ && (c2_status & abigail::elf_reader::STATUS_DEBUG_INFO_NOT_FOUND)
+ // ... but we found ELF symbols ...
+ && !(c2_status & abigail::elf_reader::STATUS_NO_SYMBOLS_FOUND))
+ {
+ string basenam, dirnam;
+ base_name(elf2.path, basenam);
+ dir_name(elf2.path, dirnam);
+ // ... the input file seems to contain CTF
+ // archive, so let's try to see if the file
+ // contains a CTF archive, who knows ...
+ if (dir_contains_ctf_archive(dirnam, basenam))
+ {
+ // The file does contain CTF debug information
+ // finally!
+ ctxt_ctf = abigail::ctf_reader::create_read_context(elf2.path,
+ di_dirs2,
+ env.get());
+ ABG_ASSERT(ctxt_ctf);
+ corpus2 = abigail::ctf_reader::read_corpus(ctxt_ctf.get(),
+ c2_status);
+ }
+ }
+#endif
}
bool bail_out = false;
@@ -1612,17 +1669,44 @@ compare_to_self(const elf_file& elf,
di_dirs,
env.get());
ABG_ASSERT(ctxt_ctf);
- corp = abigail::ctf_reader::read_corpus(ctxt_ctf.get(),
- c_status);
+ corp = abigail::ctf_reader::read_corpus(ctxt_ctf.get(), c_status);
}
else
#endif
{
ctxt_dwarf =
- create_read_context(elf.path, di_dirs, env.get(),
- /*read_all_types=*/opts.show_all_types);
+ create_read_context(elf.path, di_dirs, env.get(),
+ /*read_all_types=*/opts.show_all_types);
corp = read_corpus_from_elf(*ctxt_dwarf, c_status);
+
+#ifdef WITH_CTF
+ if (// We were not instructed to use CTF ...
+ !opts.use_ctf
+ // ... and yet, no debug info was found ...
+ && (c_status & abigail::elf_reader::STATUS_DEBUG_INFO_NOT_FOUND)
+ // ... but we found ELF symbols ...
+ && !(c_status & abigail::elf_reader::STATUS_NO_SYMBOLS_FOUND))
+ {
+ string basenam, dirnam;
+ base_name(elf.path, basenam);
+ dir_name(elf.path, dirnam);
+ // ... the input file seems to contain CTF
+ // archive, so let's try to see if the file
+ // contains a CTF archive, who knows ...
+ if (dir_contains_ctf_archive(dirnam, basenam))
+ {
+ // The file does contain CTF debug information
+ // finally!
+ ctxt_ctf = abigail::ctf_reader::create_read_context(elf.path,
+ di_dirs,
+ env.get());
+ ABG_ASSERT(ctxt_ctf);
+ corp = abigail::ctf_reader::read_corpus(ctxt_ctf.get(),
+ c_status);
+ }
+ }
+#endif
}
if (!(c_status & abigail::elf_reader::STATUS_OK))
On 10/4/22 04:04, Dodji Seketeli wrote:
> Hello Guillermo,
>
HI Dodji,
> Thank you for this patch.
>
Your welcome!.
> Please find my comments below.
>
> "Guillermo E. Martinez via Libabigail" <libabigail@sourceware.org> a
> écrit:
>
>
>> By default, `abidw', `abidiff', `abipkgdiff' and `kmidiff' tools use
>> debug information in `DWARF` format, if present, otherwise now
>> automatically the tools try to extract and build the binaries IR using
>> debug information in `CTF` format without use of `--ctf' option, if
>> present, finally, if neither is found, they use only `ELF` symbols to
>> extract, build, compare and report which of them were added or removed.
>>
>> In case of neither DWARF nor CTF debug information is present
>> (`STATUS_DEBUG_INFO_NOT_FOUND') and `--ctf' option was not passed in
>> the command line, those tools use the symbol object build by DWARF
>> reader (default) to completed theirs task.
>>
>> Tools don't allow comparing corpora built with between different debug
>> information i.e DWARF vs CTF, the first corpus built dictate the
>> expected debug information in the second one.
>>
>> This work for libraries and Linux kernel. The `--ctf' option is
>> preserved to explicitly indicate to those tools that we want to use
>> CTF support.
>>
>> * doc/manuals/abidiff.rst: Adjust usage tool information
>> to indicates fallback for CTF debug info when DWARF info
>> is not present.
>> * doc/manuals/abidw.rst: Likewise.
>> * doc/manuals/abipkgdiff.rst: Likewise.
>> * doc/manuals/kmidiff.rst: Likewise.
>
> I have made some slight fixes to changes to these doc files. Nothing
> major you'll see the changes in the updated version of the patch that I
> am posting at the end of this message.
>
OK.
> [...]
>
>> * src/abg-ctf-reader.cc (slurp_elf_info): Report status
>> when debug information is not present.
>> (read_corpus): Add code to locate `vmlinux.ctfa' just when
>> `main corpus' is being processed.
>> * src/abg-tools-utils.cc (maybe_load_vmlinux_dwarf_corpus):
>> Replace by a reference `origin' argument to notify whether
>> CTF reader needs to be executed.
>> * tests/data/test-diff-pkg-ctf/dirpkg-3-report-2.txt: Adjust
>> test case.
>> * tests/test-diff-pkg.cc (in_out_specs): Add `--ctf' option
>> in test case.
>> * tools/abidiff.cc (main): Add `origin' depending of command
>> line argument by default it is `corpus::DWARF_ORIGIN'. Add
>> `dwarf_corpus' to be used as default corpus (containing ELF
>> symbols), when DWARF nor CTF debug information was found.
>> * tools/abidw.cc: Likewise.
>> * tools/abipkgdiff.cc: Likewise.
>>
>> Signed-off-by: Guillermo E. Martinez <guillermo.e.martinez@oracle.com>
>> ---
>> doc/manuals/abidiff.rst | 15 +--
>> doc/manuals/abidw.rst | 15 ++-
>> doc/manuals/abipkgdiff.rst | 13 +-
>> doc/manuals/kmidiff.rst | 9 +-
>> src/abg-ctf-reader.cc | 33 +++--
>> src/abg-tools-utils.cc | 22 +++-
>> .../test-diff-pkg-ctf/dirpkg-3-report-2.txt | 16 +++
>> tests/test-diff-pkg.cc | 2 +-
>> tools/abidiff.cc | 82 ++++++++-----
>> tools/abidw.cc | 54 +++++---
>> tools/abipkgdiff.cc | 116 +++++++++++-------
>> 11 files changed, 257 insertions(+), 120 deletions(-)
>>
>> diff --git a/doc/manuals/abidiff.rst b/doc/manuals/abidiff.rst
>> index 0c711d9e..caeceded 100644
>> --- a/doc/manuals/abidiff.rst
>> +++ b/doc/manuals/abidiff.rst
>> @@ -12,11 +12,12 @@ This tool can also compare the textual representations of the ABI of
>> two ELF binaries (as emitted by ``abidw``) or an ELF binary against a
>> textual representation of another ELF binary.
>>
>> -For a comprehensive ABI change report that includes changes about
>> -function and variable sub-types, the two input shared libraries must
>> -be accompanied with their debug information in `DWARF`_ format.
>> -Otherwise, only `ELF`_ symbols that were added or removed are
>> -reported.
>> +For a comprehensive ABI change report between two input shared
>> +libraries that includes changes about function and variable sub-types,
>> +``abidiff`` uses by default, debug information in `DWARF`_ format, if
>> +present, otherwise it compares interfaces using debug information in
>> +`CTF`_ format, if present, finally, if neither is found, it uses only
>> +`ELF`_ symbols to report which of them were added or removed.
>>
>> .. include:: tools-use-libabigail.txt
>>
>> @@ -558,7 +559,7 @@ Options
>>
>> * ``--ctf``
>>
>> - When comparing binaries, extract ABI information from CTF debug
>> + When comparing binaries, extract ABI information from `CTF`_ debug
>> information, if present.
>>
>> * ``--stats``
>> @@ -785,4 +786,4 @@ Usage examples
>>
>> .. _ELF: http://en.wikipedia.org/wiki/Executable_and_Linkable_Format
>> .. _DWARF: http://www.dwarfstd.org
>> -
>> +.. _CTF: https://raw.githubusercontent.com/wiki/oracle/binutils-gdb/files/ctf-spec.pdf
>> diff --git a/doc/manuals/abidw.rst b/doc/manuals/abidw.rst
>> index a3055c7e..20948805 100644
>> --- a/doc/manuals/abidw.rst
>> +++ b/doc/manuals/abidw.rst
>> @@ -8,8 +8,7 @@ representation of its ABI to standard output. The emitted
>> representation format, named ``ABIXML``, includes all the globally
>> defined functions and variables, along with a complete representation
>> of their types. It also includes a representation of the globally
>> -defined ELF symbols of the file. The input shared library must
>> -contain associated debug information in `DWARF`_ format.
>> +defined ELF symbols of the file.
>>
>> When given the ``--linux-tree`` option, this program can also handle a
>> `Linux kernel`_ tree. That is, a directory tree that contains both
>> @@ -19,8 +18,13 @@ interface between the kernel and its module, to standard output. In
>> this case, we don't call it an ABI, but a KMI (Kernel Module
>> Interface). The emitted KMI includes all the globally defined
>> functions and variables, along with a complete representation of their
>> -types. The input binaries must contain associated debug information
>> -in `DWARF`_ format.
>> +types.
>> +
>> +To generate either ABI or KMI representation, by default ``abidw``
>> +uses debug information in `DWARF`_ format, if present, otherwise it
>> +looks for debug information in `CTF`_ format, if present, finally, if
>> +neither is found, it uses only `ELF`_ symbols to report which of them
>> +were added or removed.
>>
>> .. include:: tools-use-libabigail.txt
>>
>> @@ -326,7 +330,7 @@ Options
>>
>> * ``--ctf``
>>
>> - Extract ABI information from CTF debug information, if present in
>> + Extract ABI information from `CTF`_ debug information, if present in
>> the given object.
>>
>> * ``--annotate``
>> @@ -365,3 +369,4 @@ standard `here
>> .. _DWARF: http://www.dwarfstd.org
>> .. _GNU: http://www.gnu.org
>> .. _Linux Kernel: https://kernel.org/
>> +.. _CTF: https://raw.githubusercontent.com/wiki/oracle/binutils-gdb/files/ctf-spec.pdf
>> diff --git a/doc/manuals/abipkgdiff.rst b/doc/manuals/abipkgdiff.rst
>> index 9114775a..771bb034 100644
>> --- a/doc/manuals/abipkgdiff.rst
>> +++ b/doc/manuals/abipkgdiff.rst
>> @@ -13,12 +13,18 @@ binaries.
>> For a comprehensive ABI change report that includes changes about
>> function and variable sub-types, the two input packages must be
>> accompanied with their debug information packages that contain debug
>> -information either in `DWARF`_ or in `CTF` formats. Please note
>> +information either in `DWARF`_ or in `CTF`_ formats. Please note
>> however that some packages contain binaries that embed the debug
>> information directly in a section of said binaries. In those cases,
>> obviously, no separate debug information package is needed as the tool
>> will find the debug information inside the binaries.
>>
>> +By default, ``abipkgdiff`` uses debug information in `DWARF`_ format,
>> +if present, otherwise it compares binaries interfaces using debug
>> +information in `CTF`_ format, if present, finally, if neither is
>> +found, it uses only `ELF`_ symbols to report which of them were added
>> +or removed.
>> +
>> .. include:: tools-use-libabigail.txt
>>
>> .. _abipkgdiff_invocation_label:
>> @@ -525,8 +531,8 @@ Options
>>
>> * ``--ctf``
>>
>> - This is used to compare packages with CTF debug information, if
>> - present.
>> + This is used to compare packages with `CTF`_ debug information,
>> + if present.
>>
>> .. _abipkgdiff_return_value_label:
>>
>> @@ -546,4 +552,5 @@ In the later case, the value of the exit code is the same as for the
>> .. _Deb: https://en.wikipedia.org/wiki/Deb_%28file_format%29
>> .. _tar: https://en.wikipedia.org/wiki/Tar_%28computing%29
>> .. _DWARF: http://www.dwarfstd.org
>> +.. _CTF: https://raw.githubusercontent.com/wiki/oracle/binutils-gdb/files/ctf-spec.pdf
>> .. _Development Package: https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Devel_Packages
>> diff --git a/doc/manuals/kmidiff.rst b/doc/manuals/kmidiff.rst
>> index 53010189..a27d2456 100644
>> --- a/doc/manuals/kmidiff.rst
>> +++ b/doc/manuals/kmidiff.rst
>> @@ -74,6 +74,11 @@ functions and variables) between the Kernel and its modules. In
>> practice, though, some users might want to compare a subset of the
>> those interfaces.
>>
>> +By default, ``kmidiff`` uses debug information in `DWARF`_ format,
>> +if present, otherwise it compares interfaces using debug information
>> +in `CTF`_ format, if present, finally, if neither is found, it uses
>> +only `ELF`_ symbols to report which were added or removed.
>> +
>> Users can then define a "white list" of the interfaces to compare.
>> Such a white list is a just a file in the "INI" format that looks
>> like: ::
>> @@ -174,7 +179,7 @@ Options
>>
>> * ``--ctf``
>>
>> - Extract ABI information from CTF debug information, if present in
>> + Extract ABI information from `CTF`_ debug information, if present in
>> the Kernel and Modules.
>>
>> * ``--impacted-interfaces | -i``
>> @@ -242,3 +247,5 @@ Options
>> .. _ELF: http://en.wikipedia.org/wiki/Executable_and_Linkable_Format
>> .. _ksymtab: http://en.wikipedia.org/wiki/Executable_and_Linkable_Format
>> .. _Linux Kernel: https://kernel.org
>> +.. _DWARF: http://www.dwarfstd.org
>> +.. _CTF: https://raw.githubusercontent.com/wiki/oracle/binutils-gdb/files/ctf-spec.pdf
>> diff --git a/src/abg-ctf-reader.cc b/src/abg-ctf-reader.cc
>> index 9148a646..b3cde365 100644
>> --- a/src/abg-ctf-reader.cc
>> +++ b/src/abg-ctf-reader.cc
>> @@ -45,6 +45,7 @@ namespace ctf_reader
>> using std::dynamic_pointer_cast;
>> using abigail::tools_utils::dir_name;
>> using abigail::tools_utils::file_exists;
>> +using abigail::tools_utils::base_name;
>>
>> class read_context
>> {
>> @@ -1545,7 +1546,12 @@ slurp_elf_info(read_context *ctxt,
>> corp->set_architecture_name(elf_helpers::e_machine_to_string(ehdr->e_machine));
>>
>> find_alt_debuginfo(ctxt, &ctf_dbg_scn);
>> - ABG_ASSERT(ctxt->symtab);
>> + if (!ctxt->symtab)
>> + {
>> + status |= elf_reader::STATUS_NO_SYMBOLS_FOUND;
>> + return;
>> + }
>> +
>> corp->set_symtab(ctxt->symtab);
>>
>> if ((corp->get_origin() & corpus::LINUX_KERNEL_BINARY_ORIGIN)
>> @@ -1564,7 +1570,8 @@ slurp_elf_info(read_context *ctxt,
>> ctf_scn = ctf_dbg_scn;
>> else
>> {
>> - status |= elf_reader::STATUS_DEBUG_INFO_NOT_FOUND;
>> + status |= (elf_reader::STATUS_OK |
>> + elf_reader::STATUS_DEBUG_INFO_NOT_FOUND);
>> return;
>> }
>> }
>> @@ -1676,15 +1683,15 @@ read_corpus(read_context *ctxt, elf_reader::status &status)
>> origin |= corpus::LINUX_KERNEL_BINARY_ORIGIN;
>> corp->set_origin(origin);
>>
>> - if (ctxt->cur_corpus_group_)
>> - ctxt->cur_corpus_group_->add_corpus(ctxt->cur_corpus_);
>> -
>> slurp_elf_info(ctxt, corp, status);
>> - if (!is_linux_kernel
>> - && ((status & elf_reader::STATUS_DEBUG_INFO_NOT_FOUND) |
>> - (status & elf_reader::STATUS_NO_SYMBOLS_FOUND)))
>> + if ((status & elf_reader::STATUS_NO_SYMBOLS_FOUND) ||
>> + (!is_linux_kernel &&
>> + (status & elf_reader::STATUS_DEBUG_INFO_NOT_FOUND)))
>> return corp;
>>
>> + if (ctxt->cur_corpus_group_)
>> + ctxt->cur_corpus_group_->add_corpus(ctxt->cur_corpus_);
>> +
>> // Set the set of exported declaration that are defined.
>> ctxt->exported_decls_builder
>> (ctxt->cur_corpus_->get_exported_decls_builder().get());
>> @@ -1695,9 +1702,13 @@ read_corpus(read_context *ctxt, elf_reader::status &status)
>> {
>> if (ctxt->ctfa == NULL)
>> {
>> - std::string ctfa_filename;
>> - if (find_ctfa_file(ctxt, ctfa_filename))
>> - ctxt->ctfa = ctf_arc_open(ctfa_filename.c_str(), &errp);
>> + std::string filename;
>> + base_name(ctxt->filename, filename);
>> +
>> + // locate vmlinux.ctfa only when reader is processing
>> + // vmlinux file, i.e the main corpus in the group.
>> + if (filename == "vmlinux" && find_ctfa_file(ctxt, filename))
>> + ctxt->ctfa = ctf_arc_open(filename.c_str(), &errp);
>> }
>> }
>> else
>> diff --git a/src/abg-tools-utils.cc b/src/abg-tools-utils.cc
>> index fe9ebc72..7d00f726 100644
>> --- a/src/abg-tools-utils.cc
>> +++ b/src/abg-tools-utils.cc
>> @@ -2493,6 +2493,11 @@ get_binary_paths_from_kernel_dist(const string& dist_root,
>> /// made of vmlinux kernel file and the linux kernel modules found
>> /// under @p root directory and under its sub-directories, recursively.
>> ///
>> +/// If the vmlinux file doens't have DWARF info, it looks for
>> +/// vmlinux.ctfa, if it's present, it assumes that kernel was build
>> +/// with CTF support, then it updates @ref origin, given chance to
>> +/// CTF reader to build the IR for kernel build directory.
>> +///
>> /// @param origin the debug type information in vmlinux kernel and
>> /// the linux kernel modules to be used to build the corpora @p group.
>> ///
>> @@ -2526,7 +2531,7 @@ get_binary_paths_from_kernel_dist(const string& dist_root,
>> ///
>> /// @param env the environment to create the corpus_group in.
>> static void
>> -maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
>> +maybe_load_vmlinux_dwarf_corpus(corpus::origin &origin,
>> corpus_group_sptr& group,
>> const string& vmlinux,
>> vector<string>& modules,
>> @@ -2578,6 +2583,21 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
>> << " reading DONE:"
>> << t << "\n";
>>
>> + if (status & elf_reader::STATUS_DEBUG_INFO_NOT_FOUND)
>> + {
>> + // vmlinux doesn't have DWARF debug info, so it might contain
>> + // CTF debug info, this means vmlinux.ctfa is under kernel
>> + // build directory.
>> + string ctfa_file = root + "/vmlinux.ctfa";
>> + if (file_exists(ctfa_file))
>> + {
>> + // OK. Likely CTF could build a better IR, then let's
>> + // notify the caller to try with CTF reader.
>> + origin = corpus::CTF_ORIGIN;
>> + return;
>> + }
>> + }
>> +
>
> Hmmh, I think this change is done at logical level that is too "low",
> compared to the level at which I believe this decision should be made.
> Namely, maybe_load_vmlinux_dwarf_corpus is meant to try to load DWARF
> debug information. It should not deal with CTF debug information
> (looking for CTF archive files etc). Rather, I think that if the
> loading of DWARF failed, that function should tell its caller about it.
> It would thus be up to the caller to decide what to do with that
> information. For instance, the caller would then be able to try and load
> CTF debug information instead.
>
Ohh. I see, much better.
> So I would rather modify maybe_load_vmlinux_dwarf_corpus to make it
> return the status of the loading, so that the caller can know if the
> loading succeeded or not.
>
Yes. totally agree.
> So here is the change I would propose for the
> maybe_load_vmlinux_dwarf_corpus and maybe_load_vmlinux_ctf_corpus
> functions instead (it's part of the whole patch that I have posted at
> the end of this message):
>
> @@ -2525,8 +2542,12 @@ get_binary_paths_from_kernel_dist(const string& dist_root,
> /// @param t time to trace time spent in each step.
> ///
> /// @param env the environment to create the corpus_group in.
> -static void
> -maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
> +///
> +/// @return the status of the loading. If it's
> +/// abigail::elf_reader::STATUS_UNKNOWN, then it means nothing was
> +/// done, meaning the function got out early.
> +static abigail::elf_reader::status
> +maybe_load_vmlinux_dwarf_corpus(corpus::origin& origin,
> corpus_group_sptr& group,
> const string& vmlinux,
> vector<string>& modules,
> @@ -2539,10 +2560,11 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
> timer& t,
> environment_sptr& env)
> {
> + abigail::elf_reader::status status = abigail::elf_reader::STATUS_UNKNOWN;
> +
> if (!(origin & corpus::DWARF_ORIGIN))
> - return;
> + return status;
>
> - abigail::elf_reader::status status = abigail::elf_reader::STATUS_OK;
> dwarf_reader::read_context_sptr ctxt;
> ctxt =
> dwarf_reader::create_read_context(vmlinux, di_roots, env.get(),
> @@ -2569,6 +2591,7 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
> << vmlinux << "' ...\n" << std::flush;
>
> // Read the vmlinux corpus and add it to the group.
> + status = abigail::elf_reader::STATUS_OK;
> t.start();
> read_and_add_corpus_to_group_from_elf(*ctxt, *group, status);
> t.stop();
> @@ -2579,7 +2602,7 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
> << t << "\n";
>
At this point if `vmlinux' file doesn't have DWARF information, the `status'
returned by `maybe_load_vmlinux_dwarf_corpus' will set the bit field
`STATUS_DEBUG_INFO_NOT_FOUND', but it is not verified here, and since vmlinux
corpus was already added into the group in `read_debug_info_into_corpus'
function, it continues processing modules without the main corpus information,
Is this the expected behaviour?
> if (group->is_empty())
> - return;
> + return status;
>
> // Now add the corpora of the modules to the corpus group.
> int total_nb_modules = modules.size();
> @@ -2614,6 +2637,7 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
> << "' reading DONE: "
> << t << "\n";
> }
> + return status;
> }
>
> /// If the @ref origin is CTF_ORIGIN it build a @ref corpus_group
> @@ -2642,8 +2666,12 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
> /// @param t time to trace time spent in each step.
> ///
> /// @param env the environment to create the corpus_group in.
> +///
> +/// @return the status of the loading. If it's
> +/// abigail::elf_reader::STATUS_UNKNOWN, then it means nothing was
> +/// done, meaning the function got out early.
> #ifdef WITH_CTF
> -static void
> +static abigail::elf_reader::status
> maybe_load_vmlinux_ctf_corpus(corpus::origin origin,
> corpus_group_sptr& group,
> const string& vmlinux,
> @@ -2654,10 +2682,11 @@ maybe_load_vmlinux_ctf_corpus(corpus::origin origin,
> timer& t,
> environment_sptr& env)
> {
> + abigail::elf_reader::status status = abigail::elf_reader::STATUS_UNKNOWN;
> +
> if (!(origin & corpus::CTF_ORIGIN))
> - return;
> + return status;
>
> - abigail::elf_reader::status status = abigail::elf_reader::STATUS_OK;
> ctf_reader::read_context_sptr ctxt;
> ctxt = ctf_reader::create_read_context(vmlinux, di_roots, env.get());
> group.reset(new corpus_group(env.get(), root));
> @@ -2668,6 +2697,7 @@ maybe_load_vmlinux_ctf_corpus(corpus::origin origin,
> << vmlinux << "' ...\n" << std::flush;
>
> // Read the vmlinux corpus and add it to the group.
> + status = abigail::elf_reader::STATUS_OK;
> t.start();
> read_and_add_corpus_to_group_from_elf(ctxt.get(), *group, status);
> t.stop();
> @@ -2678,7 +2708,7 @@ maybe_load_vmlinux_ctf_corpus(corpus::origin origin,
> << t << "\n";
>
> if (group->is_empty())
> - return;
> + return status;
>
> // Now add the corpora of the modules to the corpus group.
> int total_nb_modules = modules.size();
> @@ -2707,6 +2737,7 @@ maybe_load_vmlinux_ctf_corpus(corpus::origin origin,
> << "' reading DONE: "
> << t << "\n";
> }
> + return status;
> }
> #endif
>
> I have also introduced a new function called
> tools_utils::dir_contains_ctf_archive to look for a file that ends with
> ".ctfa". This abstracts away the search for "vmlinux.ctfa" as I wasn't
> sure if those archives could exist for normal (non-kernel) binaries as
> well:
Ohh, perfect!, I'll use it in CTF reader to located the Linux archive file.
No. there is no `.ctfa' file for non-kernel binaries intead they have `.ctf'
section, I could implement a similary function to looks for `.ctf' section
using elf helpers and it can be used in `load_corpus_and_write_abixml'
implementing a similar algorithm as with when we are processing the Kernel,
looking for DWARF information, and if it is not present then, test if
`.ctf' section is in ELF file then extract it using CTF reader,
to avoid duplication use of:
abigail::ctf_reader::read_context_sptr ctxt
= abigail::ctf_reader::create_read_context(opts.in_file_path,
opts.prepared_di_root_paths,
env.get());
One for `opts.use_ctf' and other one when `STATUS_DEBUG_INFO_NOT_FOUND' is returned.
WDYT?
>
> diff --git a/src/abg-tools-utils.cc b/src/abg-tools-utils.cc
> index fe9ebc72..a23f08ea 100644
> --- a/src/abg-tools-utils.cc
> +++ b/src/abg-tools-utils.cc
> @@ -1694,6 +1694,23 @@ file_is_kernel_debuginfo_package(const string& file_name, file_type file_type)
> return result;
> }
>
> +/// Test if a directory contains a CTF archive.
> +///
> +/// @param directory the directory to consider.
> +///
> +/// @param archive_prefix the prefix of the archive file.
> +///
> +/// @return true iff @p directory contains a CTF archive file.
> +bool
> +dir_contains_ctf_archive(const string& directory,
> + const string& archive_prefix)
> +{
> + string ctf_archive = directory + "/" + archive_prefix + ".ctfa";
> + if (file_exists(ctf_archive))
> + return true;
> + return false;
> +}
> +
>
> Then, now that the caller of these functions can know if loading the
> vmlinux binary succeeded or not, we can update that caller, which is
> build_corpus_group_from_kernel_dist_under:
>
> @@ -2786,14 +2817,23 @@ build_corpus_group_from_kernel_dist_under(const string& root,
> vector<char**> di_roots;
> di_roots.push_back(&di_root_ptr);
>
> - maybe_load_vmlinux_dwarf_corpus(origin, group, vmlinux,
> - modules, root, di_roots,
> - suppr_paths, kabi_wl_paths,
> - supprs, verbose, t, env);
> + abigail::elf_reader::status status =
> + maybe_load_vmlinux_dwarf_corpus(origin, group, vmlinux,
> + modules, root, di_roots,
> + suppr_paths, kabi_wl_paths,
> + supprs, verbose, t, env);
> #ifdef WITH_CTF
> + string vmlinux_basename;
> + base_name(vmlinux, vmlinux_basename);
> + if (origin == corpus::DWARF_ORIGIN
> + && (status & abigail::elf_reader::STATUS_DEBUG_INFO_NOT_FOUND)
> + && dir_contains_ctf_archive(root, vmlinux_basename))
> + // Let's try to load the binary using the CTF archive!
> + origin |= corpus::CTF_ORIGIN;
> +
> maybe_load_vmlinux_ctf_corpus(origin, group, vmlinux,
> - modules, root, di_roots,
> - verbose, t, env);
> + modules, root, di_roots,
> + verbose, t, env);
> #endif
> }
>
> If you feel like we shouldn't test if the directory 'root' contains a
> ctf archive before trying to load the CTF archive, then please remove
> the call to dir_contains_ctf_archive above. I just meant to show the
> general idea of where the decision should be made to try to load the ctf
> debug information, in my opinion.
>
Actually the `dir_contains_ctf_archive' prevents to rebuild/reset a group
in the CTF reader and discard the work done by DWARF reader if CTF information
is neither present in binary file.
> The general idea, however, is to make the reader of
> build_corpus_group_from_kernel_dist_under understand what is going on
> (that is, we try loading vmlinux with DWARF, and if it fails, then we
> try CTF) without having to go dig deep into how
> maybe_load_vmlinux_ctf_corpus works.
>
> Similarly, I amended the changes to the tools (abi{diff,dw,pkgdiff}) to
> make them follow the same path.
>
> [...]
>
>> diff --git a/tools/abidiff.cc b/tools/abidiff.cc
>> index e0bb35ac..815c68df 100644
>> --- a/tools/abidiff.cc
>> +++ b/tools/abidiff.cc
>> @@ -1118,6 +1118,13 @@ main(int argc, char* argv[])
>> return 0;
>> }
>>
>> + corpus::origin first_input_origin;
>> + corpus::origin origin =
>> +#ifdef WITH_CTF
>> + opts.use_ctf ? corpus::CTF_ORIGIN :
>> +#endif
>> + corpus::DWARF_ORIGIN;
>> +
>
> I removed these changes above. This is because we don't need to perform
> so many changes. We can just let the code as it was and add new code to
> just one place to test if loading DWARF failed. If it failed then we
> try to load CTF.
>
> See below.
>
>> prepare_di_root_paths(opts);
>>
>> if (!maybe_check_suppression_files(opts))
>> @@ -1150,7 +1157,7 @@ main(int argc, char* argv[])
>> abigail::elf_reader::status c1_status =
>> abigail::elf_reader::STATUS_OK,
>> c2_status = abigail::elf_reader::STATUS_OK;
>> - corpus_sptr c1, c2;
>> + corpus_sptr c1, c2, dwarf_corpus;
>
> Likewise, removed.
>
>> corpus_group_sptr g1, g2;
>> bool files_suppressed = false;
>>
>> @@ -1180,20 +1187,9 @@ main(int argc, char* argv[])
>> case abigail::tools_utils::FILE_TYPE_ELF: // fall through
>> case abigail::tools_utils::FILE_TYPE_AR:
>> {
>> -#ifdef WITH_CTF
>> - if (opts.use_ctf)
>> - {
>> - abigail::ctf_reader::read_context_sptr ctxt
>> - = abigail::ctf_reader::create_read_context(opts.file1,
>> - opts.prepared_di_root_paths1,
>> - env.get());
>> - ABG_ASSERT(ctxt);
>> - c1 = abigail::ctf_reader::read_corpus(ctxt.get(),
>> - c1_status);
>> - }
>> - else
>> -#endif
>
> I left this code in. So no change here either.
>
>> + if (origin & corpus::DWARF_ORIGIN)
>> {
>> + first_input_origin = corpus::DWARF_ORIGIN;
>
> I remove this change as well. So, still no change.
>
>> abigail::dwarf_reader::read_context_sptr ctxt =
>> abigail::dwarf_reader::create_read_context
>> (opts.file1, opts.prepared_di_root_paths1,
>> @@ -1205,6 +1201,7 @@ main(int argc, char* argv[])
>> set_suppressions(*ctxt, opts);
>> abigail::dwarf_reader::set_do_log(*ctxt, opts.do_log);
>> c1 = abigail::dwarf_reader::read_corpus_from_elf(*ctxt, c1_status);
>> + dwarf_corpus = c1;
>
> Likewise.
>
> But then, it's here that we are going to inspect c1_status to see if
> loading DWARF failed. If it failed, then we'll try to load CTF. So,
> here is the change I am adding to the process of loading the corpus c1:
>
>
> @@ -1205,6 +1208,36 @@ main(int argc, char* argv[])
> set_suppressions(*ctxt, opts);
> abigail::dwarf_reader::set_do_log(*ctxt, opts.do_log);
> c1 = abigail::dwarf_reader::read_corpus_from_elf(*ctxt, c1_status);
> +
> +#ifdef WITH_CTF
> + if (// We were not instructed to use CTF ...
> + !opts.use_ctf
This is always true, because we are in the else block of `opts.use_ctf'.
> + // ... and yet, no debug info was found ...
> + && (c1_status & STATUS_DEBUG_INFO_NOT_FOUND)
> + // ... but we found ELF symbols ...
> + && !(c1_status & STATUS_NO_SYMBOLS_FOUND))
> + {
> + string basenam, dirnam;
> + base_name(opts.file1, basenam);
> + dir_name(opts.file1, dirnam);
> + // ... the input file seems to contain CTF
> + // archive, so let's try to see if the file
> + // contains a CTF archive, who knows ...
> + if (dir_contains_ctf_archive(dirnam, basenam))
Non-kernel binaries contains `.ctf' section instead of `ctfa' file,
so I can implement a `file_contains_ctf_section' function to test if
it is a valid input file for CTF reader.
> + {
> + // The file does contain CTF debug information finally!
> + abigail::ctf_reader::read_context_sptr ctxt =
> + abigail::ctf_reader::create_read_context
> + (opts.file1,
> + opts.prepared_di_root_paths1,
> + env.get());
> + ABG_ASSERT(ctxt);
> +
> + c1 = abigail::ctf_reader::read_corpus(ctxt.get(),
> + c1_status);
> + }
> + }
> +#endif
>
> OK, maybe the statement
>
> "if (dir_contains_ctf_archive(dirnam, basenam))"
>
> above is not necessary as I am not sure if the CTF archive file is
> supposed to be present for normal binaries or not. If it's not
> necessary, then please remove that line as well as the use of the
> basenam/dirnam variables.
>
> But you get the general idea. We just test if loading DWARF failed
> (even if we were not instructed to use CTF) and then we try to load CTF.
> The change is at one place right after trying to load the DWARF,
> logically.
>
> I do something similar to the loading of the second corpus c2.
>
>
> Here is the entire change to the abidiff file:
>
> diff --git a/tools/abidiff.cc b/tools/abidiff.cc
> index e0bb35ac..0d9e59c2 100644
> --- a/tools/abidiff.cc
> +++ b/tools/abidiff.cc
> @@ -47,6 +47,9 @@ using abigail::suppr::suppressions_type;
> using abigail::suppr::read_suppressions;
> using namespace abigail::dwarf_reader;
> using namespace abigail::elf_reader;
> +using abigail::tools_utils::base_name;
> +using abigail::tools_utils::dir_name;
> +using abigail::tools_utils::dir_contains_ctf_archive;
> using abigail::tools_utils::emit_prefix;
> using abigail::tools_utils::check_file;
> using abigail::tools_utils::guess_file_type;
> @@ -1205,6 +1208,36 @@ main(int argc, char* argv[])
> set_suppressions(*ctxt, opts);
> abigail::dwarf_reader::set_do_log(*ctxt, opts.do_log);
> c1 = abigail::dwarf_reader::read_corpus_from_elf(*ctxt, c1_status);
> +
> +#ifdef WITH_CTF
> + if (// We were not instructed to use CTF ...
> + !opts.use_ctf
> + // ... and yet, no debug info was found ...
> + && (c1_status & STATUS_DEBUG_INFO_NOT_FOUND)
> + // ... but we found ELF symbols ...
> + && !(c1_status & STATUS_NO_SYMBOLS_FOUND))
> + {
> + string basenam, dirnam;
> + base_name(opts.file1, basenam);
> + dir_name(opts.file1, dirnam);
> + // ... the input file seems to contain CTF
> + // archive, so let's try to see if the file
> + // contains a CTF archive, who knows ...
> + if (dir_contains_ctf_archive(dirnam, basenam))
> + {
> + // The file does contain CTF debug information finally!
> + abigail::ctf_reader::read_context_sptr ctxt =
> + abigail::ctf_reader::create_read_context
> + (opts.file1,
> + opts.prepared_di_root_paths1,
> + env.get());
> + ABG_ASSERT(ctxt);
> +
> + c1 = abigail::ctf_reader::read_corpus(ctxt.get(),
> + c1_status);
> + }
> + }
> +#endif
> if (!c1
> || (opts.fail_no_debug_info
> && (c1_status & STATUS_ALT_DEBUG_INFO_NOT_FOUND)
> @@ -1212,7 +1245,7 @@ main(int argc, char* argv[])
> return handle_error(c1_status, ctxt.get(),
> argv[0], opts);
> }
> - }
> + }
> break;
> case abigail::tools_utils::FILE_TYPE_XML_CORPUS:
> {
> @@ -1289,13 +1322,44 @@ main(int argc, char* argv[])
> set_suppressions(*ctxt, opts);
>
> c2 = abigail::dwarf_reader::read_corpus_from_elf(*ctxt, c2_status);
> +
> +#ifdef WITH_CTF
> + if (// We were not instructed to use CTF ...
> + !opts.use_ctf
> + // ... and yet, no debug info was found ...
> + && (c2_status & STATUS_DEBUG_INFO_NOT_FOUND)
> + // ... but we found ELF symbols ...
> + && !(c2_status & STATUS_NO_SYMBOLS_FOUND))
> + {
> + string basenam, dirnam;
> + base_name(opts.file2, basenam);
> + dir_name(opts.file2, dirnam);
> + // ... the input file seems to contain CTF
> + // archive, so let's try to see if the file
> + // contains a CTF archive, who knows ...
> + if (dir_contains_ctf_archive(dirnam, basenam))
> + {
> + // The file does contain CTF debug information finally!
> + abigail::ctf_reader::read_context_sptr ctxt =
> + abigail::ctf_reader::create_read_context
> + (opts.file2,
> + opts.prepared_di_root_paths2,
> + env.get());
> + ABG_ASSERT(ctxt);
> +
> + c2 = abigail::ctf_reader::read_corpus(ctxt.get(),
> + c2_status);
> + }
> + }
> +#endif
> +
> if (!c2
> || (opts.fail_no_debug_info
> && (c2_status & STATUS_ALT_DEBUG_INFO_NOT_FOUND)
> && (c2_status & STATUS_DEBUG_INFO_NOT_FOUND)))
> return handle_error(c2_status, ctxt.get(), argv[0], opts);
> }
> - }
> + }
> break;
> case abigail::tools_utils::FILE_TYPE_XML_CORPUS:
> {
>
>
> By the way, I think there should be at least one test that exercises the
> workflow "try to load DWARF from the binary, fail, try to load CTF,
> succeed", without having the --ctf provided. And we should have that
> test for all the tools that were modified. We don't need to have that
> for a Linux Kernel, obviously, as that would be too big to have in the
> tarball.
>
OK, I'll add test cases.
> Below is the entire amended patch that I came up with. I suspect the
> use of the dir_contains_ctf_archive function everywhere is wrong, but
> I'll let you be the judge of that. I think the documentation of when
> that archive is present/necessary would be good to improve/add.
>
> Thanks again for the patch!
Thanks so much for comments!.
>
>
> From 481b547d7871a544b690943b16e6a173a729932f Mon Sep 17 00:00:00 2001
> [...]
Hello Guillermo,
"Guillermo E. Martinez" <guillermo.e.martinez@oracle.com> a écrit:
[...]
>> I have also introduced a new function called
>> tools_utils::dir_contains_ctf_archive to look for a file that ends with
>> ".ctfa". This abstracts away the search for "vmlinux.ctfa" as I wasn't
>> sure if those archives could exist for normal (non-kernel) binaries as
>> well:
>
> Ohh, perfect!, I'll use it in CTF reader to located the Linux archive file.
ACK.
[...]
>> @@ -2525,8 +2542,12 @@ get_binary_paths_from_kernel_dist(const string& dist_root,
>> /// @param t time to trace time spent in each step.
>> ///
>> /// @param env the environment to create the corpus_group in.
>> -static void
>> -maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
>> +///
>> +/// @return the status of the loading. If it's
>> +/// abigail::elf_reader::STATUS_UNKNOWN, then it means nothing was
>> +/// done, meaning the function got out early.
>> +static abigail::elf_reader::status
>> +maybe_load_vmlinux_dwarf_corpus(corpus::origin& origin,
>> corpus_group_sptr& group,
>> const string& vmlinux,
>> vector<string>& modules,
>> @@ -2539,10 +2560,11 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
>> timer& t,
>> environment_sptr& env)
>> {
>> + abigail::elf_reader::status status = abigail::elf_reader::STATUS_UNKNOWN;
>> +
>> if (!(origin & corpus::DWARF_ORIGIN))
>> - return;
>> + return status;
>>
>> - abigail::elf_reader::status status = abigail::elf_reader::STATUS_OK;
>> dwarf_reader::read_context_sptr ctxt;
>> ctxt =
>> dwarf_reader::create_read_context(vmlinux, di_roots, env.get(),
>> @@ -2569,6 +2591,7 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
>> << vmlinux << "' ...\n" << std::flush;
>>
>> // Read the vmlinux corpus and add it to the group.
>> + status = abigail::elf_reader::STATUS_OK;
>> t.start();
>> read_and_add_corpus_to_group_from_elf(*ctxt, *group, status);
>> t.stop();
>> @@ -2579,7 +2602,7 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
>> << t << "\n";
>>
>
> At this point if `vmlinux' file doesn't have DWARF information, the `status'
> returned by `maybe_load_vmlinux_dwarf_corpus' will set the bit field
> `STATUS_DEBUG_INFO_NOT_FOUND', but it is not verified here, and since vmlinux
> corpus was already added into the group in `read_debug_info_into_corpus'
> function, it continues processing modules without the main corpus information,
I see. You are right. Yes, the debug info is not found in vmlinux and yet the
whole thing continues, collecting just information from the ELF symbol
table, basically, and from the modules. Pretty useless, I guess.
> Is this the expected behaviour?
Hehe, no :-)
I guess maybe the caller should look for the .debug_info section in the
vmlinux section (or for split debug info), prior to even calling
maybe_load_vmlinux_dwarf_corpus. If there is no debug info, then the
function should proceed directly to calling
maybe_load_vmlinux_ctf_corpus? What do you think?
[...]
>> I have also introduced a new function called
>> tools_utils::dir_contains_ctf_archive to look for a file that ends with
>> ".ctfa". This abstracts away the search for "vmlinux.ctfa" as I wasn't
>> sure if those archives could exist for normal (non-kernel) binaries as
>> well:
>
> Ohh, perfect!, I'll use it in CTF reader to located the Linux archive file.
> No. there is no `.ctfa' file for non-kernel binaries intead they have `.ctf'
> section, I could implement a similary function to looks for `.ctf' section
> using elf helpers
Right, abg-elf-helpers.h does have find_section_by_name. That can be
used to look for the debug info, I guess. However, we also need to
support finding the debug info when it's split out into a different
place, like when it's packaged in a separate debug-info package. Today,
abg-dwarf-reader.cc uses dwfl (dwarf front-end library, I believe) to do
this, as dwfl knows how to find the DWARF debug info, wherever it is.
You can see how this is done in read_context::load_debug_info(), in
abg-dwarf-reader.cc, around line 2654. Look for the comment "Look for
split debuginfo files". Basically, dwfl_module_getdwarf returns a
pointer to the debug info it's found, if it has found one. I think we
should split this logic out to make it re-usable somehow.
If you think this is worthwhile, I can think of splitting it out and
stick it into elf-helpers, maybe?
> and it can be used in `load_corpus_and_write_abixml'
> implementing a similar algorithm as with when we are processing the Kernel,
> looking for DWARF information, and if it is not present then, test if
> `.ctf' section is in ELF file then extract it using CTF reader,
> to avoid duplication use of:
>
> abigail::ctf_reader::read_context_sptr ctxt
> = abigail::ctf_reader::create_read_context(opts.in_file_path,
> opts.prepared_di_root_paths,
> env.get());
>
> One for `opts.use_ctf' and other one when `STATUS_DEBUG_INFO_NOT_FOUND' is returned.
> WDYT?
Yes, along with the testing for the presence of DWARF debug info, that
might be useful, indeed.
[...]
>> But then, it's here that we are going to inspect c1_status to see if
>> loading DWARF failed. If it failed, then we'll try to load CTF. So,
>> here is the change I am adding to the process of loading the corpus c1:
>>
>>
>> @@ -1205,6 +1208,36 @@ main(int argc, char* argv[])
>> set_suppressions(*ctxt, opts);
>> abigail::dwarf_reader::set_do_log(*ctxt, opts.do_log);
>> c1 = abigail::dwarf_reader::read_corpus_from_elf(*ctxt, c1_status);
>> +
>> +#ifdef WITH_CTF
>> + if (// We were not instructed to use CTF ...
>> + !opts.use_ctf
>
> This is always true, because we are in the else block of `opts.use_ctf'.
Right.
>
>> + // ... and yet, no debug info was found ...
>> + && (c1_status & STATUS_DEBUG_INFO_NOT_FOUND)
>> + // ... but we found ELF symbols ...
>> + && !(c1_status & STATUS_NO_SYMBOLS_FOUND))
>> + {
>> + string basenam, dirnam;
>> + base_name(opts.file1, basenam);
>> + dir_name(opts.file1, dirnam);
>> + // ... the input file seems to contain CTF
>> + // archive, so let's try to see if the file
>> + // contains a CTF archive, who knows ...
>> + if (dir_contains_ctf_archive(dirnam, basenam))
>
> Non-kernel binaries contains `.ctf' section instead of `ctfa' file,
> so I can implement a `file_contains_ctf_section' function to test if
> it is a valid input file for CTF reader.
Great, thanks.
OK, I'll look into trying to put together some facility to look for the
presence of DWARF debug info, so tools can decide ahead of time what
front-end to use.
[...]
Cheers,
Hey again Guillermo,
So, even before you sent the patch, I was "playing" with the idea of
re-organizing the code of the readers to make front-ends/readers be
first class citizens in libabigail.
The idea is to have all the front-ends share/implement an abstract
interface: the Front End Interface, named abigail::fe_iface. So there
would be abigail::elf_reader::reader, abigail::dwarf_reader::reader and
abigail::ctf_reader::reader and abigtail::abixml_reader::reader
front-ends, all implementing the abigail::fe_iface. They would all have
to grok their intended input, build the IR as a result and pass it to
the middle-end.
Most of the craft of analyzing the ELF part would be of course in
located in the abigail::elf_reader::reader type.
The abigail::{dwarf,ctf}_reader::reader types would just have to use the
elf_reader::reader type to handle ELF stuff.
An example of what that would look like is in the
"users/dodji/front-end" branch at
https://sourceware.org/git/?p=libabigail.git;a=shortlog;h=refs/heads/users/dodji/front-end.
The interface of the new elf-reader type is at
https://sourceware.org/git/?p=libabigail.git;a=blob;f=include/abg-elf-reader.h;hb=refs/heads/users/dodji/front-end.
The abigail::fe_iface interface it implements is at
https://sourceware.org/git/?p=libabigail.git;a=blob;f=include/abg-fe-iface.h;hb=refs/heads/users/dodji/front-end.
It's not yet documented, but after the discussion we had on your patch,
I tried to implement a helper function
abigail::tools_utils::file_has_dwarf_debug_info() using the new
abigail::elf_reader::reader type.
You can see it at
https://sourceware.org/git/?p=libabigail.git;a=blob;f=src/abg-tools-utils.cc;hb=refs/heads/users/dodji/front-end#l423
So, that function file_has_dwarf_debug_info() can be used to determine
if an ELF file has debug info (even taking into account split debug
information). Based on that information, the tool could chose the right
front-end to use.
This is still work-in-progress, but the branch passes "make distcheck",
for what it's worth.
If you are interested, maybe you could base the subsequent versions of
your patch on this branch? What do you think?
In any case, I'll keep exploring this topic on that branch and I'll send
a message to the list a bit later to present it a little bit further.
Cheers,
On 10/6/22 02:42, Dodji Seketeli wrote:
> Hello Guillermo,
>
Hello Dodji,
Thanks for your comments!
> "Guillermo E. Martinez" <guillermo.e.martinez@oracle.com> a écrit:
>
> [...]
>
>>> I have also introduced a new function called
>>> tools_utils::dir_contains_ctf_archive to look for a file that ends with
>>> ".ctfa". This abstracts away the search for "vmlinux.ctfa" as I wasn't
>>> sure if those archives could exist for normal (non-kernel) binaries as
>>> well:
>>
>> Ohh, perfect!, I'll use it in CTF reader to located the Linux archive file.
>
> ACK.
>
> [...]
>
>>> @@ -2525,8 +2542,12 @@ get_binary_paths_from_kernel_dist(const string& dist_root,
>>> /// @param t time to trace time spent in each step.
>>> ///
>>> /// @param env the environment to create the corpus_group in.
>>> -static void
>>> -maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
>>> +///
>>> +/// @return the status of the loading. If it's
>>> +/// abigail::elf_reader::STATUS_UNKNOWN, then it means nothing was
>>> +/// done, meaning the function got out early.
>>> +static abigail::elf_reader::status
>>> +maybe_load_vmlinux_dwarf_corpus(corpus::origin& origin,
>>> corpus_group_sptr& group,
>>> const string& vmlinux,
>>> vector<string>& modules,
>>> @@ -2539,10 +2560,11 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
>>> timer& t,
>>> environment_sptr& env)
>>> {
>>> + abigail::elf_reader::status status = abigail::elf_reader::STATUS_UNKNOWN;
>>> +
>>> if (!(origin & corpus::DWARF_ORIGIN))
>>> - return;
>>> + return status;
>>>
>>> - abigail::elf_reader::status status = abigail::elf_reader::STATUS_OK;
>>> dwarf_reader::read_context_sptr ctxt;
>>> ctxt =
>>> dwarf_reader::create_read_context(vmlinux, di_roots, env.get(),
>>> @@ -2569,6 +2591,7 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
>>> << vmlinux << "' ...\n" << std::flush;
>>>
>>> // Read the vmlinux corpus and add it to the group.
>>> + status = abigail::elf_reader::STATUS_OK;
>>> t.start();
>>> read_and_add_corpus_to_group_from_elf(*ctxt, *group, status);
>>> t.stop();
>>> @@ -2579,7 +2602,7 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
>>> << t << "\n";
>>>
>>
>> At this point if `vmlinux' file doesn't have DWARF information, the `status'
>> returned by `maybe_load_vmlinux_dwarf_corpus' will set the bit field
>> `STATUS_DEBUG_INFO_NOT_FOUND', but it is not verified here, and since vmlinux
>> corpus was already added into the group in `read_debug_info_into_corpus'
>> function, it continues processing modules without the main corpus information,
>
> I see. You are right. Yes, the debug info is not found in vmlinux and yet the
> whole thing continues, collecting just information from the ELF symbol
> table, basically, and from the modules. Pretty useless, I guess.
>
>> Is this the expected behaviour?
>
> Hehe, no :-)
>
> I guess maybe the caller should look for the .debug_info section in the
> vmlinux section (or for split debug info), prior to even calling
> maybe_load_vmlinux_dwarf_corpus. If there is no debug info, then the
> function should proceed directly to calling
> maybe_load_vmlinux_ctf_corpus? What do you think?
>
Yes, it sounds good!!, just I would like to know your opinion about of what
will happen when neither DWARF nor CTF debug information is found, the current
behavior is to extract symbols information and compare them, so which symbol
information should I use DWARF::symtab or CTF::symtab?
And an additional use case is whether the tools `kmidiff', `abidiff' could
compare a DWARF IR with CTF IR? I exercised it with some libraries and binaries
using `abidiff' (finding a couple of problems in CTF reader (already fixed)
and three possible issues for DWARF, I will submit information and the test
cases about those) but in general seems to be work!, but before to continue
I would like to know your thoughts.
> [...]
>
>>> I have also introduced a new function called
>>> tools_utils::dir_contains_ctf_archive to look for a file that ends with
>>> ".ctfa". This abstracts away the search for "vmlinux.ctfa" as I wasn't
>>> sure if those archives could exist for normal (non-kernel) binaries as
>>> well:
>>
>> Ohh, perfect!, I'll use it in CTF reader to located the Linux archive file.
>> No. there is no `.ctfa' file for non-kernel binaries intead they have `.ctf'
>> section, I could implement a similary function to looks for `.ctf' section
>> using elf helpers
>
> Right, abg-elf-helpers.h does have find_section_by_name. That can be
> used to look for the debug info, I guess. However, we also need to
> support finding the debug info when it's split out into a different
> place, like when it's packaged in a separate debug-info package. Today,
> abg-dwarf-reader.cc uses dwfl (dwarf front-end library, I believe) to do
> this, as dwfl knows how to find the DWARF debug info, wherever it is.
>
Ohh, your are right, I saw `find_alt_debug_info', and in case of CTF front-end
we don't use dwfl, it is done by `find_alt_debuginfo', reading directly the content
of `.gnu_debuglink' section, I'm not sure if CTF reader can use `find_alt_debug_info'
because it calls dwfl_* functions seems be DWARF specific. So I'll investigate.
> You can see how this is done in read_context::load_debug_info(), in
> abg-dwarf-reader.cc, around line 2654. Look for the comment "Look for
> split debuginfo files". Basically, dwfl_module_getdwarf returns a
> pointer to the debug info it's found, if it has found one. I think we
> should split this logic out to make it re-usable somehow.
>
> If you think this is worthwhile, I can think of splitting it out and
> stick it into elf-helpers, maybe?
>
It will be really useful!, but I'm not sure `dwfl_module_getdwarf'
can operate in ELF without `.debug_*' sections.
>
>> and it can be used in `load_corpus_and_write_abixml'
>> implementing a similar algorithm as with when we are processing the Kernel,
>> looking for DWARF information, and if it is not present then, test if
>> `.ctf' section is in ELF file then extract it using CTF reader,
>> to avoid duplication use of:
>>
>> abigail::ctf_reader::read_context_sptr ctxt
>> = abigail::ctf_reader::create_read_context(opts.in_file_path,
>> opts.prepared_di_root_paths,
>> env.get());
>>
>> One for `opts.use_ctf' and other one when `STATUS_DEBUG_INFO_NOT_FOUND' is returned.
>> WDYT?
>
> Yes, along with the testing for the presence of DWARF debug info, that
> might be useful, indeed.
>
Agree.
> [...]
>
>>> But then, it's here that we are going to inspect c1_status to see if
>>> loading DWARF failed. If it failed, then we'll try to load CTF. So,
>>> here is the change I am adding to the process of loading the corpus c1:
>>>
>>>
>>> @@ -1205,6 +1208,36 @@ main(int argc, char* argv[])
>>> set_suppressions(*ctxt, opts);
>>> abigail::dwarf_reader::set_do_log(*ctxt, opts.do_log);
>>> c1 = abigail::dwarf_reader::read_corpus_from_elf(*ctxt, c1_status);
>>> +
>>> +#ifdef WITH_CTF
>>> + if (// We were not instructed to use CTF ...
>>> + !opts.use_ctf
>>
>> This is always true, because we are in the else block of `opts.use_ctf'.
>
> Right.
>
>>
>>> + // ... and yet, no debug info was found ...
>>> + && (c1_status & STATUS_DEBUG_INFO_NOT_FOUND)
>>> + // ... but we found ELF symbols ...
>>> + && !(c1_status & STATUS_NO_SYMBOLS_FOUND))
>>> + {
>>> + string basenam, dirnam;
>>> + base_name(opts.file1, basenam);
>>> + dir_name(opts.file1, dirnam);
>>> + // ... the input file seems to contain CTF
>>> + // archive, so let's try to see if the file
>>> + // contains a CTF archive, who knows ...
>>> + if (dir_contains_ctf_archive(dirnam, basenam))
>>
>> Non-kernel binaries contains `.ctf' section instead of `ctfa' file,
>> so I can implement a `file_contains_ctf_section' function to test if
>> it is a valid input file for CTF reader.
>
> Great, thanks.
>
> OK, I'll look into trying to put together some facility to look for the
> presence of DWARF debug info, so tools can decide ahead of time what
> front-end to use.
>
Really nice!.
> [...]
>
> Cheers,
>
Regards,
guillermo
Hello Guillermo,
Guillermo Martinez <guillermo.e.martinez@oracle.com> a écrit:
> Yes, it sounds good!!, just I would like to know your opinion about of what
> will happen when neither DWARF nor CTF debug information is found, the current
> behavior is to extract symbols information and compare them, so which symbol
> information should I use DWARF::symtab or CTF::symtab?
In the new branch users/dodji/front-end, both the CTF and DWARF readers
are going to be (re)using the same ELF Reader type. That was one of the
goals of that branch. And so the symtab object is going to come from
there, so which ever you chose right now, it won't matter in the end
when the "front-end" branch is merged into master.
> And an additional use case is whether the tools `kmidiff', `abidiff' could
> compare a DWARF IR with CTF IR?
Why not? We compare ABIXML againt DWARF or CTF already. So why not
trying to compare CTF against DWARF, see what breaks and then try to fix
it? :-)
> I exercised it with some libraries and binaries using `abidiff'
> (finding a couple of problems in CTF reader (already fixed) and three
> possible issues for DWARF, I will submit information and the test
> cases about those) but in general seems to be work!, but before to
> continue I would like to know your thoughts.
There you go \o/ I think it's a good idea.
[...]
>> Right, abg-elf-helpers.h does have find_section_by_name. That can be
>> used to look for the debug info, I guess. However, we also need to
>> support finding the debug info when it's split out into a different
>> place, like when it's packaged in a separate debug-info package. Today,
>> abg-dwarf-reader.cc uses dwfl (dwarf front-end library, I believe) to do
>> this, as dwfl knows how to find the DWARF debug info, wherever it is.
>>
>
> Ohh, your are right, I saw `find_alt_debug_info', and in case of CTF front-end
> we don't use dwfl, it is done by `find_alt_debuginfo', reading directly the content
> of `.gnu_debuglink' section, I'm not sure if CTF reader can use `find_alt_debug_info'
> because it calls dwfl_* functions seems be DWARF specific. So I'll investigate.
>
>> You can see how this is done in read_context::load_debug_info(), in
>> abg-dwarf-reader.cc, around line 2654. Look for the comment "Look for
>> split debuginfo files". Basically, dwfl_module_getdwarf returns a
>> pointer to the debug info it's found, if it has found one. I think we
>> should split this logic out to make it re-usable somehow.
>>
>> If you think this is worthwhile, I can think of splitting it out and
>> stick it into elf-helpers, maybe?
>>
>
> It will be really useful!, but I'm not sure `dwfl_module_getdwarf'
> can operate in ELF without `.debug_*' sections.
In the "front-end" branch, I've put the new file_has_dwarf_debug_info
for you to see at
https://sourceware.org/git/?p=libabigail.git;a=blob;f=src/abg-tools-utils.cc;hb=refs/heads/users/dodji/front-end#l423
Maybe the new elf_reader::reader type (in the new src/abg-elf-reader.cc
file in the "front-end" branch) should grow a new
elf_reader::reader::ctf_debug_info() member function that returns a
pointer to the CTF object, just like what
elf_reader::reader::dwarf_debug_info does? That function would look
into the .ctf section, or use file_alt_debuginfo to locate the
.gnu_debuglink section and use its content to locate the split debuginfo
file.
I think it's fairly doable. What do you think?
[...]
Cheers,
On 10/6/22 09:12, Dodji Seketeli wrote:
> Hey again Guillermo,
>
> So, even before you sent the patch, I was "playing" with the idea of
> re-organizing the code of the readers to make front-ends/readers be
> first class citizens in libabigail.
>
Ohh, cool.
> The idea is to have all the front-ends share/implement an abstract
> interface: the Front End Interface, named abigail::fe_iface. So there
> would be abigail::elf_reader::reader, abigail::dwarf_reader::reader and
> abigail::ctf_reader::reader and abigtail::abixml_reader::reader
> front-ends, all implementing the abigail::fe_iface. They would all have
> to grok their intended input, build the IR as a result and pass it to
> the middle-end.
>
OK, perfect!.
> Most of the craft of analyzing the ELF part would be of course in
> located in the abigail::elf_reader::reader type.
>
> The abigail::{dwarf,ctf}_reader::reader types would just have to use the
> elf_reader::reader type to handle ELF stuff.
>
OK.
> An example of what that would look like is in the
> "users/dodji/front-end" branch at
> https://sourceware.org/git/?p=libabigail.git;a=shortlog;h=refs/heads/users/dodji/front-end.
>
> The interface of the new elf-reader type is at
> https://sourceware.org/git/?p=libabigail.git;a=blob;f=include/abg-elf-reader.h;hb=refs/heads/users/dodji/front-end.
>
> The abigail::fe_iface interface it implements is at
> https://sourceware.org/git/?p=libabigail.git;a=blob;f=include/abg-fe-iface.h;hb=refs/heads/users/dodji/front-end.
>
> It's not yet documented, but after the discussion we had on your patch,
> I tried to implement a helper function
> abigail::tools_utils::file_has_dwarf_debug_info() using the new
> abigail::elf_reader::reader type.
>
> You can see it at
> https://sourceware.org/git/?p=libabigail.git;a=blob;f=src/abg-tools-utils.cc;hb=refs/heads/users/dodji/front-end#l423
>
> So, that function file_has_dwarf_debug_info() can be used to determine
> if an ELF file has debug info (even taking into account split debug
> information). Based on that information, the tool could chose the right
> front-end to use.
>
Perfect!.
> This is still work-in-progress, but the branch passes "make distcheck",
> for what it's worth.
>
> If you are interested, maybe you could base the subsequent versions of
> your patch on this branch? What do you think?
>
Totally agree.
> In any case, I'll keep exploring this topic on that branch and I'll send
> a message to the list a bit later to present it a little bit further.
>
Thanks!.
> Cheers,
>
On 10/7/22 06:38, Dodji Seketeli wrote:
> Hello Guillermo,
>
> Guillermo Martinez <guillermo.e.martinez@oracle.com> a écrit:
>
>> Yes, it sounds good!!, just I would like to know your opinion about of what
>> will happen when neither DWARF nor CTF debug information is found, the current
>> behavior is to extract symbols information and compare them, so which symbol
>> information should I use DWARF::symtab or CTF::symtab?
> In the new branch users/dodji/front-end, both the CTF and DWARF readers
> are going to be (re)using the same ELF Reader type. That was one of the
> goals of that branch. And so the symtab object is going to come from
> there, so which ever you chose right now, it won't matter in the end
> when the "front-end" branch is merged into master.
>
>> And an additional use case is whether the tools `kmidiff', `abidiff' could
>> compare a DWARF IR with CTF IR?
> Why not? We compare ABIXML againt DWARF or CTF already. So why not
> trying to compare CTF against DWARF, see what breaks and then try to fix
> it? :-)
I'm strongly in favor of this.
>
>> I exercised it with some libraries and binaries using `abidiff'
>> (finding a couple of problems in CTF reader (already fixed) and three
>> possible issues for DWARF, I will submit information and the test
>> cases about those) but in general seems to be work!, but before to
>> continue I would like to know your thoughts.
> There you go \o/ I think it's a good idea.
>
> [...]
>
>>> Right, abg-elf-helpers.h does have find_section_by_name. That can be
>>> used to look for the debug info, I guess. However, we also need to
>>> support finding the debug info when it's split out into a different
>>> place, like when it's packaged in a separate debug-info package. Today,
>>> abg-dwarf-reader.cc uses dwfl (dwarf front-end library, I believe) to do
>>> this, as dwfl knows how to find the DWARF debug info, wherever it is.
>>>
>> Ohh, your are right, I saw `find_alt_debug_info', and in case of CTF front-end
>> we don't use dwfl, it is done by `find_alt_debuginfo', reading directly the content
>> of `.gnu_debuglink' section, I'm not sure if CTF reader can use `find_alt_debug_info'
>> because it calls dwfl_* functions seems be DWARF specific. So I'll investigate.
>>
>>> You can see how this is done in read_context::load_debug_info(), in
>>> abg-dwarf-reader.cc, around line 2654. Look for the comment "Look for
>>> split debuginfo files". Basically, dwfl_module_getdwarf returns a
>>> pointer to the debug info it's found, if it has found one. I think we
>>> should split this logic out to make it re-usable somehow.
>>>
>>> If you think this is worthwhile, I can think of splitting it out and
>>> stick it into elf-helpers, maybe?
>>>
>> It will be really useful!, but I'm not sure `dwfl_module_getdwarf'
>> can operate in ELF without `.debug_*' sections.
> In the "front-end" branch, I've put the new file_has_dwarf_debug_info
> for you to see at
> https://sourceware.org/git/?p=libabigail.git;a=blob;f=src/abg-tools-utils.cc;hb=refs/heads/users/dodji/front-end#l423
>
> Maybe the new elf_reader::reader type (in the new src/abg-elf-reader.cc
> file in the "front-end" branch) should grow a new
> elf_reader::reader::ctf_debug_info() member function that returns a
> pointer to the CTF object, just like what
> elf_reader::reader::dwarf_debug_info does? That function would look
> into the .ctf section, or use file_alt_debuginfo to locate the
> .gnu_debuglink section and use its content to locate the split debuginfo
> file.
>
> I think it's fairly doable. What do you think?
>
> [...]
>
> Cheers,
>
@@ -12,11 +12,12 @@ This tool can also compare the textual representations of the ABI of
two ELF binaries (as emitted by ``abidw``) or an ELF binary against a
textual representation of another ELF binary.
-For a comprehensive ABI change report that includes changes about
-function and variable sub-types, the two input shared libraries must
-be accompanied with their debug information in `DWARF`_ format.
-Otherwise, only `ELF`_ symbols that were added or removed are
-reported.
+For a comprehensive ABI change report between two input shared
+libraries that includes changes about function and variable sub-types,
+``abidiff`` uses by default, debug information in `DWARF`_ format, if
+present, otherwise it compares interfaces using debug information in
+`CTF`_ format, if present, finally, if neither is found, it uses only
+`ELF`_ symbols to report which of them were added or removed.
.. include:: tools-use-libabigail.txt
@@ -558,7 +559,7 @@ Options
* ``--ctf``
- When comparing binaries, extract ABI information from CTF debug
+ When comparing binaries, extract ABI information from `CTF`_ debug
information, if present.
* ``--stats``
@@ -785,4 +786,4 @@ Usage examples
.. _ELF: http://en.wikipedia.org/wiki/Executable_and_Linkable_Format
.. _DWARF: http://www.dwarfstd.org
-
+.. _CTF: https://raw.githubusercontent.com/wiki/oracle/binutils-gdb/files/ctf-spec.pdf
@@ -8,8 +8,7 @@ representation of its ABI to standard output. The emitted
representation format, named ``ABIXML``, includes all the globally
defined functions and variables, along with a complete representation
of their types. It also includes a representation of the globally
-defined ELF symbols of the file. The input shared library must
-contain associated debug information in `DWARF`_ format.
+defined ELF symbols of the file.
When given the ``--linux-tree`` option, this program can also handle a
`Linux kernel`_ tree. That is, a directory tree that contains both
@@ -19,8 +18,13 @@ interface between the kernel and its module, to standard output. In
this case, we don't call it an ABI, but a KMI (Kernel Module
Interface). The emitted KMI includes all the globally defined
functions and variables, along with a complete representation of their
-types. The input binaries must contain associated debug information
-in `DWARF`_ format.
+types.
+
+To generate either ABI or KMI representation, by default ``abidw``
+uses debug information in `DWARF`_ format, if present, otherwise it
+looks for debug information in `CTF`_ format, if present, finally, if
+neither is found, it uses only `ELF`_ symbols to report which of them
+were added or removed.
.. include:: tools-use-libabigail.txt
@@ -326,7 +330,7 @@ Options
* ``--ctf``
- Extract ABI information from CTF debug information, if present in
+ Extract ABI information from `CTF`_ debug information, if present in
the given object.
* ``--annotate``
@@ -365,3 +369,4 @@ standard `here
.. _DWARF: http://www.dwarfstd.org
.. _GNU: http://www.gnu.org
.. _Linux Kernel: https://kernel.org/
+.. _CTF: https://raw.githubusercontent.com/wiki/oracle/binutils-gdb/files/ctf-spec.pdf
@@ -13,12 +13,18 @@ binaries.
For a comprehensive ABI change report that includes changes about
function and variable sub-types, the two input packages must be
accompanied with their debug information packages that contain debug
-information either in `DWARF`_ or in `CTF` formats. Please note
+information either in `DWARF`_ or in `CTF`_ formats. Please note
however that some packages contain binaries that embed the debug
information directly in a section of said binaries. In those cases,
obviously, no separate debug information package is needed as the tool
will find the debug information inside the binaries.
+By default, ``abipkgdiff`` uses debug information in `DWARF`_ format,
+if present, otherwise it compares binaries interfaces using debug
+information in `CTF`_ format, if present, finally, if neither is
+found, it uses only `ELF`_ symbols to report which of them were added
+or removed.
+
.. include:: tools-use-libabigail.txt
.. _abipkgdiff_invocation_label:
@@ -525,8 +531,8 @@ Options
* ``--ctf``
- This is used to compare packages with CTF debug information, if
- present.
+ This is used to compare packages with `CTF`_ debug information,
+ if present.
.. _abipkgdiff_return_value_label:
@@ -546,4 +552,5 @@ In the later case, the value of the exit code is the same as for the
.. _Deb: https://en.wikipedia.org/wiki/Deb_%28file_format%29
.. _tar: https://en.wikipedia.org/wiki/Tar_%28computing%29
.. _DWARF: http://www.dwarfstd.org
+.. _CTF: https://raw.githubusercontent.com/wiki/oracle/binutils-gdb/files/ctf-spec.pdf
.. _Development Package: https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Devel_Packages
@@ -74,6 +74,11 @@ functions and variables) between the Kernel and its modules. In
practice, though, some users might want to compare a subset of the
those interfaces.
+By default, ``kmidiff`` uses debug information in `DWARF`_ format,
+if present, otherwise it compares interfaces using debug information
+in `CTF`_ format, if present, finally, if neither is found, it uses
+only `ELF`_ symbols to report which were added or removed.
+
Users can then define a "white list" of the interfaces to compare.
Such a white list is a just a file in the "INI" format that looks
like: ::
@@ -174,7 +179,7 @@ Options
* ``--ctf``
- Extract ABI information from CTF debug information, if present in
+ Extract ABI information from `CTF`_ debug information, if present in
the Kernel and Modules.
* ``--impacted-interfaces | -i``
@@ -242,3 +247,5 @@ Options
.. _ELF: http://en.wikipedia.org/wiki/Executable_and_Linkable_Format
.. _ksymtab: http://en.wikipedia.org/wiki/Executable_and_Linkable_Format
.. _Linux Kernel: https://kernel.org
+.. _DWARF: http://www.dwarfstd.org
+.. _CTF: https://raw.githubusercontent.com/wiki/oracle/binutils-gdb/files/ctf-spec.pdf
@@ -45,6 +45,7 @@ namespace ctf_reader
using std::dynamic_pointer_cast;
using abigail::tools_utils::dir_name;
using abigail::tools_utils::file_exists;
+using abigail::tools_utils::base_name;
class read_context
{
@@ -1545,7 +1546,12 @@ slurp_elf_info(read_context *ctxt,
corp->set_architecture_name(elf_helpers::e_machine_to_string(ehdr->e_machine));
find_alt_debuginfo(ctxt, &ctf_dbg_scn);
- ABG_ASSERT(ctxt->symtab);
+ if (!ctxt->symtab)
+ {
+ status |= elf_reader::STATUS_NO_SYMBOLS_FOUND;
+ return;
+ }
+
corp->set_symtab(ctxt->symtab);
if ((corp->get_origin() & corpus::LINUX_KERNEL_BINARY_ORIGIN)
@@ -1564,7 +1570,8 @@ slurp_elf_info(read_context *ctxt,
ctf_scn = ctf_dbg_scn;
else
{
- status |= elf_reader::STATUS_DEBUG_INFO_NOT_FOUND;
+ status |= (elf_reader::STATUS_OK |
+ elf_reader::STATUS_DEBUG_INFO_NOT_FOUND);
return;
}
}
@@ -1676,15 +1683,15 @@ read_corpus(read_context *ctxt, elf_reader::status &status)
origin |= corpus::LINUX_KERNEL_BINARY_ORIGIN;
corp->set_origin(origin);
- if (ctxt->cur_corpus_group_)
- ctxt->cur_corpus_group_->add_corpus(ctxt->cur_corpus_);
-
slurp_elf_info(ctxt, corp, status);
- if (!is_linux_kernel
- && ((status & elf_reader::STATUS_DEBUG_INFO_NOT_FOUND) |
- (status & elf_reader::STATUS_NO_SYMBOLS_FOUND)))
+ if ((status & elf_reader::STATUS_NO_SYMBOLS_FOUND) ||
+ (!is_linux_kernel &&
+ (status & elf_reader::STATUS_DEBUG_INFO_NOT_FOUND)))
return corp;
+ if (ctxt->cur_corpus_group_)
+ ctxt->cur_corpus_group_->add_corpus(ctxt->cur_corpus_);
+
// Set the set of exported declaration that are defined.
ctxt->exported_decls_builder
(ctxt->cur_corpus_->get_exported_decls_builder().get());
@@ -1695,9 +1702,13 @@ read_corpus(read_context *ctxt, elf_reader::status &status)
{
if (ctxt->ctfa == NULL)
{
- std::string ctfa_filename;
- if (find_ctfa_file(ctxt, ctfa_filename))
- ctxt->ctfa = ctf_arc_open(ctfa_filename.c_str(), &errp);
+ std::string filename;
+ base_name(ctxt->filename, filename);
+
+ // locate vmlinux.ctfa only when reader is processing
+ // vmlinux file, i.e the main corpus in the group.
+ if (filename == "vmlinux" && find_ctfa_file(ctxt, filename))
+ ctxt->ctfa = ctf_arc_open(filename.c_str(), &errp);
}
}
else
@@ -2493,6 +2493,11 @@ get_binary_paths_from_kernel_dist(const string& dist_root,
/// made of vmlinux kernel file and the linux kernel modules found
/// under @p root directory and under its sub-directories, recursively.
///
+/// If the vmlinux file doens't have DWARF info, it looks for
+/// vmlinux.ctfa, if it's present, it assumes that kernel was build
+/// with CTF support, then it updates @ref origin, given chance to
+/// CTF reader to build the IR for kernel build directory.
+///
/// @param origin the debug type information in vmlinux kernel and
/// the linux kernel modules to be used to build the corpora @p group.
///
@@ -2526,7 +2531,7 @@ get_binary_paths_from_kernel_dist(const string& dist_root,
///
/// @param env the environment to create the corpus_group in.
static void
-maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
+maybe_load_vmlinux_dwarf_corpus(corpus::origin &origin,
corpus_group_sptr& group,
const string& vmlinux,
vector<string>& modules,
@@ -2578,6 +2583,21 @@ maybe_load_vmlinux_dwarf_corpus(corpus::origin origin,
<< " reading DONE:"
<< t << "\n";
+ if (status & elf_reader::STATUS_DEBUG_INFO_NOT_FOUND)
+ {
+ // vmlinux doesn't have DWARF debug info, so it might contain
+ // CTF debug info, this means vmlinux.ctfa is under kernel
+ // build directory.
+ string ctfa_file = root + "/vmlinux.ctfa";
+ if (file_exists(ctfa_file))
+ {
+ // OK. Likely CTF could build a better IR, then let's
+ // notify the caller to try with CTF reader.
+ origin = corpus::CTF_ORIGIN;
+ return;
+ }
+ }
+
if (group->is_empty())
return;
@@ -0,0 +1,16 @@
+================ changes of 'libobj-v0.so'===============
+ Functions changes summary: 0 Removed, 1 Changed (1 filtered out), 0 Added functions
+ Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
+
+ 1 function with some indirect sub-type change:
+
+ [C] 'function void foo(S1*)' has some indirect sub-type changes:
+ parameter 1 of type 'S1*' has sub-type changes:
+ in pointed to type 'struct S1':
+ type size changed from 0 to 32 (in bits)
+ type alignment changed from 0 to 32
+ 1 data member insertion:
+ 'int mem2', at offset 0 (in bits)
+
+================ end of changes of 'libobj-v0.so'===============
+
@@ -855,7 +855,7 @@ static InOutSpec in_out_specs[] =
{ // Just like the previous tests, but loc info is emitted.
"data/test-diff-pkg-ctf/dirpkg-3-dir1",
"data/test-diff-pkg-ctf/dirpkg-3-dir2",
- "--no-default-suppression --no-abignore",
+ "--ctf --no-default-suppression --no-abignore",
"data/test-diff-pkg-ctf/dirpkg-3.suppr",
"",
"",
@@ -1118,6 +1118,13 @@ main(int argc, char* argv[])
return 0;
}
+ corpus::origin first_input_origin;
+ corpus::origin origin =
+#ifdef WITH_CTF
+ opts.use_ctf ? corpus::CTF_ORIGIN :
+#endif
+ corpus::DWARF_ORIGIN;
+
prepare_di_root_paths(opts);
if (!maybe_check_suppression_files(opts))
@@ -1150,7 +1157,7 @@ main(int argc, char* argv[])
abigail::elf_reader::status c1_status =
abigail::elf_reader::STATUS_OK,
c2_status = abigail::elf_reader::STATUS_OK;
- corpus_sptr c1, c2;
+ corpus_sptr c1, c2, dwarf_corpus;
corpus_group_sptr g1, g2;
bool files_suppressed = false;
@@ -1180,20 +1187,9 @@ main(int argc, char* argv[])
case abigail::tools_utils::FILE_TYPE_ELF: // fall through
case abigail::tools_utils::FILE_TYPE_AR:
{
-#ifdef WITH_CTF
- if (opts.use_ctf)
- {
- abigail::ctf_reader::read_context_sptr ctxt
- = abigail::ctf_reader::create_read_context(opts.file1,
- opts.prepared_di_root_paths1,
- env.get());
- ABG_ASSERT(ctxt);
- c1 = abigail::ctf_reader::read_corpus(ctxt.get(),
- c1_status);
- }
- else
-#endif
+ if (origin & corpus::DWARF_ORIGIN)
{
+ first_input_origin = corpus::DWARF_ORIGIN;
abigail::dwarf_reader::read_context_sptr ctxt =
abigail::dwarf_reader::create_read_context
(opts.file1, opts.prepared_di_root_paths1,
@@ -1205,6 +1201,7 @@ main(int argc, char* argv[])
set_suppressions(*ctxt, opts);
abigail::dwarf_reader::set_do_log(*ctxt, opts.do_log);
c1 = abigail::dwarf_reader::read_corpus_from_elf(*ctxt, c1_status);
+ dwarf_corpus = c1;
if (!c1
|| (opts.fail_no_debug_info
&& (c1_status & STATUS_ALT_DEBUG_INFO_NOT_FOUND)
@@ -1212,7 +1209,31 @@ main(int argc, char* argv[])
return handle_error(c1_status, ctxt.get(),
argv[0], opts);
}
- }
+#ifdef WITH_CTF
+ if (((c1_status & STATUS_OK) &&
+ (c1_status & STATUS_DEBUG_INFO_NOT_FOUND)) ||
+ (origin & corpus::CTF_ORIGIN))
+ {
+ first_input_origin = corpus::CTF_ORIGIN;
+ abigail::ctf_reader::read_context_sptr ctxt
+ = abigail::ctf_reader::create_read_context(opts.file1,
+ opts.prepared_di_root_paths1,
+ env.get());
+ ABG_ASSERT(ctxt);
+ c1 = abigail::ctf_reader::read_corpus(ctxt.get(),
+ c1_status);
+ // If CTF was not explicitly selected in the command line and
+ // there is no CTF information, then compare DWARF corpora
+ // by default only with symbols information.
+ if ((origin & corpus::DWARF_ORIGIN) &&
+ (c1_status & abigail::elf_reader::STATUS_DEBUG_INFO_NOT_FOUND))
+ {
+ first_input_origin = corpus::DWARF_ORIGIN;
+ c1 = dwarf_corpus;
+ }
+ }
+#endif
+ }
break;
case abigail::tools_utils::FILE_TYPE_XML_CORPUS:
{
@@ -1264,19 +1285,7 @@ main(int argc, char* argv[])
case abigail::tools_utils::FILE_TYPE_ELF: // Fall through
case abigail::tools_utils::FILE_TYPE_AR:
{
-#ifdef WITH_CTF
- if (opts.use_ctf)
- {
- abigail::ctf_reader::read_context_sptr ctxt
- = abigail::ctf_reader::create_read_context(opts.file2,
- opts.prepared_di_root_paths2,
- env.get());
- ABG_ASSERT(ctxt);
- c2 = abigail::ctf_reader::read_corpus(ctxt.get(),
- c2_status);
- }
- else
-#endif
+ if (first_input_origin & corpus::DWARF_ORIGIN)
{
abigail::dwarf_reader::read_context_sptr ctxt =
abigail::dwarf_reader::create_read_context
@@ -1295,7 +1304,19 @@ main(int argc, char* argv[])
&& (c2_status & STATUS_DEBUG_INFO_NOT_FOUND)))
return handle_error(c2_status, ctxt.get(), argv[0], opts);
}
- }
+#ifdef WITH_CTF
+ else
+ {
+ abigail::ctf_reader::read_context_sptr ctxt
+ = abigail::ctf_reader::create_read_context(opts.file2,
+ opts.prepared_di_root_paths2,
+ env.get());
+ ABG_ASSERT(ctxt);
+ c2 = abigail::ctf_reader::read_corpus(ctxt.get(),
+ c2_status);
+ }
+#endif
+ }
break;
case abigail::tools_utils::FILE_TYPE_XML_CORPUS:
{
@@ -1334,7 +1355,8 @@ main(int argc, char* argv[])
if (!!c1 != !!c2
|| !!t1 != !!t2
- || !!g1 != !!g2)
+ || !!g1 != !!g2
+ || c1_status != c2_status)
{
emit_prefix(argv[0], cerr)
<< "the two input should be of the same kind\n";
@@ -551,25 +551,15 @@ load_corpus_and_write_abixml(char* argv[],
// First of all, read a libabigail IR corpus from the file specified
// in OPTS.
- corpus_sptr corp;
+ corpus_sptr corp, dwarf_corpus;
elf_reader::status s = elf_reader::STATUS_UNKNOWN;
+ corpus::origin origin =
#ifdef WITH_CTF
- if (opts.use_ctf)
- {
- abigail::ctf_reader::read_context_sptr ctxt
- = abigail::ctf_reader::create_read_context(opts.in_file_path,
- opts.prepared_di_root_paths,
- env.get());
- assert (ctxt);
- t.start();
- corp = abigail::ctf_reader::read_corpus (ctxt, s);
- t.stop();
- if (opts.do_log)
- emit_prefix(argv[0], cerr)
- << "read corpus from elf file in: " << t << "\n";
- }
- else
+ opts.use_ctf ? corpus::CTF_ORIGIN :
#endif
+ corpus::DWARF_ORIGIN;
+
+ if (origin & corpus::DWARF_ORIGIN)
{
dwarf_reader::read_context_sptr c
= abigail::dwarf_reader::create_read_context(opts.in_file_path,
@@ -616,7 +606,7 @@ load_corpus_and_write_abixml(char* argv[],
}
if (opts.exported_interfaces_only.has_value())
- env->analyze_exported_interfaces_only(*opts.exported_interfaces_only);
+ env->analyze_exported_interfaces_only(*opts.exported_interfaces_only);
t.start();
corp = dwarf_reader::read_corpus_from_elf(ctxt, s);
@@ -632,7 +622,35 @@ load_corpus_and_write_abixml(char* argv[],
if (opts.do_log)
emit_prefix(argv[0], cerr)
<< "reset read context in: " << t << "\n";
+
+ dwarf_corpus = corp;
}
+#ifdef WITH_CTF
+ if (((s & abigail::elf_reader::STATUS_OK) &&
+ (s & elf_reader::STATUS_DEBUG_INFO_NOT_FOUND)) ||
+ (origin & corpus::CTF_ORIGIN))
+ {
+ abigail::ctf_reader::read_context_sptr ctxt
+ = abigail::ctf_reader::create_read_context(opts.in_file_path,
+ opts.prepared_di_root_paths,
+ env.get());
+ assert (ctxt);
+ t.start();
+ corp = abigail::ctf_reader::read_corpus (ctxt, s);
+ t.stop();
+
+ // If CTF was not explicitly selected in the command line and
+ // there is no CTF information, then compare DWARF corpora
+ // by default only with symbols information.
+ if ((origin & corpus::DWARF_ORIGIN) &&
+ (s & abigail::elf_reader::STATUS_DEBUG_INFO_NOT_FOUND))
+ corp = dwarf_corpus;
+
+ if (opts.do_log)
+ emit_prefix(argv[0], cerr)
+ << "read corpus from elf file in: " << t << "\n";
+ }
+#endif
// If we couldn't create a corpus, emit some (hopefully) useful
// diagnostics and return and error.
@@ -834,7 +852,7 @@ load_kernel_corpus_group_and_write_abixml(char* argv[],
global_timer.start();
t.start();
-corpus::origin origin =
+ corpus::origin origin =
#ifdef WITH_CTF
opts.use_ctf ? corpus::CTF_ORIGIN :
#endif
@@ -136,6 +136,7 @@ using abigail::tools_utils::build_corpus_group_from_kernel_dist_under;
using abigail::tools_utils::load_default_system_suppressions;
using abigail::tools_utils::load_default_user_suppressions;
using abigail::tools_utils::abidiff_status;
+using abigail::corpus;
using abigail::ir::corpus_sptr;
using abigail::ir::corpus_group_sptr;
using abigail::comparison::diff_context;
@@ -1320,33 +1321,54 @@ compare(const elf_file& elf1,
<< elf1.path
<< " ...\n";
- corpus_sptr corpus1;
+ corpus_sptr corpus1, dwarf_corpus;
+ corpus::origin first_input_origin;
+ corpus::origin origin =
+#ifdef WITH_CTF
+ opts.use_ctf ? corpus::CTF_ORIGIN :
+#endif
+ corpus::DWARF_ORIGIN;
+
#ifdef WITH_CTF
abigail::ctf_reader::read_context_sptr ctxt_ctf;
#endif
read_context_sptr ctxt_dwarf;
{
+ if (origin & corpus::DWARF_ORIGIN)
+ {
+ first_input_origin = corpus::DWARF_ORIGIN;
+ ctxt_dwarf = create_read_context(elf1.path, di_dirs1, env.get(),
+ /*load_all_types=*/opts.show_all_types);
+ add_read_context_suppressions(*ctxt_dwarf, priv_types_supprs1);
+ if (!opts.kabi_suppressions.empty())
+ add_read_context_suppressions(*ctxt_dwarf, opts.kabi_suppressions);
+
+ corpus1 = read_corpus_from_elf(*ctxt_dwarf, c1_status);
+ dwarf_corpus = corpus1;
+ }
#ifdef WITH_CTF
- if (opts.use_ctf)
+ if (((c1_status & abigail::elf_reader::STATUS_OK) &&
+ (c1_status & abigail::elf_reader::STATUS_DEBUG_INFO_NOT_FOUND)) ||
+ (origin & corpus::CTF_ORIGIN))
{
+ first_input_origin = corpus::CTF_ORIGIN;
ctxt_ctf = abigail::ctf_reader::create_read_context(elf1.path,
di_dirs1,
env.get());
ABG_ASSERT(ctxt_ctf);
corpus1 = abigail::ctf_reader::read_corpus(ctxt_ctf.get(),
c1_status);
+ // If CTF was not explicitly selected in the command line and
+ // there is no CTF information, then compare DWARF corpora
+ // by default only with symbols information.
+ if ((origin & corpus::DWARF_ORIGIN) &&
+ (c1_status & abigail::elf_reader::STATUS_DEBUG_INFO_NOT_FOUND))
+ {
+ first_input_origin = corpus::DWARF_ORIGIN;
+ corpus1 = dwarf_corpus;
+ }
}
- else
#endif
- {
- ctxt_dwarf = create_read_context(elf1.path, di_dirs1, env.get(),
- /*load_all_types=*/opts.show_all_types);
- add_read_context_suppressions(*ctxt_dwarf, priv_types_supprs1);
- if (!opts.kabi_suppressions.empty())
- add_read_context_suppressions(*ctxt_dwarf, opts.kabi_suppressions);
-
- corpus1 = read_corpus_from_elf(*ctxt_dwarf, c1_status);
- }
bool bail_out = false;
if (!(c1_status & abigail::elf_reader::STATUS_OK))
@@ -1392,11 +1414,7 @@ compare(const elf_file& elf1,
emit_prefix("abipkgdiff", cerr)
<< "Could not find alternate debug info file";
string alt_di_path;
-#ifdef WITH_CTF
- if (opts.use_ctf)
- ;
- else
-#endif
+ if (first_input_origin & corpus::DWARF_ORIGIN)
abigail::dwarf_reader::refers_to_alt_debug_info(*ctxt_dwarf,
alt_di_path);
if (!alt_di_path.empty())
@@ -1431,17 +1449,7 @@ compare(const elf_file& elf1,
corpus_sptr corpus2;
{
-#ifdef WITH_CTF
- if (opts.use_ctf)
- {
- ctxt_ctf = abigail::ctf_reader::create_read_context(elf2.path,
- di_dirs2,
- env.get());
- corpus2 = abigail::ctf_reader::read_corpus(ctxt_ctf.get(),
- c2_status);
- }
- else
-#endif
+ if (first_input_origin & corpus::DWARF_ORIGIN)
{
ctxt_dwarf = create_read_context(elf2.path, di_dirs2, env.get(),
/*load_all_types=*/opts.show_all_types);
@@ -1452,6 +1460,16 @@ compare(const elf_file& elf1,
corpus2 = read_corpus_from_elf(*ctxt_dwarf, c2_status);
}
+#ifdef WITH_CTF
+ else
+ {
+ ctxt_ctf = abigail::ctf_reader::create_read_context(elf2.path,
+ di_dirs2,
+ env.get());
+ corpus2 = abigail::ctf_reader::read_corpus(ctxt_ctf.get(),
+ c2_status);
+ }
+#endif
bool bail_out = false;
if (!(c2_status & abigail::elf_reader::STATUS_OK))
@@ -1497,11 +1515,8 @@ compare(const elf_file& elf1,
emit_prefix("abipkgdiff", cerr)
<< "Could not find alternate debug info file";
string alt_di_path;
-#ifdef WITH_CTF
- if (opts.use_ctf)
- ;
- else
-#endif
+
+ if (first_input_origin & corpus::DWARF_ORIGIN)
abigail::dwarf_reader::refers_to_alt_debug_info(*ctxt_dwarf,
alt_di_path);
if (!alt_di_path.empty())
@@ -1599,14 +1614,31 @@ compare_to_self(const elf_file& elf,
<< elf.path
<< " ...\n";
- corpus_sptr corp;
+ corpus_sptr corp, dwarf_corpus;
+ corpus::origin origin =
+#ifdef WITH_CTF
+ opts.use_ctf ? corpus::CTF_ORIGIN :
+#endif
+ corpus::DWARF_ORIGIN;
#ifdef WITH_CTF
abigail::ctf_reader::read_context_sptr ctxt_ctf;
#endif
+
read_context_sptr ctxt_dwarf;
{
+ if (origin & corpus::DWARF_ORIGIN)
+ {
+ ctxt_dwarf =
+ create_read_context(elf.path, di_dirs, env.get(),
+ /*read_all_types=*/opts.show_all_types);
+
+ corp = read_corpus_from_elf(*ctxt_dwarf, c_status);
+ dwarf_corpus = corp;
+ }
#ifdef WITH_CTF
- if (opts.use_ctf)
+ if (((c_status & abigail::elf_reader::STATUS_OK) &&
+ (c_status & abigail::elf_reader::STATUS_DEBUG_INFO_NOT_FOUND)) ||
+ (origin & corpus::CTF_ORIGIN))
{
ctxt_ctf = abigail::ctf_reader::create_read_context(elf.path,
di_dirs,
@@ -1614,16 +1646,14 @@ compare_to_self(const elf_file& elf,
ABG_ASSERT(ctxt_ctf);
corp = abigail::ctf_reader::read_corpus(ctxt_ctf.get(),
c_status);
+ // If CTF was not explicitly selected in the command line and
+ // there is no CTF information, then use DWARF corpus
+ // by default just with symbols information.
+ if ((origin & corpus::DWARF_ORIGIN) &&
+ (c_status & abigail::elf_reader::STATUS_DEBUG_INFO_NOT_FOUND))
+ corp = dwarf_corpus;
}
- else
#endif
- {
- ctxt_dwarf =
- create_read_context(elf.path, di_dirs, env.get(),
- /*read_all_types=*/opts.show_all_types);
-
- corp = read_corpus_from_elf(*ctxt_dwarf, c_status);
- }
if (!(c_status & abigail::elf_reader::STATUS_OK))
{