From patchwork Sun Apr 12 01:47:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Wielaard X-Patchwork-Id: 39086 From: mark@klomp.org (Mark Wielaard) Date: Sun, 12 Apr 2020 03:47:56 +0200 Subject: [PATCH 1/2] Add --header-file option to add individual public header files. In-Reply-To: References: Message-ID: <20200412014757.30688-1-mark@klomp.org> Sometimes public header files are in the same directory as private header files. In such cases --headers-dir cannot be used. Add --header-file to add individual public header files. * include/abg-tools-utils.h (gen_suppr_spec_from_headers): Add hdr_files string vector argument. * src/abg-tools-utils.cc (handle_file_entry): New function that adds one specific file to the type_suppression. Implementation lifted from... (handle_fts_entry): ...here. Call handle_file_entry for each file. (gen_suppr_spec_from_headers): Also takes a header_files string vector as argument. Call handle_file_entry for each file entry. * tools/abidiff.cc (options): Add header_files1 and header_files2 string vectors. (display_usage): Print --header-file1 and --header-file2 usage. (parse_command_line): Handle --header-file1, --hf1 and --header-file2, --hf2. (set_diff_context_from_opts): Call gen_suppr_spec_from_headers with header_files1 and header_files2. (set_suppressions): Likewise. * tools/abidw.cc (options): Add header_files string vector. (display_usage): Print --header-file usage. (parse_command_line): Handle --header-file1, --hf1. (maybe_check_header_files): New function. (set_suppressions): Call gen_suppr_spec_from_headers with header_files. (main): Call maybe_check_header_files. * tools/abilint.cc (options): Add header_files string vector. (display_usage): Print --header-file usage. (parse_command_line): Handle --header-file1, --hf1. (set_suppressions): Call gen_suppr_spec_from_headers with header_files. * tools/abipkgdiff.cc (maybe_create_private_types_suppressions): Call gen_suppr_spec_from_headers with an empty string vector. * doc/manuals/abidiff.rst: Document --header-file1, --hf1 and --header-file2, --hf2. Add new options to documentation of --drop-private-types. * doc/manuals/abidw.rst: Document --header-file, --hf. * doc/manuals/abilint.rst: Likewise. Signed-off-by: Mark Wielaard --- doc/manuals/abidiff.rst | 33 ++++++++++----- doc/manuals/abidw.rst | 6 +++ doc/manuals/abilint.rst | 6 +++ include/abg-tools-utils.h | 3 +- src/abg-tools-utils.cc | 86 ++++++++++++++++++++++++++------------- tools/abidiff.cc | 46 +++++++++++++++++---- tools/abidw.cc | 35 +++++++++++++++- tools/abilint.cc | 15 ++++++- tools/abipkgdiff.cc | 4 +- 9 files changed, 183 insertions(+), 51 deletions(-) diff --git a/doc/manuals/abidiff.rst b/doc/manuals/abidiff.rst index 8d914db5..6aac73bb 100644 --- a/doc/manuals/abidiff.rst +++ b/doc/manuals/abidiff.rst @@ -104,12 +104,24 @@ Options library that the tool has to consider. The tool will thus filter out ABI changes on types that are not defined in public headers. + * ``--header-file1 | --hf1`` + + Specifies where to find one public header of the first shared + library that the tool has to consider. The tool will thus filter + out ABI changes on types that are not defined in public headers. + * ``--headers-dir2 | --hd2`` Specifies where to find the public headers of the second shared library that the tool has to consider. The tool will thus filter out ABI changes on types that are not defined in public headers. + * ``--header-file2 | --hf2`` + + Specifies where to find one public header of the second shared + library that the tool has to consider. The tool will thus filter + out ABI changes on types that are not defined in public headers. + * ``--no-linux-kernel-mode`` Without this option, if abidiff detects that the binaries it is @@ -141,13 +153,13 @@ Options * ``--drop-private-types`` - This option is to be used with the ``--headers-dir1`` and - ``--headers-dir2`` options. With this option, types that are - *NOT* defined in the headers are entirely dropped from the - internal representation build by Libabigail to represent the ABI. - They thus don't have to be filtered out from the final ABI change - report because they are not even present in Libabigail's - representation. + This option is to be used with the ``--headers-dir1``, + ``header-file1``, ``header-file2`` and ``--headers-dir2`` options. + With this option, types that are *NOT* defined in the headers are + entirely dropped from the internal representation build by + Libabigail to represent the ABI. They thus don't have to be + filtered out from the final ABI change report because they are not + even present in Libabigail's representation. Without this option however, those private types are kept in the internal representation and later filtered out from the report. @@ -218,9 +230,10 @@ Options This option might incur some serious performance degradation as the number of types analyzed can be huge. However, if paired with - the ``--headers-dir{1,2}`` options, the additional non-reachable - types analyzed are restricted to those defined in public headers - files, thus hopefully making the performance hit acceptable. + the ``--headers-dir{1,2}`` and/or ``header-file{1,2}`` options, + the additional non-reachable types analyzed are restricted to + those defined in public headers files, thus hopefully making the + performance hit acceptable. Also, using this option alongside suppression specifications (by also using the ``--suppressions`` option) might help keep the number of diff --git a/doc/manuals/abidw.rst b/doc/manuals/abidw.rst index 4843d009..8f6c3c39 100644 --- a/doc/manuals/abidw.rst +++ b/doc/manuals/abidw.rst @@ -132,6 +132,12 @@ Options library that the tool has to consider. The tool will thus filter out types that are not defined in public headers. + * ``--header-file | --hf`` + + Specifies where to find one of the public headers of the abi file + that the tool has to consider. The tool will thus filter out + types that are not defined in public headers. + * ``--no-linux-kernel-mode`` Without this option, if abipkgiff detects that the binaries it is diff --git a/doc/manuals/abilint.rst b/doc/manuals/abilint.rst index 9e07f47f..4cc0b428 100644 --- a/doc/manuals/abilint.rst +++ b/doc/manuals/abilint.rst @@ -76,6 +76,12 @@ Options library that the tool has to consider. The tool will thus filter out types that are not defined in public headers. + * ``--header-file | --hf`` + + Specifies where to find one of the public headers of the abi file + that the tool has to consider. The tool will thus filter out + types that are not defined in public headers. + * ``--stdin | --`` Read the input content from standard input. diff --git a/include/abg-tools-utils.h b/include/abg-tools-utils.h index 053519a4..564fe646 100644 --- a/include/abg-tools-utils.h +++ b/include/abg-tools-utils.h @@ -84,7 +84,8 @@ string trim_leading_string(const string& from, const string& to_trim); void convert_char_stars_to_char_star_stars(const vector&, vector&); suppr::type_suppression_sptr -gen_suppr_spec_from_headers(const string& hdrs_root_dir); +gen_suppr_spec_from_headers(const string& hdrs_root_dir, + const vector& hdr_files); suppr::suppressions_type gen_suppr_spec_from_kernel_abi_whitelists diff --git a/src/abg-tools-utils.cc b/src/abg-tools-utils.cc index 11f6129e..b05ed645 100644 --- a/src/abg-tools-utils.cc +++ b/src/abg-tools-utils.cc @@ -1787,6 +1787,40 @@ make_path_absolute_to_be_freed(const char*p) return result; } +/// This is a sub-routine of gen_suppr_spec_from_headers and +/// handle_fts_entry. +/// +/// @param file add to the to the vector returned by +/// type_suppression::get_source_locations_to_keep() +/// +/// @param suppr the file name is going to be added to the vector +/// returned by the method type_suppression::get_source_locations_to_keep +/// of this instance. +/// If this smart pointer is nil then a new instance @ref +/// type_suppression is created and this variable is made to point to +/// it. +static void +handle_file_entry(const string& file, + type_suppression_sptr& suppr) +{ + if (!suppr) + { + suppr.reset(new type_suppression(get_private_types_suppr_spec_label(), + /*type_name_regexp=*/"", + /*type_name=*/"")); + + // Types that are defined in system headers are usually + // OK to be considered as public types. + suppr->set_source_location_to_keep_regex_str("^/usr/include/"); + suppr->set_is_artificial(true); + } + + // And types that are defined in header files that are under + // the header directory file we are looking are to be + // considered public types too. + suppr->get_source_locations_to_keep().insert(file); +} + /// This is a sub-routine of gen_suppr_spec_from_headers. /// /// @param entry if this file represents a regular (or symlink) file, @@ -1815,23 +1849,7 @@ handle_fts_entry(const FTSENT *entry, if (string_ends_with(fname, ".h") || string_ends_with(fname, ".hpp") || string_ends_with(fname, ".hxx")) - { - if (!suppr) - { - suppr.reset(new type_suppression(get_private_types_suppr_spec_label(), - /*type_name_regexp=*/"", - /*type_name=*/"")); - - // Types that are defined in system headers are usually - // OK to be considered as public types. - suppr->set_source_location_to_keep_regex_str("^/usr/include/"); - suppr->set_is_artificial(true); - } - // And types that are defined in header files that are under - // the header directory file we are looking are to be - // considered public types too. - suppr->get_source_locations_to_keep().insert(fname); - } + handle_file_entry (fname, suppr); } } @@ -1845,25 +1863,35 @@ handle_fts_entry(const FTSENT *entry, /// @return the resulting type suppression generated, if any file was /// found in the directory tree @p headers_root_dir. type_suppression_sptr -gen_suppr_spec_from_headers(const string& headers_root_dir) +gen_suppr_spec_from_headers(const string& headers_root_dir, + const vector& header_files) { type_suppression_sptr result; - if (headers_root_dir.empty()) - // We were given no headers root dir so the resulting suppression - // specification shall be empty. + if (headers_root_dir.empty() && header_files.empty()) + // We were given no headers root dir and no header files + // so the resulting suppression specification shall be empty. return result; - char* paths[] = {const_cast(headers_root_dir.c_str()), 0}; + if (!headers_root_dir.empty()) + { + char* paths[] = {const_cast(headers_root_dir.c_str()), 0}; - FTS *file_hierarchy = fts_open(paths, FTS_LOGICAL|FTS_NOCHDIR, NULL); - if (!file_hierarchy) - return result; + FTS *file_hierarchy = fts_open(paths, FTS_LOGICAL|FTS_NOCHDIR, NULL); + if (!file_hierarchy) + return result; + + FTSENT *entry; + while ((entry = fts_read(file_hierarchy))) + handle_fts_entry(entry, result); + fts_close(file_hierarchy); + } + + for (vector::const_iterator file = header_files.begin(); + file != header_files.end(); + ++file) + handle_file_entry(*file, result); - FTSENT *entry; - while ((entry = fts_read(file_hierarchy))) - handle_fts_entry(entry, result); - fts_close(file_hierarchy); return result; } diff --git a/tools/abidiff.cc b/tools/abidiff.cc index 44ad1878..162d5ebc 100644 --- a/tools/abidiff.cc +++ b/tools/abidiff.cc @@ -79,7 +79,9 @@ struct options vector keep_fn_regex_patterns; vector keep_var_regex_patterns; string headers_dir1; + vector header_files1; string headers_dir2; + vector header_files2; bool drop_private_types; bool linux_kernel_mode; bool no_default_supprs; @@ -183,7 +185,9 @@ display_usage(const string& prog_name, ostream& out) << " --debug-info-dir1|--d1 the root for the debug info of file1\n" << " --debug-info-dir2|--d2 the root for the debug info of file2\n" << " --headers-dir1|--hd1 the path to headers of file1\n" + << " --header-file1|--hf1 the path to one header of file1\n" << " --headers-dir2|--hd2 the path to headers of file2\n" + << " --header-file2|--hf2 the path to one header of file2\n" << " --drop-private-types drop private types from " "internal representation\n" << " --no-linux-kernel-mode don't consider the input binaries as " @@ -317,6 +321,19 @@ parse_command_line(int argc, char* argv[], options& opts) opts.headers_dir1 = argv[j]; ++i; } + else if (!strcmp(argv[i], "--header-file1") + || !strcmp(argv[i], "--hf1")) + { + int j = i + 1; + if (j >= argc) + { + opts.missing_operand = true; + opts.wrong_option = argv[i]; + return true; + } + opts.header_files1.push_back(argv[j]); + ++i; + } else if (!strcmp(argv[i], "--headers-dir2") || !strcmp(argv[i], "--hd2")) { @@ -330,6 +347,19 @@ parse_command_line(int argc, char* argv[], options& opts) opts.headers_dir2 = argv[j]; ++i; } + else if (!strcmp(argv[i], "--header-file2") + || !strcmp(argv[i], "--hf2")) + { + int j = i + 1; + if (j >= argc) + { + opts.missing_operand = true; + opts.wrong_option = argv[i]; + return true; + } + opts.header_files2.push_back(argv[j]); + ++i; + } else if (!strcmp(argv[i], "--kmi-whitelist") || !strcmp(argv[i], "-w")) { @@ -690,22 +720,22 @@ set_diff_context_from_opts(diff_context_sptr ctxt, load_default_user_suppressions(supprs); } - if (!opts.headers_dir1.empty()) + if (!opts.headers_dir1.empty() || !opts.header_files1.empty()) { // Generate suppression specification to avoid showing ABI // changes on types that are not defined in public headers. suppression_sptr suppr = - gen_suppr_spec_from_headers(opts.headers_dir1); + gen_suppr_spec_from_headers(opts.headers_dir1, opts.header_files1); if (suppr) ctxt->add_suppression(suppr); } - if (!opts.headers_dir2.empty()) + if (!opts.headers_dir2.empty() || !opts.header_files2.empty()) { // Generate suppression specification to avoid showing ABI // changes on types that are not defined in public headers. suppression_sptr suppr = - gen_suppr_spec_from_headers(opts.headers_dir2); + gen_suppr_spec_from_headers(opts.headers_dir2, opts.header_files2); if (suppr) ctxt->add_suppression(suppr); } @@ -740,7 +770,7 @@ set_suppressions(ReadContextType& read_ctxt, const options& opts) read_suppressions(*i, supprs); if (read_context_get_path(read_ctxt) == opts.file1 - && !opts.headers_dir1.empty()) + && (!opts.headers_dir1.empty() || !opts.header_files1.empty())) { // Generate suppression specification to avoid showing ABI // changes on types that are not defined in public headers for @@ -750,7 +780,7 @@ set_suppressions(ReadContextType& read_ctxt, const options& opts) // corpus loading, they are going to be dropped from the // internal representation altogether. suppression_sptr suppr = - gen_suppr_spec_from_headers(opts.headers_dir1); + gen_suppr_spec_from_headers(opts.headers_dir1, opts.header_files1); if (suppr) { if (opts.drop_private_types) @@ -760,7 +790,7 @@ set_suppressions(ReadContextType& read_ctxt, const options& opts) } if (read_context_get_path(read_ctxt) == opts.file2 - && !opts.headers_dir2.empty()) + && (!opts.headers_dir2.empty() || !opts.header_files2.empty())) { // Generate suppression specification to avoid showing ABI // changes on types that are not defined in public headers for @@ -770,7 +800,7 @@ set_suppressions(ReadContextType& read_ctxt, const options& opts) // corpus loading, they are going to be dropped from the // internal representation altogether. suppression_sptr suppr = - gen_suppr_spec_from_headers(opts.headers_dir2); + gen_suppr_spec_from_headers(opts.headers_dir2, opts.header_files2); if (suppr) { if (opts.drop_private_types) diff --git a/tools/abidw.cc b/tools/abidw.cc index 3b531999..a8b9ad32 100644 --- a/tools/abidw.cc +++ b/tools/abidw.cc @@ -88,6 +88,7 @@ struct options vector di_root_paths; vector prepared_di_root_paths; string headers_dir; + vector header_files; string vmlinux; vector suppression_paths; vector kabi_whitelist_paths; @@ -149,6 +150,7 @@ display_usage(const string& prog_name, ostream& out) << " --version|-v display program version information and exit\n" << " --debug-info-dir|-d look for debug info under 'dir-path'\n" << " --headers-dir|--hd the path to headers of the elf file\n" + << " --header-file|--hf the path one header of the elf file\n" << " --out-file write the output to 'file-path'\n" << " --noout do not emit anything after reading the binary\n" << " --suppressions|--suppr specify a suppression file\n" @@ -217,6 +219,15 @@ parse_command_line(int argc, char* argv[], options& opts) opts.headers_dir = argv[j]; ++i; } + else if (!strcmp(argv[i], "--header-file") + || !strcmp(argv[i], "--hf")) + { + int j = i + 1; + if (j >= argc) + return false; + opts.header_files.push_back(argv[j]); + ++i; + } else if (!strcmp(argv[i], "--out-file")) { if (argc <= i + 1 @@ -349,6 +360,24 @@ maybe_check_suppression_files(const options& opts) return true; } +/// Check that the header files supplied are present. +/// If not, emit an error on stderr. +/// +/// @param opts the options instance to use. +/// +/// @return true if all header files are present, false otherwise. +static bool +maybe_check_header_files(const options& opts) +{ + for (vector::const_iterator file = opts.header_files.begin(); + file != opts.header_files.end(); + ++file) + if (!check_file(*file, cerr, "abidw")) + return false; + + return true; +} + /// Set suppression specifications to the @p read_context used to load /// the ABI corpus from the ELF/DWARF file. /// @@ -371,7 +400,8 @@ set_suppressions(read_context& read_ctxt, options& opts) read_suppressions(*i, supprs); suppression_sptr suppr = - abigail::tools_utils::gen_suppr_spec_from_headers(opts.headers_dir); + abigail::tools_utils::gen_suppr_spec_from_headers(opts.headers_dir, + opts.header_files); if (suppr) supprs.push_back(suppr); @@ -721,6 +751,9 @@ main(int argc, char* argv[]) if (!maybe_check_suppression_files(opts)) return 1; + if (!maybe_check_header_files(opts)) + return 1; + abigail::tools_utils::file_type type = abigail::tools_utils::guess_file_type(opts.in_file_path); if (type != abigail::tools_utils::FILE_TYPE_ELF diff --git a/tools/abilint.cc b/tools/abilint.cc index b06ffd72..e9d6f93e 100644 --- a/tools/abilint.cc +++ b/tools/abilint.cc @@ -86,6 +86,7 @@ struct options abg_compat::shared_ptr di_root_path; vector suppression_paths; string headers_dir; + vector header_files; options() : display_version(false), @@ -107,6 +108,8 @@ display_usage(const string& prog_name, ostream& out) << " --debug-info-dir the path under which to look for " << " --headers-dir|--hd the path to headers of the elf file\n" "debug info for the elf \n" + << " --header-file|--hf the path to one header of the elf file\n" + "debug info for the elf \n" << " --suppressions|--suppr specify a suppression file\n" << " --diff for xml inputs, perform a text diff between " "the input and the memory model saved back to disk\n" @@ -161,6 +164,15 @@ parse_command_line(int argc, char* argv[], options& opts) opts.headers_dir = argv[j]; ++i; } + else if (!strcmp(argv[i], "--header-file") + || !strcmp(argv[i], "--hf")) + { + int j = i + 1; + if (j >= argc) + return false; + opts.header_files.push_back(argv[j]); + ++i; + } else if (!strcmp(argv[i], "--suppressions") || !strcmp(argv[i], "--suppr")) { @@ -240,7 +252,8 @@ set_suppressions(ReadContextType& read_ctxt, const options& opts) read_suppressions(*i, supprs); suppression_sptr suppr = - abigail::tools_utils::gen_suppr_spec_from_headers(opts.headers_dir); + abigail::tools_utils::gen_suppr_spec_from_headers(opts.headers_dir, + opts.header_files); if (suppr) supprs.push_back(suppr); diff --git a/tools/abipkgdiff.cc b/tools/abipkgdiff.cc index 611b4d07..ea4efb0e 100644 --- a/tools/abipkgdiff.cc +++ b/tools/abipkgdiff.cc @@ -1529,8 +1529,10 @@ maybe_create_private_types_suppressions(package& pkg, const options &opts) if (!is_dir(headers_path)) return false; + // We don't list individual files, just look under the headers_path. + vector no_header_files; suppression_sptr suppr = - gen_suppr_spec_from_headers(headers_path); + gen_suppr_spec_from_headers(headers_path, no_header_files); if (suppr) { From patchwork Sun Apr 12 01:47:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Wielaard X-Patchwork-Id: 39087 From: mark@klomp.org (Mark Wielaard) Date: Sun, 12 Apr 2020 03:47:57 +0200 Subject: [PATCH 2/2] Add --drop-private-types to abidw. In-Reply-To: <20200412014757.30688-1-mark@klomp.org> References: <20200412014757.30688-1-mark@klomp.org> Message-ID: <20200412014757.30688-2-mark@klomp.org> To create smaller abi XML files it is useful to be able to drop the private types from the representation. * tools/abidw.cc (options): Add drop_private_types bool. (display_usage): Add --drop-private-types. (parse_command_line): Parse --drop-private-types, set opts. (set_suppressions): Call set_drops_artifact_from_ir when drop_private_types set. * doc/manuals/abidw.rst: Document --drop-private-types. Signed-off-by: Mark Wielaard --- doc/manuals/abidw.rst | 8 ++++++++ tools/abidw.cc | 13 +++++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/doc/manuals/abidw.rst b/doc/manuals/abidw.rst index 8f6c3c39..05a90003 100644 --- a/doc/manuals/abidw.rst +++ b/doc/manuals/abidw.rst @@ -138,6 +138,14 @@ Options that the tool has to consider. The tool will thus filter out types that are not defined in public headers. + * ``--drop-private-types`` + + This option is to be used with the ``--headers-dir`` and/or + ``header-file`` options. With this option, types that are *NOT* + defined in the headers are entirely dropped from the internal + representation build by Libabigail to represent the ABI and will + not end up in the abi XML file. + * ``--no-linux-kernel-mode`` Without this option, if abipkgiff detects that the binaries it is diff --git a/tools/abidw.cc b/tools/abidw.cc index a8b9ad32..8b163ccd 100644 --- a/tools/abidw.cc +++ b/tools/abidw.cc @@ -109,6 +109,7 @@ struct options bool abidiff; bool annotate; bool do_log; + bool drop_private_types; options() : display_version(), @@ -126,7 +127,8 @@ struct options show_locs(true), abidiff(), annotate(), - do_log() + do_log(), + drop_private_types(false) {} ~options() @@ -158,6 +160,7 @@ display_usage(const string& prog_name, ostream& out) << " --no-corpus-path do not take the path to the corpora into account\n" << " --no-show-locs do not show location information\n" << " --short-locs only print filenames rather than paths\n" + << " --drop-private-types drop private types from representation\n" << " --no-comp-dir-path do not show compilation path information\n" << " --check-alternate-debug-info check alternate debug info " "of \n" @@ -294,6 +297,8 @@ parse_command_line(int argc, char* argv[], options& opts) } else if (!strcmp(argv[i], "--load-all-types")) opts.load_all_types = true; + else if (!strcmp(argv[i], "--drop-private-types")) + opts.drop_private_types = true; else if (!strcmp(argv[i], "--no-linux-kernel-mode")) opts.linux_kernel_mode = false; else if (!strcmp(argv[i], "--abidiff")) @@ -403,7 +408,11 @@ set_suppressions(read_context& read_ctxt, options& opts) abigail::tools_utils::gen_suppr_spec_from_headers(opts.headers_dir, opts.header_files); if (suppr) - supprs.push_back(suppr); + { + if (opts.drop_private_types) + suppr->set_drops_artifact_from_ir(true); + supprs.push_back(suppr); + } using abigail::tools_utils::gen_suppr_spec_from_kernel_abi_whitelists; const suppressions_type& wl_suppr =