From patchwork Fri Jul 7 11:43:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dodji Seketeli X-Patchwork-Id: 72291 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A603B383BEA1 for ; Fri, 7 Jul 2023 12:11:16 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org A603B383BEA1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1688731876; bh=E3js7HcGJFcnOd1b+lG8xcr3fkH7HN0zld7d4d2e+Uk=; h=To:Cc:Subject:References:Date:In-Reply-To:List-Id: List-Unsubscribe:List-Archive:List-Help:List-Subscribe:From: Reply-To:From; b=nSnfEc3j+rgrlvoKKAAqCqAMduX8/c26GcnsMAXOsckkYApo4hmvOkKsrFjhXY3ti dZCgGltgzp5JQDHrKFRRx59WkY0IIRs6UYEbl18XTf1aq4prAknXnVJKoIqX2j2VUP 5NIU1wx67UHK1q3esGzaVEPFYpawLU8mzdhnkE7s= X-Original-To: libabigail@sourceware.org Delivered-To: libabigail@sourceware.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 7DB08392A8A4 for ; Fri, 7 Jul 2023 11:43:11 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 7DB08392A8A4 Received: from mail-ua1-f72.google.com (mail-ua1-f72.google.com [209.85.222.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-472-CN0yA4rQMTyTXm75hk6p6A-1; Fri, 07 Jul 2023 07:43:09 -0400 X-MC-Unique: CN0yA4rQMTyTXm75hk6p6A-1 Received: by mail-ua1-f72.google.com with SMTP id a1e0cc1a2514c-794c5fa59c6so421377241.2 for ; Fri, 07 Jul 2023 04:43:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688730189; x=1691322189; h=mime-version:user-agent:message-id:in-reply-to:date:references :organization:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=E3js7HcGJFcnOd1b+lG8xcr3fkH7HN0zld7d4d2e+Uk=; b=Gm2k8fqK3qyvMApgYOB0A6lBVf+MKx7WUNFxV0U7qP0XvxpBSjrjhNoDSiCQSWBZyj pCwhjtjGVjF0QUZ2wFi5CyQZtk7rJZBTEBnw2L2wTcNyEEp+dqNJzg2OxaXS00eJabEh VGUzuWhywHe96MlQe6Rfa9KjAdlOqp0OymDwgd7av8DOBZsMYSoVYTMD6N1/PWCOGkh3 5Q90uUifKYsxoTI9mNXxCem2R9zbOghW7SpUwzhUefy99mYXKVe9PE+Mz6ia6mAm+Baw x1VyVvEjIHDHjBfwsBGJD8lYFRm4uuM2k6Z6Eq1S2UdUR+lH+6FI4J9cIT0eX9C8xizc gZVQ== X-Gm-Message-State: ABy/qLYXn1fh9qRxqgk7pr9E8cXxnY5IrajK4STf5a9Hrvx5FKSeUby3 Id6ICnRT94un1YvtjlW0aMQJZgmqhXF1TwjWlReZdmJ/gQECaVtLJRz3dLS+0a7wJiIvzP5ZQJj csluOkITWYIElyzIMmuuZfZLPXHR9 X-Received: by 2002:a05:6102:524:b0:444:eedd:1aec with SMTP id m4-20020a056102052400b00444eedd1aecmr3091704vsa.30.1688730188999; Fri, 07 Jul 2023 04:43:08 -0700 (PDT) X-Google-Smtp-Source: APBJJlEjfJvS+/1379tfJ2mikNps6LUD70pnxPnhC/SZVZ++pr9Q8vpuzduxmSV5jnDyCPjLEn/9wg== X-Received: by 2002:a05:6102:524:b0:444:eedd:1aec with SMTP id m4-20020a056102052400b00444eedd1aecmr3091694vsa.30.1688730188678; Fri, 07 Jul 2023 04:43:08 -0700 (PDT) Received: from localhost ([88.120.130.27]) by smtp.gmail.com with ESMTPSA id j3-20020a0cf503000000b0062b76c29978sm979904qvm.6.2023.07.07.04.43.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 07 Jul 2023 04:43:08 -0700 (PDT) Received: by localhost (Postfix, from userid 1000) id 9E1DFB4FFD; Fri, 7 Jul 2023 13:43:06 +0200 (CEST) To: Dodji Seketeli Cc: libabigail@sourceware.org Subject: [PATCH 3/4, applied] abidw: Add --{follow,list}-dependencies & --add-binaries support Organization: Red Hat / France References: <87edlk0x3p.fsf@redhat.com> X-Operating-System: CentOS Stream release 9 X-URL: http://www.redhat.com Date: Fri, 07 Jul 2023 13:43:06 +0200 In-Reply-To: <87edlk0x3p.fsf@redhat.com> (Dodji Seketeli's message of "Fri, 07 Jul 2023 13:39:22 +0200") Message-ID: <871qhk0wxh.fsf@redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-11.9 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libabigail@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list of the Libabigail project List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-Patchwork-Original-From: Dodji Seketeli via Libabigail From: Dodji Seketeli Reply-To: Dodji Seketeli Errors-To: libabigail-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libabigail" Hello, This implements the --follow-dependencies , --list-dependencies, --add-binaries and --added-binaries-dir options for the abidw command, as documented in the README-ABIDIFF-BINARIES-SET-SUPPORT.md file. * tools/abidw.cc (options::{added_bins_dirs, added_bins, follow_dependencies, list_dependencies}): Add new data members. (options::options): Initialize follow_dependencies and list_dependencies. (display_usage): Add usage strings for --add-binaries, --follow-dependencies, --list-dependencies, --added-binaries-dir. (parse_command_line): Parse options --add-binaries, --follow-dependencies, --list-dependencies, --added-binaries-dir. (load_corpus_and_write_abixml): Implement the --list-dependencies, --follow-dependencies and --add-binaries sub-commands. * doc/manuals/abidw.rst: Document the --add-binaries, --follow-dependencies, --list-dependencies, --added-binaries-dir options. Signed-off-by: Dodji Seketeli Applied to master. --- doc/manuals/abidw.rst | 72 +++++++++++++++++++++++- tools/abidw.cc | 125 ++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 190 insertions(+), 7 deletions(-) diff --git a/doc/manuals/abidw.rst b/doc/manuals/abidw.rst index 3cba434c..1e308e66 100644 --- a/doc/manuals/abidw.rst +++ b/doc/manuals/abidw.rst @@ -44,14 +44,49 @@ Options Display a short help about the command and exit. - * `--version | -v` + * ``--version | -v`` Display the version of the program and exit. - * `--abixml-version` + * ``--abixml-version`` Display the version of the ABIXML format emitted by this program and exit. + * ``--add-binaries`` <*bin1,bin2,...*> + + For each of the comma-separated binaries given in argument to this + option, if the binary is found in the directory specified by the + `--added-binaries-dir` option, then load the ABI corpus of the + binary and add it to a set of ABI corpora (called a ABI Corpus + Group) made of the binary denoted by the Argument of + ``abidw``. That corpus group is then serialized out. + + * ``--follow-dependencies`` + + For each dependency of the input binary of ``abidw``, if it is + found in the directory specified by the ``--added-binaries-dir`` + option, then construct an ABI corpus out of the dependency and add + it to a set of ABI corpora (called an ABI Corpus Group) along with + the ABI corpus of the input binary of the program. The ABI Corpus + Group is then serialized out. + + * ``--list-dependencies`` + + For each dependency of the input binary of``abidw``, if it's found + in the directory specified by the ``--added-binaries-dir`` option, + then the name of the dependency is printed out. + + * ``--added-binaries-dir | --abd`` <*dir-path*> + + This option is to be used in conjunction with the + ``--add-binaries``, the ``--follow-dependencies`` or the + ``--list-dependencies`` option. Binaries listed as arguments of + the ``--add-binaries`` option or being dependencies of the input + binary in the case of the ``--follow-dependencies`` option and + found in the directory <*dir-path*> are going to be loaded as ABI + corpus and added to the set of ABI corpora (called an ABI corpus + group) built and serialized. + * ``--debug-info-dir | -d`` <*dir-path*> In cases where the debug info for *path-to-elf-file* is in a @@ -368,6 +403,39 @@ Options Emit verbose logs about the progress of miscellaneous internal things. +Usage examples +============== + + 1. Emitting an ``ABIXML`` representation of a binary: :: + + $ abidw binary > binary.abi + + + 2. Emitting an ``ABIXML`` representation of a set of binaries + specified on the command line: :: + + $ abidw --added-binaries=bin1,bin2,bin3 \ + --added-binaries-dir /some/where \ + binary > binaries.abi + + Note that the binaries bin1, bin2 and bin3 are to be found in the + directory ``/some/where``. A representation of the ABI of the + set of binaries ``binary, bin1, bin2`` and ``bin3`` called an + ``ABI corpus group`` is serialized in the file binaries.abi. + + 3. Emitting an ``ABIXML`` representation of a binary and its + dependencies: :: + + $ abidw --follow-dependencies \ + --added-binaries-dir /some/where \ + binary > binary.abi + + Note that only the dependencies that are found in the directory + ``/some/where`` are analysed. Their ABIs, along with the ABI the + binary named ``binary`` are represented as an ABI corpus group + and serialized in the file ``binary.abi``, in the ABIXML format. + + Notes ===== diff --git a/tools/abidw.cc b/tools/abidw.cc index eead7889..035dd9ce 100644 --- a/tools/abidw.cc +++ b/tools/abidw.cc @@ -22,6 +22,7 @@ #include #include #include +#include #include "abg-config.h" #include "abg-tools-utils.h" #include "abg-corpus.h" @@ -43,7 +44,9 @@ using std::cout; using std::ostream; using std::ofstream; using std::vector; +using std::set; using std::shared_ptr; +using std::static_pointer_cast; using abg_compat::optional; using abigail::tools_utils::emit_prefix; using abigail::tools_utils::temp_file; @@ -52,6 +55,9 @@ using abigail::tools_utils::check_file; using abigail::tools_utils::build_corpus_group_from_kernel_dist_under; using abigail::tools_utils::timer; using abigail::tools_utils::create_best_elf_based_reader; +using abigail::tools_utils::stick_corpus_and_dependencies_into_corpus_group; +using abigail::tools_utils::stick_corpus_and_binaries_into_corpus_group; +using abigail::tools_utils::add_dependencies_into_corpus_group; using abigail::ir::environment_sptr; using abigail::ir::environment; using abigail::corpus; @@ -71,6 +77,7 @@ using abigail::xml_writer::create_write_context; using abigail::xml_writer::type_id_style_kind; using abigail::xml_writer::write_context_sptr; using abigail::xml_writer::write_corpus; +using abigail::xml_writer::write_corpus_group; using abigail::abixml::read_corpus_from_abixml_file; using namespace abigail; @@ -84,6 +91,8 @@ struct options vector prepared_di_root_paths; vector headers_dirs; vector header_files; + vector added_bins_dirs; + vector added_bins; string vmlinux; vector suppression_paths; vector kabi_whitelist_paths; @@ -104,6 +113,8 @@ struct options bool corpus_group_for_linux; bool show_stats; bool noout; + bool follow_dependencies; + bool list_dependencies; #ifdef WITH_CTF bool use_ctf; #endif @@ -148,6 +159,8 @@ struct options corpus_group_for_linux(false), show_stats(), noout(), + follow_dependencies(), + list_dependencies(), #ifdef WITH_CTF use_ctf(false), #endif @@ -229,6 +242,12 @@ display_usage(const string& prog_name, ostream& out) << " --vmlinux the path to the vmlinux binary to consider to emit " "the ABI of the union of vmlinux and its modules\n" << " --abidiff compare the loaded ABI against itself\n" + << " --add-binaries build a corpus group with " + "the added inaries\n" + << " --follow-dependencies build a corpus group with the dependencies\n" + << " --list-dependencies list the dependencies of a given binary\n" + << " --added-binaries-dir|--abd where to look for dependencies " + "or added binaries\n" #ifdef WITH_DEBUG_SELF_COMPARISON << " --debug-abidiff debug the process of comparing the loaded ABI against itself\n" #endif @@ -294,6 +313,15 @@ parse_command_line(int argc, char* argv[], options& opts) opts.headers_dirs.push_back(argv[j]); ++i; } + else if (!strcmp(argv[i], "--added-binaries-dir") + || !strcmp(argv[i], "--abd")) + { + int j = i + 1; + if (j >= argc) + return false; + opts.added_bins_dirs.push_back(argv[j]); + ++i; + } else if (!strcmp(argv[i], "--header-file") || !strcmp(argv[i], "--hf")) { @@ -344,6 +372,28 @@ parse_command_line(int argc, char* argv[], options& opts) } else if (!strcmp(argv[i], "--noout")) opts.noout = true; + else if (!strcmp(argv[i], "--follow-dependencies")) + opts.follow_dependencies = true; + else if (!strcmp(argv[i], "--list-dependencies")) + opts.list_dependencies = true; + else if (!strncmp(argv[i], "--add-binaries=", + strlen("--add-binaries="))) + tools_utils::get_comma_separated_args_of_option(argv[i], + "--add-binaries=", + opts.added_bins); + else if (!strcmp(argv[i], "--add-binaries")) + { + int j = i + 1; + if (j >= argc) + return false; + + string s = argv[j]; + if (s.find(',')) + tools_utils::split_string(s, ",", opts.added_bins); + else + opts.added_bins.push_back(s); + ++i; + } #ifdef WITH_CTF else if (!strcmp(argv[i], "--ctf")) opts.use_ctf = true; @@ -598,6 +648,7 @@ load_corpus_and_write_abixml(char* argv[], #endif corpus_sptr corp; + corpus_group_sptr corp_group; fe_iface::status s = fe_iface::STATUS_UNKNOWN; corpus::origin requested_fe_kind = corpus::DWARF_ORIGIN; #ifdef WITH_CTF @@ -720,6 +771,50 @@ load_corpus_and_write_abixml(char* argv[], return 1; } + if (opts.list_dependencies) + { + // Show the dependencies of the corpus and display them. + set dependencies; + if (tools_utils::get_dependencies(*corp, opts.added_bins_dirs, + dependencies)) + { + cout << "Dependencies of '" << corp->get_path() + << "':\n\t"; + int n = 0; + for (const auto& dep : dependencies) + { + if (n) + cout << ", "; + cout << dep; + ++n; + } + cout << "\n"; + } + } + + if (!opts.added_bins.empty()) + corp_group = + stick_corpus_and_binaries_into_corpus_group(reader, corp, + opts.added_bins, + opts.added_bins_dirs); + + if (opts.follow_dependencies) + { + // load the dependencies of the corpus and put them all into a + // corpus group. + + // If a corpus_group already exists, use that one ... + if (!corp_group->is_empty()) + add_dependencies_into_corpus_group(reader, *corp, + opts.added_bins_dirs, + *corp_group); + else + // .. otherwise, create a new corpus group. + corp_group = + stick_corpus_and_dependencies_into_corpus_group(reader, corp, + opts.added_bins_dirs); + } + // Clear some resources to gain back some space. t.start(); reader.reset(); @@ -744,7 +839,10 @@ load_corpus_and_write_abixml(char* argv[], // against the ABI of the input ELF file. temp_file_sptr tmp_file = temp_file::create(); set_ostream(*write_ctxt, tmp_file->get_stream()); - write_corpus(*write_ctxt, corp, 0); + if (corp_group) + write_corpus_group(*write_ctxt, corp_group, 0); + else + write_corpus(*write_ctxt, corp, 0); tmp_file->get_stream().flush(); #ifdef WITH_DEBUG_SELF_COMPARISON @@ -763,7 +861,14 @@ load_corpus_and_write_abixml(char* argv[], #endif t.start(); fe_iface::status sts; - corpus_sptr corp2 = rdr->read_corpus(sts); + corpus_sptr corp2; + corpus_group_sptr corp_group2; + + if (corp_group) + corp_group2 = abixml::read_corpus_group_from_input(*rdr); + else + corp2 = rdr->read_corpus(sts); + t.stop(); if (opts.do_log) emit_prefix(argv[0], cerr) @@ -781,7 +886,11 @@ load_corpus_and_write_abixml(char* argv[], set_diff_context(ctxt); ctxt->show_locs(opts.show_locs); t.start(); - corpus_diff_sptr diff = compute_diff(corp, corp2, ctxt); + corpus_diff_sptr diff = + corp_group2 + ? compute_diff(corp_group, corp_group2, ctxt) + : compute_diff(corp, corp2, ctxt); + t.stop(); if (opts.do_log) emit_prefix(argv[0], cerr) @@ -822,7 +931,10 @@ load_corpus_and_write_abixml(char* argv[], } set_ostream(*write_ctxt, of); t.start(); - write_corpus(*write_ctxt, corp, 0); + if (corp_group) + write_corpus_group(*write_ctxt, corp_group, 0); + else + write_corpus(*write_ctxt, corp, 0); t.stop(); if (opts.do_log) emit_prefix(argv[0], cerr) @@ -833,7 +945,10 @@ load_corpus_and_write_abixml(char* argv[], else { t.start(); - exit_code = !write_corpus(*write_ctxt, corp, 0); + exit_code = + corp_group + ? !write_corpus_group(*write_ctxt, corp_group, 0) + : !write_corpus(*write_ctxt, corp, 0); t.stop(); if (opts.do_log) emit_prefix(argv[0], cerr)