From patchwork Wed May 4 22:29:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Guillermo E. Martinez" X-Patchwork-Id: 53480 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 75C643858C51 for ; Wed, 4 May 2022 22:30:08 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 75C643858C51 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1651703408; bh=YHO4/nJ5MZYCPWVENUmbsEcu3XcBFwDBZEIU0F3hwJI=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Help:List-Subscribe:From:Reply-To:From; b=bj8W6frrjyw+b2z/P174mTqjQVFrNfWXWvLAHNwG9kWw95JaEdNpAG/KHUzn0uPq7 0rW0l1jaHUmk1LzU2IAS4I91VwcDKU2N4KU/HlUYXuh8zc5LmxghLcTs3aEfdTa/ZH pIb0BemJvNWDNRdlurpWqDOValoJeLB0BDpN8EJo= X-Original-To: libabigail@sourceware.org Delivered-To: libabigail@sourceware.org Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by sourceware.org (Postfix) with ESMTPS id 7766C3858D1E for ; Wed, 4 May 2022 22:29:57 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 7766C3858D1E Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 244Jx82E018740 for ; Wed, 4 May 2022 22:29:55 GMT Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3frwnt9v0q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 04 May 2022 22:29:55 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 244MLNXT033037 for ; Wed, 4 May 2022 22:29:54 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2175.outbound.protection.outlook.com [104.47.57.175]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fs1a64npk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 04 May 2022 22:29:53 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JEUKLMolPgVHjzJnPjPQNABOTkmZ0nqmWuuAUBrTkPXJk+NQ3mJnBG9yR3E2GSsdyy7FfW0bbew+zvDXNE7ais1j+I2xeOXplxI71CJFZ7aX9NijFS/xI0oXiprJ3Mvb9G/70HISF9dhq/Np/xqfgeBBRCf41hPO2QlkMo2+thOSpgTtp/cfk2XMOwYaiSJpVBIV+buBc5G7IwjOmoR/2oqnUnAsLwvLvJUZflO2KCQ1aHDDWLkyqfPKAq5trhlM4bDlTXSLcIzcLWmzyp+9oKXfyPIMQR1JrpH1nt/tHPd3Dc57mgngVU+RrvMty6cFUv1fJZ7SyARAlAbEHxnbsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YHO4/nJ5MZYCPWVENUmbsEcu3XcBFwDBZEIU0F3hwJI=; b=Val0xH9NEdpkd7b5XGPnl4zYR9GPutAKuY6Ohw47djstEykjE6k2t7gm7f++bNP8flwBg6190r7Flsd5707eUnH8h3t4a4iEi7nLNR06xpGeE6MUaK5RMSEpFCWCzPlszgkBKJ1X3f0sMOw5bQ5cMMfDUG7WoAXfTp5smy2h7ICmj90xgNIe0mmaQ3WULkwv/oJpdKMgv44hlk4/NYkSQjJM6j1taSzPzZGHxK9KCNRH9TtZfwe61pc+x/NWN9MqMkq+o95Hlvrbf0Rp2w+bYzE2Honm/TJ5ib4VGeTwtNiAS6fkH0JbFl6lgTvE+Ir2KxjaL5syQtBGNDXEtNmy6g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none Received: from MWHPR10MB1407.namprd10.prod.outlook.com (2603:10b6:300:23::20) by CH2PR10MB4247.namprd10.prod.outlook.com (2603:10b6:610:7a::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.24; Wed, 4 May 2022 22:29:49 +0000 Received: from MWHPR10MB1407.namprd10.prod.outlook.com ([fe80::107d:de35:fa5f:3cc0]) by MWHPR10MB1407.namprd10.prod.outlook.com ([fe80::107d:de35:fa5f:3cc0%7]) with mapi id 15.20.5206.024; Wed, 4 May 2022 22:29:49 +0000 To: libabigail@sourceware.org Subject: [PATCH v3] ctf-reader: Add support to read CTF information from Linux kernel Date: Wed, 4 May 2022 17:29:30 -0500 Message-Id: <20220504222930.1831587-1-guillermo.e.martinez@oracle.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <877d72fx7e.fsf@seketeli.org> References: <877d72fx7e.fsf@seketeli.org> X-ClientProxiedBy: SJ0PR05CA0002.namprd05.prod.outlook.com (2603:10b6:a03:33b::7) To MWHPR10MB1407.namprd10.prod.outlook.com (2603:10b6:300:23::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bec6a765-90e5-40ec-6a24-08da2e1d9907 X-MS-TrafficTypeDiagnostic: CH2PR10MB4247:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Ub01u8vbjft/vVdFCoLJh1GXgkhWcP2kf6BnBXjspb3a+EHgs8QFRyQ3UKisBPKy3QA1h7CFfUNz/OH6bpeThEneDW8xSB4um88X+sLW2Pr5kTsy8ycCQhHcxjx3BI+hQ1UULtgIcjDs/giLhsz3ZzpaXHrA482ZQdx/zEl38SyEHdzbIImJf7anCokOrIp1uE+rTl6I2bmYLsbAutsRlTO5PH7B2lvAtHpkvnAuYTXnV3Efnmlwkcf0zNPocnBj85IQRUY24qrl2PdWkKvdzfxQaQnDrKHlxAGQn29iXA4LMueeWBllKN8VvnyyCcZUDZAp7QZMQzlwTNtqiQADh2pgpBJJ7YagqaRRB9Ih+Xp7Rs69rZI19qdnJdK92x0tsGXUc7A/Vu/YwR4gmwjVqwdFtNRBBqxvfzAWr9OkB07Jdha7MpiI0nzj8yPl7ulu1ZSJna4o5MaOZEM9Q2BVQlLvg2vpTWWJoTHDj8tOZXrbZ955Kv9Krnc8BEZBWhHuZvcMIFMmHV1L2C55KNtCVw+nNNnj1faJk+XcHkDzF5g6LW3DGOF3bKZHyn0ndHxQiG2GdNWB5N+JmeFTnd2qcYr8FL9MSDszAHBMwlNwWJKtkpNu9oWAJSP5kJR2O+8+NHCtfgV9RuvNzPwDCNMjoXLEfcnDGfY/C/NWjxTmWs6dr8lEdagDgdj7gQBKv8uNiiDOUxeJ4bZlA4vFbPVK+QTEZGlOjnF5CsrlBPgYkx/eyy2YNZcaQbf14I7s3JTjCqTk3rPBZm/FHsLODN0Brw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MWHPR10MB1407.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(316002)(6916009)(508600001)(30864003)(2906002)(966005)(6486002)(6512007)(19627235002)(83380400001)(66556008)(4326008)(66476007)(86362001)(8676002)(6506007)(66946007)(6666004)(52116002)(103116003)(5660300002)(8936002)(186003)(1076003)(38100700002)(107886003)(2616005)(36756003)(579004)(559001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2 X-MS-Exchange-AntiSpam-MessageData-0: d41rnHCJBokYrXr3R2L64yfr0DmVHIdHhrmCu7C/eTaYCLWYz+jNMr8G2eXZfsJWZqus40xFREOFaI7jldQHlvVBRy53pSn1RQ+U72lZ5n9373YfAM4oMpokygNHnb/5CCVjGOWlbKX/WkBmbiP9amkiRBToOAKrm5EGotkdVQgQvyymYqD14rcPcvPkMZ6iRSUMKoWmYPrEhAnnnhBfEx2dELH0eSHmWRiSj0bd962/NHyiRa8v8dkw0y5d4G65ONkriVepCAB74qBbGeK/caOgyOeLPEPpzHkZcGQHOCroFKc+CyDumwYhscCTkAedGUnhhmDuze8rbgmFDtOTGOwyMfHWHdlvPnnaKaoSlMOJlKDn3DXLRww8aTDh7Vz+Kh6FbPxCvDybJ5RT9evIek1sfNP8/XLf2Z8QOUj0r4Dx6+MzoxpOLkXAYbElaTXKHByBjtCcwAm2oaeMc5vL5lUIaXnyDwiV1uXLXlv0xWzjpp2nYutsbrM/UiXXipL6BKHeO1/06GjXyYZR75oeUel+0OyPH3UuQmLm8lof9AkUyUoGTqRZtPB4+C7D9qi2LcvemI8YEDRFdcPfpRhwgeFalke9s/5m9IsfIlhM4cydZ4iCPiD09bCS/ImEtyApxs40XelVBG91jkjuHaEcdcZREgWtWsvb8jECQ1AtNOtw9KlVGk0LCL2Q7gOkDElVWxu4WYmgs6nKyknIszVI5CIBeU4RbOJEjNbEcnX2NXA1Crsd1ImAXa84mUhSSf8cszMY6+nyrwDXh2YSS6dat0xOSBjeYaHql9joX3kXSV1C2/qCjYrWZOcP9DEgMhkCXzL7/cSBOqfxZMCkUJXTQ1podn9qwPQfhEFHxjZlrsUAHPYXF7tIS50UBzGPQyuF6DXEVS885IEOYSgz/eHf9EMTt/AvPKKTvg6nPDxeyIZtuhgK6t5n6IEWvXerIsPQZIu4UWhEaMAfpZ5skRUp5psObEcoBXLkXSqkQynYJz+z+RlW/run2EFjr60KuCVvkgFCFSYcNX772nEqIt8UbvPws7ZeglLJUEHSKuXsXDEZeCuq7NUmQirxD/vO+TxPJ5vrzXJAcnkxmmOBA6cckqir8xFyV3MJAtE7XFOC5lWxyyGfarsEgtt/WI198b48rMG22EayAap4lwiPfA2edKhkfWS9qI6DvX08Rraobt/b17QyQpUGEcAnT7Xh7GWARbmSA165aOj+2xYkIM1F+qj+MqY/hV/Nln9HUbgNZFhZ2pcUpzomBBVIogvStpOAqTF2pcUR7LKcI9cAR7EH1WGt7yY6OYtb2P6xmgvZXKVqXAKBYyiTJEWDROW4NDlFuPOfGWAYL/AAbaH61wa2RbhsYgfgYSLPSILtKLaJy/YvUCxGPidaNGt4ZpYkWLdmXo53w4v2eBBC6TVL0Eb07NblBVK7uSluk8/T/GeRUIv5qQbHyCCd8PLHJZmx1D5BgeAaIqs2A562DhGsYGPev+o3I66If6yeHGr05+klmXqLNqy2MY1v55A4Y3CJpJH9q80FPpSrBxY8GGOhRJ0PqlEOtheqLqkI08PJhUwrZWhu8sDLmXoPT85m4XmX+paCf0QpU8PzXmkKRAngk5pfQifg9AOHKvoKInAq5wGSNMOElnuAXVG/m4mR+Cx8sE8dKmLRBcOWbTV1u846LqrT8llrco0CgTxPogG+sLWtwo6rMDtIjPiW/YhK/QdXdQBzfssd8vg6rUsHwTdUpznOdS3OadLJHBRfESnvncrXs/5K6LPwtyhpNwRd0tpHBJbuyB0orvOj X-MS-Exchange-AntiSpam-MessageData-1: Ppt90pAn5awhsA== X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: bec6a765-90e5-40ec-6a24-08da2e1d9907 X-MS-Exchange-CrossTenant-AuthSource: MWHPR10MB1407.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 May 2022 22:29:48.9237 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: NE/XZ5ici3ZAjrL4Z6pPasSRm0HjfsZKoEcCq1K1UIdLds4MVyTMtcSz57PSg8ECr0zXEv2xejQMTSD8/sr0h5TvdWB0ID6rlS2WMT+5g0k= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR10MB4247 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-05-04_06:2022-05-04, 2022-05-04 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 bulkscore=0 malwarescore=0 spamscore=0 mlxlogscore=999 mlxscore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2205040124 X-Proofpoint-ORIG-GUID: aXfAkmKep1vrgrKh_9HBhxCZDqpr2Naq X-Proofpoint-GUID: aXfAkmKep1vrgrKh_9HBhxCZDqpr2Naq X-Spam-Status: No, score=-12.4 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_LOW, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libabigail@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list of the Libabigail project List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-Patchwork-Original-From: "Guillermo E. Martinez via Libabigail" From: "Guillermo E. Martinez" Reply-To: "Guillermo E. Martinez" Errors-To: libabigail-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libabigail" Hello libabigail team, This is the v3 patch meant to read CTF information from Linux. Changes from v2 are: * Fix code according to comments: https://sourceware.org/pipermail/libabigail/2022q2/004331.html * Merge patch to improve performace in ctf-reader in this one: https://sourceware.org/pipermail/libabigail/2022q2/004310.html Please take a look and let me know your comments, I'll really appreciate them. Kind regards, Guillermo This patch is meant to extract ABI information from CTF stored in Linux kernel build directory it depends on: vmlinux.ctfa file. In order to generate CTF information in Linux kernel must be able to support 'make ctf', which cause the compiler to be run with -gctf emitting CTF information on the kernel. The target 'ctf' in Linux Makefile generate a 'vmlinux.ctfa', that will be used for ctf reader in libabigail. 'vmlinux.ctfa' has multiple 'ctf dictionaries' called: CTF archive members one for built-in kernel modules (in `vmlinux') and one for each out-of-tree kernel module organized in a parent-child hierarchy. In vmlinux.ctfa file there is a CTF archive member: `shared_ctf' it is a parent dictionary, containing share symbols and CTF types used by more than one kernel object. These common types are stored in 'types_map' in ctf reader, ignoring the ctf dictionary name, CTF API has the machinery to looking for a share type in parent dictionary referred in a child dictionary. This CTF layout can be dumped by using objdump tool. Due to _same_ ctf archive is used to built vmlinux corpus and the corpora for kernel module belongs to a group, open/close the CTF archive is hight time consuming during ctf extraction. So, performance is improved up to 300% (from ~2m:50s to ~50s), keeping open per _group_ `ctf_archive_t' pointer from vmlinux (the main_corpus_from_current_group), since this ctf archive file contains the information for kernel modules as well, then it's reused by each module to be processed in the group, invoking to `reset_read_context' ctf reader function. Doing so, `read_context::ctfa` member should be updated iff `LINUX_KERNEL_BINARY_ORIGIN' is set in corpus::origin member and the file to be process is not 'vmlinux', otherwise it must contain valid information from the main element in the corpus group, `ctf_close' must be called after processing all group's members, it is executed from `reader_context' destructor. The basic algorithm used to generate the Linux corpus follow: 1. Looking for: vmlinux, *.ko objects, and vmlinux.ctfa files. The first files are used to extract the ELF symbols, and the last one contains the CTF type information for non-static variables and functions symbols. 2. `process_ctf_archive' iterates on public symbols for vmlinux and its modules, using the name of the symbol, ctf reader search for CTF information in its dictionary, if the information was found it builds a `var_decl' or `function_decl' depending of `ctf_type_kind' result. This algorithm is also applied to ELF files (exec, dyn, rel), so instead of iterating on all ctf_types it uses public symbols as main loop. * abg-elf-reader-common.h: Include ctf-api.h file. (read_and_add_corpus_to_group_from_elf, set_read_context_corpus_group) (reset_read_context, dic_type_key): Declare new member functions. * include/abg-ir.cc (types_defined_same_linux_kernel_corpus_public): Use bitwise to know the corpus `origin'. * src/abg-ctf-reader.cc: Include map, algorithms header files. (read_context::type_map): Change from unordered_map to std::map storing ctf dictionary name as part of the key. (read_context::is_elf_exec): Add new member variable. (read_context::{cur_corpus_, cur_corpus_group_}): Likewise. (read_context::unknown_types_set): Likewise. (read_context::{current_corpus_group, main_corpus_from_current_group, has_corpus_group, current_corpus_is_main_corpus_from_current_group, should_reuse_type_from_corpus_group}): Add new member functions. (read_context::{add_unknown_type, lookup_unknown_type, initialize}): Likewise. (read_context::{add_type, lookup_type}): Add new `ctf_dict_t' type argument. (ctf_reader::{process_ctf_typedef, process_ctf_base_type, process_ctf_function_type, process_ctf_forward_type, process_ctf_struct_type, process_ctf_union_type, process_ctf_array_type, process_ctf_qualified_type, process_ctf_enum_type}): Add code to `reuse' types already registered in main corpus `should_reuse_type_from_corpus_group'. Use new `lookup_type' and `add_type' operations on `read_context::types_map'. Replace function calls to the new ctf interface. Add verifier to not build types duplicated by recursive calling chain. (ctf_reader::process_ctf_type): Add code to return immediately if the ctf type is unknown. Add unknown types to `unknown_types_set'. (ctf_reader::process_ctf_archive): Change comment. Add code to iterate over global symbols, searching by symbol name in the ctf dictionary using `ctf_lookup_{variable,by_symbol_name}' depending of the ELF file type and corpus type, creating a `{var,fuc}_decl' using the return type of `ctf_type_kind'. Also close the ctf dict and call `canonicalize_all_types'. (slurp_elf_info): Set `is_elf_exec' depending of ELF type. Also return success if corpus origin is Linux and symbol table was read. (ctf_reader::read_corpus): Add current corpus. Set corpus origin to `LINUX_KERNEL_BINARY_ORIGIN' if `is_linux_kernel' returns true. Verify the ctf reader status, now the ctf archive is 'opened' using `ctf_arc{open,bufopen}' depending if the corpus origin has `corpus::LINUX_KERNEL_BINARY_ORIGIN' bit set. Use `sort_{function,variables}' calls after extract ctf information. `ctf_close' is called from `read_context' destructor. (read:context::{set_read_context_corpus_group, reset_read_context, read_and_add_corpus_to_group_from_elf, dic_type_key): Add new member function implementation. * include/abg-tools-utils.h (build_corpus_group_from_kernel_dist_under): Add `origin' parameter with default `corpus::DWARF_ORIGIN'. * src/abg-tools-utils.cc: Use `abg-ctf-reader.h' file. (maybe_load_vmlinux_dwarf_corpus): Add new function. (maybe_load_vmlinux_ctf_corpus): Likewise. (build_corpus_group_from_kernel_dist_under): Update comments. Add new `origin' argument. Use `maybe_load_vmlinux_dwarf_corpus' or `maybe_load_vmlinux_ctf_corpus' according to `origin' value. * src/abg-corpus.h (corpus::origin): Update `origin' type values in enum. * src/abg-corpus-priv.h (corpus::priv): Replace `origin' type from `corpus::origin' to `uint32_t'. * src/abg-corpus.cc (corpus::{get,set}_origin): Replace data type from `corpus::origin' to `uint32_t'. * tools/abidw.cc (main): Use of --ctf argument to set format debug. * tests/test-read-ctf.cc: Add new tests to harness. * tests/data/test-read-ctf/test-PR27700.abi: New test expected result. * tests/data/test-read-ctf/test-anonymous-fields.o.abi: Likewise. * tests/data/test-read-ctf/test-enum-many-ctf.o.hash.abi: Likewise. * tests/data/test-read-ctf/test-enum-many.o.hash.abi: Likewise. * tests/data/test-read-ctf/test-enum-symbol-ctf.o.hash.abi: Likewise. * tests/data/test-read-common/test-PR26568-2.o: Adjust. * tests/data/test-read-ctf/test-PR26568-1.o.abi: Likewise. * tests/data/test-read-ctf/test-PR26568-2.o.abi: Likewise. * tests/data/test-read-ctf/test-ambiguous-struct-A.o.hash.abi: Likewise. * tests/data/test-read-ctf/test-ambiguous-struct-B.c: Likewise. * tests/data/test-read-ctf/test-ambiguous-struct-B.o: Likewise. * tests/data/test-read-ctf/test-ambiguous-struct-B.o.hash.abi: Likewise. * tests/data/test-read-ctf/test-array-of-pointers.abi: Likewise. * tests/data/test-read-ctf/test-callback.abi: Likewise. * tests/data/test-read-ctf/test-callback2.abi: Likewise. * tests/data/test-read-ctf/test-conflicting-type-syms-a.o.hash.abi: Likewise. * tests/data/test-read-ctf/test-conflicting-type-syms-b.o.hash.abi: Likewise. * tests/data/test-read-ctf/test-dynamic-array.o.abi: Likewise. * tests/data/test-read-ctf/test-enum-ctf.o.abi: Likewise. * tests/data/test-read-ctf/test-enum-symbol.o.hash.abi: Likewise. * tests/data/test-read-ctf/test-enum.o.abi: Likewise. * tests/data/test-read-ctf/test-forward-type-decl.abi: Likewise. * tests/data/test-read-ctf/test-functions-declaration.abi: Likewise. * tests/data/test-read-ctf/test-list-struct.abi: Likewise. * tests/data/test-read-ctf/test0: Likewise. * tests/data/test-read-ctf/test0.abi: Likewise. * tests/data/test-read-ctf/test0.c: Likewise. * tests/data/test-read-ctf/test0.hash.abi: Likewise. * tests/data/test-read-ctf/test1.so.abi: Likewise. * tests/data/test-read-ctf/test1.so.hash.abi: Likewise. * tests/data/test-read-ctf/test2.so.abi: Likewise. * tests/data/test-read-ctf/test2.so.hash.abi: Likewise. * tests/data/test-read-ctf/test3.so.abi: Likewise. * tests/data/test-read-ctf/test3.so.hash.abi: Likewise. * tests/data/test-read-ctf/test4.so.abi: Likewise. * tests/data/test-read-ctf/test4.so.hash.abi: Likewise. * tests/data/test-read-ctf/test5.o.abi: Likewise. * tests/data/test-read-ctf/test7.o.abi: Likewise. * tests/data/test-read-ctf/test8.o.abi: Likewise. * tests/data/test-read-ctf/test9.o.abi: Likewise. Signed-off-by: Guillermo E. Martinez Signed-off-by: Guillermo E. Martinez Signed-off-by: Dodji Seketeli Signed-off-by: Guillermo E. Martinez Signed-off-by: Dodji Seketeli --- include/abg-corpus.h | 12 +- include/abg-ctf-reader.h | 16 + include/abg-tools-utils.h | 3 +- src/abg-corpus-priv.h | 2 +- src/abg-corpus.cc | 4 +- src/abg-ctf-reader.cc | 492 ++++++++++++++---- src/abg-dwarf-reader.cc | 8 +- src/abg-ir.cc | 2 +- src/abg-tools-utils.cc | 317 ++++++++--- tests/data/test-read-common/test-PR26568-2.o | Bin 3048 -> 3488 bytes .../test-read-ctf/PR27700/test-PR27700.abi | 3 +- tests/data/test-read-ctf/test-PR26568-1.o.abi | 34 +- tests/data/test-read-ctf/test-PR26568-2.o.abi | 22 +- tests/data/test-read-ctf/test-PR27700.abi | 21 + .../test-ambiguous-struct-A.o.hash.abi | 3 +- .../test-read-ctf/test-ambiguous-struct-B.c | 2 +- .../test-read-ctf/test-ambiguous-struct-B.o | Bin 1344 -> 1344 bytes .../test-ambiguous-struct-B.o.hash.abi | 5 +- .../test-read-ctf/test-anonymous-fields.o.abi | 18 +- .../test-read-ctf/test-array-of-pointers.abi | 2 +- tests/data/test-read-ctf/test-callback.abi | 30 +- tests/data/test-read-ctf/test-callback2.abi | 2 +- .../test-conflicting-type-syms-a.o.hash.abi | 4 +- .../test-conflicting-type-syms-b.o.hash.abi | 4 +- .../test-read-ctf/test-dynamic-array.o.abi | 2 +- tests/data/test-read-ctf/test-enum-ctf.o.abi | 24 + .../test-enum-many-ctf.o.hash.abi | 69 +++ .../test-read-ctf/test-enum-many.o.hash.abi | 6 +- .../test-enum-symbol-ctf.o.hash.abi | 16 + .../test-read-ctf/test-enum-symbol.o.hash.abi | 3 +- tests/data/test-read-ctf/test-enum.o.abi | 6 +- .../test-read-ctf/test-forward-type-decl.abi | 2 +- .../test-functions-declaration.abi | 4 +- tests/data/test-read-ctf/test-list-struct.abi | 4 +- tests/data/test-read-ctf/test0 | Bin 16656 -> 16896 bytes tests/data/test-read-ctf/test0.abi | 30 +- tests/data/test-read-ctf/test0.c | 6 + tests/data/test-read-ctf/test0.hash.abi | 18 +- tests/data/test-read-ctf/test1.so.abi | 11 +- tests/data/test-read-ctf/test1.so.hash.abi | 7 +- tests/data/test-read-ctf/test2.so.abi | 8 + tests/data/test-read-ctf/test2.so.hash.abi | 8 + tests/data/test-read-ctf/test3.so.abi | 4 + tests/data/test-read-ctf/test3.so.hash.abi | 4 + tests/data/test-read-ctf/test4.so.abi | 6 + tests/data/test-read-ctf/test4.so.hash.abi | 6 + tests/data/test-read-ctf/test5.o.abi | 36 +- tests/data/test-read-ctf/test7.o.abi | 43 +- tests/data/test-read-ctf/test8.o.abi | 2 +- tests/data/test-read-ctf/test9.o.abi | 2 +- tests/test-read-ctf.cc | 20 +- tools/abidw.cc | 7 +- 52 files changed, 990 insertions(+), 370 deletions(-) create mode 100644 tests/data/test-read-ctf/test-PR27700.abi create mode 100644 tests/data/test-read-ctf/test-enum-ctf.o.abi create mode 100644 tests/data/test-read-ctf/test-enum-many-ctf.o.hash.abi create mode 100644 tests/data/test-read-ctf/test-enum-symbol-ctf.o.hash.abi base-commit: c96463e1ad974b7c4561886d7a3aa8a3c9a35607 diff --git a/include/abg-corpus.h b/include/abg-corpus.h index 652a8294..690edba5 100644 --- a/include/abg-corpus.h +++ b/include/abg-corpus.h @@ -44,10 +44,10 @@ public: enum origin { ARTIFICIAL_ORIGIN = 0, - NATIVE_XML_ORIGIN, - DWARF_ORIGIN, - CTF_ORIGIN, - LINUX_KERNEL_BINARY_ORIGIN + NATIVE_XML_ORIGIN = 1, + DWARF_ORIGIN = 1 << 1, + CTF_ORIGIN = 1 << 2, + LINUX_KERNEL_BINARY_ORIGIN = 1 << 3 }; private: @@ -115,11 +115,11 @@ public: corpus_group* get_group(); - origin + uint32_t get_origin() const; void - set_origin(origin); + set_origin(uint32_t); string& get_format_major_version_number() const; diff --git a/include/abg-ctf-reader.h b/include/abg-ctf-reader.h index 3343f0d8..ba7289aa 100644 --- a/include/abg-ctf-reader.h +++ b/include/abg-ctf-reader.h @@ -19,6 +19,8 @@ #include "abg-suppression.h" #include "abg-elf-reader-common.h" +#include "ctf-api.h" + namespace abigail { namespace ctf_reader @@ -32,8 +34,22 @@ create_read_context(const std::string& elf_path, ir::environment *env); corpus_sptr read_corpus(read_context *ctxt, elf_reader::status& status); + corpus_sptr read_corpus(const read_context_sptr &ctxt, elf_reader::status &status); + +corpus_sptr +read_and_add_corpus_to_group_from_elf(read_context*, corpus_group&, elf_reader::status&); + +void +set_read_context_corpus_group(read_context& ctxt, corpus_group_sptr& group); + +void +reset_read_context(read_context_sptr &ctxt, + const std::string& elf_path, + ir::environment* environment); +std::string +dic_type_key(ctf_dict_t *dic, ctf_id_t ctf_type); } // end namespace ctf_reader } // end namespace abigail diff --git a/include/abg-tools-utils.h b/include/abg-tools-utils.h index 68e54028..f7dccb24 100644 --- a/include/abg-tools-utils.h +++ b/include/abg-tools-utils.h @@ -311,7 +311,8 @@ build_corpus_group_from_kernel_dist_under(const string& root, vector& kabi_wl_paths, suppr::suppressions_type& supprs, bool verbose, - environment_sptr& env); + environment_sptr& env, + corpus::origin origin = corpus::DWARF_ORIGIN); }// end namespace tools_utils /// A macro that expands to aborting the program when executed. diff --git a/src/abg-corpus-priv.h b/src/abg-corpus-priv.h index 778e3365..28fd1d9b 100644 --- a/src/abg-corpus-priv.h +++ b/src/abg-corpus-priv.h @@ -670,7 +670,7 @@ struct corpus::priv environment* env; corpus_group* group; corpus::exported_decls_builder_sptr exported_decls_builder; - origin origin_; + uint32_t origin_; vector regex_patterns_fns_to_suppress; vector regex_patterns_vars_to_suppress; vector regex_patterns_fns_to_keep; diff --git a/src/abg-corpus.cc b/src/abg-corpus.cc index a517f384..94f2268c 100644 --- a/src/abg-corpus.cc +++ b/src/abg-corpus.cc @@ -851,7 +851,7 @@ corpus::init_format_version() /// Getter for the origin of the corpus. /// /// @return the origin of the corpus. -corpus::origin +uint32_t corpus::get_origin() const {return priv_->origin_;} @@ -859,7 +859,7 @@ corpus::get_origin() const /// /// @param o the new origin for the corpus. void -corpus::set_origin(origin o) +corpus::set_origin(uint32_t o) {priv_->origin_ = o;} /// Getter of the major version number of the abixml serialization diff --git a/src/abg-ctf-reader.cc b/src/abg-ctf-reader.cc index 2c6839cb..030f8fc8 100644 --- a/src/abg-ctf-reader.cc +++ b/src/abg-ctf-reader.cc @@ -16,6 +16,8 @@ #include /* For open(3) */ #include #include +#include +#include #include "ctf-api.h" @@ -58,13 +60,19 @@ public: /// A map associating CTF type ids with libabigail IR types. This /// is used to reuse already generated types. - unordered_map types_map; + std::map types_map; + + /// A set associating unknown CTF type ids + std::set unknown_types_set; /// libelf handler for the ELF file from which we read the CTF data, /// and the corresponding file descriptor. Elf *elf_handler; int elf_fd; + /// set when ELF is ET_EXEC + bool is_elf_exec; + /// The symtab read from the ELF file. symtab_reader::symtab_sptr symtab; @@ -74,26 +82,109 @@ public: ctf_sect_t symtab_sect; ctf_sect_t strtab_sect; + corpus_sptr cur_corpus_; + corpus_group_sptr cur_corpus_group_; + + /// Getter of the current corpus group being constructed. + /// + /// @return current the current corpus being constructed, if any, or + /// nil. + const corpus_group_sptr + current_corpus_group() const + {return cur_corpus_group_;} + + /// Test if there is a corpus group being built. + /// + /// @return if there is a corpus group being built, false otherwise. + bool + has_corpus_group() const + {return bool(cur_corpus_group_);} + + /// Return the main corpus from the current corpus group, if any. + /// + /// @return the main corpus of the current corpus group, if any, nil + /// if no corpus group is being constructed. + corpus_sptr + main_corpus_from_current_group() + { + if (cur_corpus_group_) + return cur_corpus_group_->get_main_corpus(); + return corpus_sptr(); + } + + /// Test if the current corpus being built is the main corpus of the + /// current corpus group. + /// + /// @return return true iff the current corpus being built is the + /// main corpus of the current corpus group. + bool + current_corpus_is_main_corpus_from_current_group() + { + corpus_sptr main_corpus = main_corpus_from_current_group(); + + if (main_corpus && main_corpus.get() == cur_corpus_.get()) + return true; + + return false; + } + + /// Return true if the current corpus is part of a corpus group + /// being built and if it's not the main corpus of the group. + /// + /// For instance, this would return true if we are loading a linux + /// kernel *module* that is part of the current corpus group that is + /// being built. In this case, it means we should re-use types + /// coming from the "vmlinux" binary that is the main corpus of the + /// group. + /// + /// @return the corpus group the current corpus belongs to, if the + /// current corpus is part of a corpus group being built. Nil otherwise. + corpus_sptr + should_reuse_type_from_corpus_group() + { + if (has_corpus_group()) + if (corpus_sptr main_corpus = main_corpus_from_current_group()) + if (!current_corpus_is_main_corpus_from_current_group()) + return current_corpus_group(); + + return corpus_sptr(); + } + /// Associate a given CTF type ID with a given libabigail IR type. - void add_type(ctf_id_t ctf_type, type_base_sptr type) + void add_type(ctf_dict_t *dic, ctf_id_t ctf_type, type_base_sptr type) { - types_map.insert(std::make_pair(ctf_type, type)); + string key = dic_type_key(dic, ctf_type); + types_map.insert(std::make_pair(key, type)); + } + + /// Insert a given CTF unknown type ID. + /// + /// @param ctf_type the unknown type ID to be added. + void add_unknown_type(ctf_id_t ctf_type) + { + unknown_types_set.insert(ctf_type); } /// Lookup a given CTF type ID in the types map. /// /// @param ctf_type the type ID of the type to lookup. - type_base_sptr lookup_type(ctf_id_t ctf_type) + type_base_sptr lookup_type(ctf_dict_t *dic, ctf_id_t ctf_type) { type_base_sptr result; + std::string key = dic_type_key(dic, ctf_type); - auto search = types_map.find(ctf_type); + auto search = types_map.find(key); if (search != types_map.end()) result = search->second; return result; } + /// Lookup a given CTF unknown type ID in the unknown set. + /// @param ctf_type the unknown type ID to lookup. + bool lookup_unknown_type(ctf_id_t ctf_type) + { return unknown_types_set.find(ctf_type) != unknown_types_set.end(); } + /// Canonicalize all the types stored in the types map. void canonicalize_all_types(void) { @@ -103,18 +194,42 @@ public: /// Constructor. /// + /// ctfa data member can be used per courpus group. + /// /// @param elf_path the path to the ELF file. - read_context(const string& elf_path, ir::environment *env) + read_context(const string& elf_path, ir::environment *env) : + ctfa(NULL) + { + initialize(elf_path, env); + } + + /// Initializer of read_context. + /// + /// @param elf_path the path to the elf file the context is to be + /// used for. + /// + /// @param environment the environment used by the current context. + /// This environment contains resources needed by the reader and by + /// the types and declarations that are to be created later. Note + /// that ABI artifacts that are to be compared all need to be + /// created within the same environment. + /// + /// Please also note that the life time of this environment object + /// must be greater than the life time of the resulting @ref + /// read_context the context uses resources that are allocated in + /// the environment. + void initialize(const string& elf_path, ir::environment *env) { types_map.clear(); filename = elf_path; ir_env = env; elf_handler = NULL; elf_fd = -1; - ctfa = NULL; + is_elf_exec = false; + symtab.reset(); + cur_corpus_group_.reset(); } - /// Destructor of the @ref read_context type. ~read_context() { ctf_close(ctfa); @@ -153,13 +268,18 @@ process_ctf_typedef(read_context *ctxt, return result; const char *typedef_name = ctf_type_name_raw(ctf_dictionary, ctf_type); + if (corpus_sptr corp = ctxt->should_reuse_type_from_corpus_group()) + if (result = lookup_typedef_type(typedef_name, *corp)) + return result; + type_base_sptr utype = lookup_type(ctxt, corp, tunit, ctf_dictionary, ctf_utype); if (!utype) return result; - result = dynamic_pointer_cast(ctxt->lookup_type(ctf_type)); + result = dynamic_pointer_cast(ctxt->lookup_type(ctf_dictionary, + ctf_type)); if (result) return result; @@ -180,7 +300,7 @@ process_ctf_typedef(read_context *ctxt, if (result) { add_decl_to_scope(result, tunit->get_global_scope()); - ctxt->add_type(ctf_type, result); + ctxt->add_type(ctf_dictionary, ctf_type, result); } return result; @@ -225,9 +345,20 @@ process_ctf_base_type(read_context *ctxt, type_base_sptr void_type = ctxt->ir_env->get_void_type(); decl_base_sptr type_declaration = get_type_declaration(void_type); result = is_type_decl(type_declaration); + canonicalize(result); } else { + if (corpus_sptr corp = ctxt->should_reuse_type_from_corpus_group()) + { + string normalized_type_name = type_name; + integral_type int_type; + if (parse_integral_type(type_name, int_type)) + normalized_type_name = int_type.to_string(); + if (result = lookup_basic_type(normalized_type_name, *corp)) + return result; + } + result = lookup_basic_type(type_name, *corp); if (!result) result.reset(new type_decl(ctxt->ir_env, @@ -242,7 +373,7 @@ process_ctf_base_type(read_context *ctxt, if (result) { add_decl_to_scope(result, tunit->get_global_scope()); - ctxt->add_type(ctf_type, result); + ctxt->add_type(ctf_dictionary, ctf_type, result); } return result; @@ -303,7 +434,8 @@ process_ctf_function_type(read_context *ctxt, function_parms.push_back(parm); } - result = dynamic_pointer_cast(ctxt->lookup_type(ctf_type)); + result = dynamic_pointer_cast(ctxt->lookup_type(ctf_dictionary, + ctf_type)); if (result) return result; @@ -319,7 +451,7 @@ process_ctf_function_type(read_context *ctxt, result->set_is_artificial(true); decl_base_sptr function_type_decl = get_type_declaration(result); add_decl_to_scope(function_type_decl, tunit->get_global_scope()); - ctxt->add_type(ctf_type, result); + ctxt->add_type(ctf_dictionary, ctf_type, result); } return result; @@ -419,6 +551,11 @@ process_ctf_forward_type(read_context *ctxt, } else { + if (!type_is_anonymous) + if (corpus_sptr corp = ctxt->should_reuse_type_from_corpus_group()) + if (result = lookup_class_type(type_name, *corp)) + return is_type(result); + class_decl_sptr struct_fwd(new class_decl(ctxt->ir_env, type_name, /*alignment=*/0, /*size=*/0, @@ -434,7 +571,7 @@ process_ctf_forward_type(read_context *ctxt, return is_type(result); add_decl_to_scope(result, tunit->get_global_scope()); - ctxt->add_type(ctf_type, is_type(result)); + ctxt->add_type(ctf_dictionary, ctf_type, is_type(result)); return is_type(result); } @@ -458,9 +595,14 @@ process_ctf_struct_type(read_context *ctxt, { class_decl_sptr result; std::string struct_type_name = ctf_type_name_raw(ctf_dictionary, - ctf_type); + ctf_type); bool struct_type_is_anonymous = (struct_type_name == ""); + if (!struct_type_is_anonymous) + if (corpus_sptr corp = ctxt->should_reuse_type_from_corpus_group()) + if (result = lookup_class_type(struct_type_name, *corp)) + return result; + /* The libabigail IR encodes C struct types in `class' IR nodes. */ result.reset(new class_decl(ctxt->ir_env, struct_type_name, @@ -479,7 +621,7 @@ process_ctf_struct_type(read_context *ctxt, at this point even if the members haven't been added to the IR node yet. */ add_decl_to_scope(result, tunit->get_global_scope()); - ctxt->add_type(ctf_type, result); + ctxt->add_type(ctf_dictionary, ctf_type, result); /* Now add the struct members as specified in the CTF type description. This is C, so named types can only be defined in the global @@ -512,6 +654,11 @@ process_ctf_union_type(read_context *ctxt, ctf_type); bool union_type_is_anonymous = (union_type_name == ""); + if (!union_type_is_anonymous) + if (corpus_sptr corp = ctxt->should_reuse_type_from_corpus_group()) + if (result = lookup_union_type(union_type_name, *corp)) + return result; + /* Create the corresponding libabigail union IR node. */ result.reset(new union_decl(ctxt->ir_env, union_type_name, @@ -528,7 +675,7 @@ process_ctf_union_type(read_context *ctxt, at this point even if the members haven't been added to the IR node yet. */ add_decl_to_scope(result, tunit->get_global_scope()); - ctxt->add_type(ctf_type, result); + ctxt->add_type(ctf_dictionary, ctf_type, result); /* Now add the union members as specified in the CTF type description. This is C, so named types can only be defined in the global @@ -584,7 +731,8 @@ process_ctf_array_type(read_context *ctxt, if (!index_type) return result; - result = dynamic_pointer_cast(ctxt->lookup_type(ctf_type)); + result = dynamic_pointer_cast(ctxt->lookup_type(ctf_dictionary, + ctf_type)); if (result) return result; @@ -623,7 +771,7 @@ process_ctf_array_type(read_context *ctxt, { decl_base_sptr array_type_decl = get_type_declaration(result); add_decl_to_scope(array_type_decl, tunit->get_global_scope()); - ctxt->add_type(ctf_type, result); + ctxt->add_type(ctf_dictionary, ctf_type, result); } return result; @@ -652,6 +800,11 @@ process_ctf_qualified_type(read_context *ctxt, if (!utype) return result; + result = dynamic_pointer_cast(ctxt->lookup_type(ctf_dictionary, + ctf_type)); + if (result) + return result; + qualified_type_def::CV qualifiers = qualified_type_def::CV_NONE; if (type_kind == CTF_K_CONST) qualifiers |= qualified_type_def::CV_CONST; @@ -668,7 +821,7 @@ process_ctf_qualified_type(read_context *ctxt, { decl_base_sptr qualified_type_decl = get_type_declaration(result); add_decl_to_scope(qualified_type_decl, tunit->get_global_scope()); - ctxt->add_type(ctf_type, result); + ctxt->add_type(ctf_dictionary, ctf_type, result); } return result; @@ -702,7 +855,8 @@ process_ctf_pointer_type(read_context *ctxt, if (!target_type) return result; - result = dynamic_pointer_cast(ctxt->lookup_type(ctf_type)); + result = dynamic_pointer_cast(ctxt->lookup_type(ctf_dictionary, + ctf_type)); if (result) return result; @@ -713,7 +867,7 @@ process_ctf_pointer_type(read_context *ctxt, if (result) { add_decl_to_scope(result, tunit->get_global_scope()); - ctxt->add_type(ctf_type, result); + ctxt->add_type(ctf_dictionary, ctf_type, result); } return result; @@ -736,6 +890,12 @@ process_ctf_enum_type(read_context *ctxt, ctf_id_t ctf_type) { enum_type_decl_sptr result; + std::string enum_name = ctf_type_name_raw(ctf_dictionary, ctf_type); + + if (!enum_name.empty()) + if (corpus_sptr corp = ctxt->should_reuse_type_from_corpus_group()) + if (result = lookup_enum_type(enum_name, *corp)) + return result; /* Build a signed integral type for the type of the enumerators, aka the underlying type. The size of the enumerators in bytes is @@ -769,13 +929,12 @@ process_ctf_enum_type(read_context *ctxt, return result; } - const char *enum_name = ctf_type_name_raw(ctf_dictionary, ctf_type); - result.reset(new enum_type_decl(enum_name, location(), - utype, enms, enum_name)); + result.reset(new enum_type_decl(enum_name.c_str(), location(), + utype, enms, enum_name.c_str())); if (result) { add_decl_to_scope(result, tunit->get_global_scope()); - ctxt->add_type(ctf_type, result); + ctxt->add_type(ctf_dictionary, ctf_type, result); } return result; @@ -804,7 +963,10 @@ process_ctf_type(read_context *ctxt, int type_kind = ctf_type_kind(ctf_dictionary, ctf_type); type_base_sptr result; - if ((result = ctxt->lookup_type(ctf_type))) + if (ctxt->lookup_unknown_type(ctf_type)) + return nullptr; + + if ((result = ctxt->lookup_type(ctf_dictionary, ctf_type))) return result; switch (type_kind) @@ -889,7 +1051,10 @@ process_ctf_type(read_context *ctxt, } if (!result) - fprintf(stderr, "NOT PROCESSED TYPE %lu\n", ctf_type); + { + fprintf(stderr, "NOT PROCESSED TYPE %lu\n", ctf_type); + ctxt->add_unknown_type(ctf_type); + } return result; } @@ -913,7 +1078,7 @@ lookup_type(read_context *ctxt, corpus_sptr corp, translation_unit_sptr tunit, ctf_dict_t *ctf_dictionary, ctf_id_t ctf_type) { - type_base_sptr result = ctxt->lookup_type(ctf_type); + type_base_sptr result = ctxt->lookup_type(ctf_dictionary, ctf_type); if (!result) result = process_ctf_type(ctxt, corp, tunit, ctf_dictionary, ctf_type); @@ -921,8 +1086,8 @@ lookup_type(read_context *ctxt, corpus_sptr corp, } /// Process a CTF archive and create libabigail IR for the types, -/// variables and function declarations found in the archive. The IR -/// is added to the given corpus. +/// variables and function declarations found in the archive, iterating +/// over public symbols. The IR is added to the given corpus. /// /// @param ctxt the read context containing the CTF archive to /// process. @@ -937,43 +1102,49 @@ process_ctf_archive(read_context *ctxt, corpus_sptr corp) ir_translation_unit->set_language(translation_unit::LANG_C); corp->add(ir_translation_unit); - /* Iterate over the CTF dictionaries in the archive. */ int ctf_err; ctf_dict_t *ctf_dict; - ctf_next_t *dict_next = NULL; - const char *archive_name; + const auto symtab = ctxt->symtab; + symtab_reader::symtab_filter filter = symtab->make_filter(); + filter.set_public_symbols(); + std::string dict_name; - while ((ctf_dict = ctf_archive_next(ctxt->ctfa, &dict_next, &archive_name, - 0 /* skip_parent */, &ctf_err)) != NULL) + if (corp->get_origin() & corpus::LINUX_KERNEL_BINARY_ORIGIN) { - /* Iterate over the CTF types stored in this archive. */ - ctf_id_t ctf_type; - int type_flag; - ctf_next_t *type_next = NULL; + tools_utils::base_name(ctxt->filename, dict_name); - while ((ctf_type = ctf_type_next(ctf_dict, &type_next, &type_flag, - 1 /* want_hidden */)) != CTF_ERR) - { - process_ctf_type(ctxt, corp, ir_translation_unit, - ctf_dict, ctf_type); - } - if (ctf_errno(ctf_dict) != ECTF_NEXT_END) - fprintf(stderr, "ERROR from ctf_type_next\n"); + if (dict_name != "vmlinux") + // remove .ko suffix + dict_name.erase(dict_name.length() - 3, 3); - /* Canonicalize all the types generated above. This must be - done "a posteriori" because the processing of types may - require other related types to not be already - canonicalized. */ - ctxt->canonicalize_all_types(); + std::replace(dict_name.begin(), dict_name.end(), '-', '_'); + } - /* Iterate over the CTF variables stored in this archive. */ + if ((ctf_dict = ctf_dict_open(ctxt->ctfa, + dict_name.empty() ? NULL : dict_name.c_str(), + &ctf_err)) == NULL) + { + fprintf(stderr, "ERROR dictionary not found\n"); + abort(); + } + + for (const auto& symbol : symtab_reader::filtered_symtab(*symtab, filter)) + { + std::string sym_name = symbol->get_name(); ctf_id_t ctf_var_type; - ctf_next_t *var_next = NULL; - const char *var_name; - while ((ctf_var_type = ctf_variable_next(ctf_dict, &var_next, &var_name)) - != CTF_ERR) + if ((corp->get_origin() & corpus::LINUX_KERNEL_BINARY_ORIGIN) + || ctxt->is_elf_exec) + ctf_var_type= ctf_lookup_variable (ctf_dict, sym_name.c_str()); + else + ctf_var_type = ctf_lookup_by_symbol_name(ctf_dict, sym_name.c_str()); + + if (ctf_var_type == (ctf_id_t) -1) + continue; + + if (ctf_type_kind (ctf_dict, ctf_var_type) != CTF_K_FUNCTION) { + const char *var_name = sym_name.c_str(); type_base_sptr var_type = lookup_type(ctxt, corp, ir_translation_unit, ctf_dict, ctf_var_type); if (!var_type) @@ -986,50 +1157,38 @@ process_ctf_archive(read_context *ctxt, corpus_sptr corp) location(), var_name)); + var_declaration->set_symbol(symbol); add_decl_to_scope(var_declaration, - ir_translation_unit->get_global_scope()); + ir_translation_unit->get_global_scope()); } - if (ctf_errno(ctf_dict) != ECTF_NEXT_END) - fprintf(stderr, "ERROR from ctf_variable_next\n"); + else + { + const char *func_name = sym_name.c_str(); + ctf_id_t ctf_sym = ctf_var_type; + type_base_sptr func_type = lookup_type(ctxt, corp, ir_translation_unit, + ctf_dict, ctf_sym); + if (!func_type) + /* Ignore function if its type can't be sorted out. */ + continue; - /* Iterate over the CTF functions stored in this archive. */ - ctf_next_t *func_next = NULL; - const char *func_name = NULL; - ctf_id_t ctf_sym; + function_decl_sptr func_declaration; + func_declaration.reset(new function_decl(func_name, + func_type, + 0 /* is_inline */, + location())); - while ((ctf_sym = ctf_symbol_next(ctf_dict, &func_next, &func_name, - 1 /* functions symbols only */) != CTF_ERR)) - { - ctf_id_t ctf_func_type = ctf_lookup_by_name(ctf_dict, func_name); - type_base_sptr func_type = lookup_type(ctxt, corp, ir_translation_unit, - ctf_dict, ctf_func_type); - if (!func_type) - /* Ignore function if its type can't be sorted out. */ - continue; - - elf_symbols func_elf_symbols = ctxt->symtab->lookup_symbol(func_name); - if (func_elf_symbols.size() == 0 - || func_elf_symbols[0]->get_binding() == elf_symbol::LOCAL_BINDING) - /* Ignore local functions. */ - continue; - - function_decl_sptr func_declaration; - func_declaration.reset(new function_decl(func_name, - func_type, - 0 /* is_inline */, - location())); - - add_decl_to_scope(func_declaration, - ir_translation_unit->get_global_scope()); - } - if (ctf_errno(ctf_dict) != ECTF_NEXT_END) - fprintf(stderr, "ERROR from ctf_symbol_next\n"); - - ctf_dict_close(ctf_dict); + func_declaration->set_symbol(symbol); + add_decl_to_scope(func_declaration, + ir_translation_unit->get_global_scope()); + } } - if (ctf_err != ECTF_NEXT_END) - fprintf(stderr, "ERROR from ctf_archive_next\n"); + ctf_dict_close(ctf_dict); + /* Canonicalize all the types generated above. This must be + done "a posteriori" because the processing of types may + require other related types to not be already + canonicalized. */ + ctxt->canonicalize_all_types(); } /// Open the ELF file described by the given read context. @@ -1113,6 +1272,7 @@ slurp_elf_info(read_context *ctxt, corpus_sptr corp) /* Set the ELF architecture. */ GElf_Ehdr eh_mem; GElf_Ehdr *ehdr = gelf_getehdr(ctxt->elf_handler, &eh_mem); + ctxt->is_elf_exec = (ehdr->e_type == ET_EXEC); corp->set_architecture_name(elf_helpers::e_machine_to_string(ehdr->e_machine)); /* Read the symtab from the ELF file and set it in the corpus. */ @@ -1121,6 +1281,9 @@ slurp_elf_info(read_context *ctxt, corpus_sptr corp) 0 /* No suppressions. */); corp->set_symtab(ctxt->symtab); + if (corp->get_origin() & corpus::LINUX_KERNEL_BINARY_ORIGIN) + return 1; + /* Get the raw ELF section contents for libctf. */ Elf_Scn *ctf_scn = elf_helpers::find_section(ctxt->elf_handler, ".ctf", SHT_PROGBITS); Elf_Scn *symtab_scn = elf_helpers::find_symbol_table_section(ctxt->elf_handler); @@ -1167,6 +1330,7 @@ read_corpus(read_context *ctxt, elf_reader::status &status) corpus_sptr corp = std::make_shared(ctxt->ir_env, ctxt->filename); + ctxt->cur_corpus_ = corp; /* Be optimist. */ status = elf_reader::STATUS_OK; @@ -1177,25 +1341,52 @@ read_corpus(read_context *ctxt, elf_reader::status &status) return corp; } - /* Set some properties of the corpus first. */ - corp->set_origin(corpus::CTF_ORIGIN); - if (!slurp_elf_info(ctxt, corp)) + bool is_linux_kernel = elf_helpers::is_linux_kernel(ctxt->elf_handler); + uint32_t origin = corpus::CTF_ORIGIN; + + if (is_linux_kernel) + origin |= corpus::LINUX_KERNEL_BINARY_ORIGIN; + corp->set_origin(origin); + + if (ctxt->cur_corpus_group_) + ctxt->cur_corpus_group_->add_corpus(ctxt->cur_corpus_); + + if (!slurp_elf_info(ctxt, corp) && !is_linux_kernel) { status = elf_reader::STATUS_NO_SYMBOLS_FOUND; return corp; } - /* Build the ctfa from the contents of the relevant ELF sections, - and process the CTF archive in the read context, if any. - Information about the types, variables, functions, etc contained - in the archive are added to the given corpus. */ int errp; - ctxt->ctfa = ctf_arc_bufopen(&ctxt->ctf_sect, &ctxt->symtab_sect, - &ctxt->strtab_sect, &errp); + if (corp->get_origin() & corpus::LINUX_KERNEL_BINARY_ORIGIN) + { + std::string filename; + if (tools_utils::base_name(ctxt->filename, filename) + && filename == "vmlinux") + { + std::string vmlinux_ctfa_path = ctxt->filename + ".ctfa"; + ctxt->ctfa = ctf_arc_open(vmlinux_ctfa_path.c_str(), &errp); + } + } + else + /* Build the ctfa from the contents of the relevant ELF sections, + and process the CTF archive in the read context, if any. + Information about the types, variables, functions, etc contained + in the archive are added to the given corpus. */ + ctxt->ctfa = ctf_arc_bufopen(&ctxt->ctf_sect, &ctxt->symtab_sect, + &ctxt->strtab_sect, &errp); + + ctxt->ir_env->canonicalization_is_done(false); if (ctxt->ctfa == NULL) status = elf_reader::STATUS_DEBUG_INFO_NOT_FOUND; else - process_ctf_archive(ctxt, corp); + { + process_ctf_archive(ctxt, corp); + ctxt->cur_corpus_->sort_functions(); + ctxt->cur_corpus_->sort_variables(); + } + + ctxt->ir_env->canonicalization_is_done(true); /* Cleanup and return. */ close_elf_handler(ctxt); @@ -1216,5 +1407,94 @@ corpus_sptr read_corpus(const read_context_sptr &ctxt, elf_reader::status &status) {return read_corpus(ctxt.get(), status);} +/// Set the @ref corpus_group being created to the current read context. +/// +/// @param ctxt the read_context to consider. +/// +/// @param group the @ref corpus_group to set. +void +set_read_context_corpus_group(read_context& ctxt, + corpus_group_sptr& group) +{ + ctxt.cur_corpus_group_ = group; +} + +/// Read a corpus and add it to a given @ref corpus_group. +/// +/// @param ctxt the reading context to consider. +/// +/// @param group the @ref corpus_group to add the new corpus to. +/// +/// @param status output parameter. The status of the read. It is set +/// by this function upon its completion. +corpus_sptr +read_and_add_corpus_to_group_from_elf(read_context* ctxt, + corpus_group& group, + elf_reader::status& status) +{ + corpus_sptr result; + corpus_sptr corp = read_corpus(ctxt, status); + if (status & elf_reader::STATUS_OK) + { + if (!corp->get_group()) + group.add_corpus(corp); + result = corp; + } + + return result; +} + +/// Re-initialize a read_context so that it can re-used to read +/// another binary. +/// +/// @param ctxt the context to re-initialize. +/// +/// @param elf_path the path to the elf file the context is to be used +/// for. +/// +/// @param environment the environment used by the current context. +/// This environment contains resources needed by the reader and by +/// the types and declarations that are to be created later. Note +/// that ABI artifacts that are to be compared all need to be created +/// within the same environment. +/// +/// Please also note that the life time of this environment object +/// must be greater than the life time of the resulting @ref +/// read_context the context uses resources that are allocated in the +/// environment. +void +reset_read_context(read_context_sptr &ctxt, + const std::string& elf_path, + ir::environment* environment) +{ + if (ctxt) + ctxt->initialize(elf_path, environment); +} + +/// Returns a key to be use in types_map dict conformed by +/// dictionary id and the CTF type id for a given type. +/// +/// CTF id types are unique by child dictionary, but CTF id +/// types in parent dictionary are unique across the all +/// dictionaries in the CTF archive, to differentiate +/// one each other this member function relies in +/// ctf_type_isparent function. +/// +/// @param dic the pointer to CTF dictionary where the @p type +/// was found. +/// +/// @param type the id for given CTF type. +std::string +dic_type_key(ctf_dict_t *dic, ctf_id_t ctf_type) +{ + std::stringstream key; + + if (ctf_type_isparent (dic, ctf_type)) + key << std::hex << ctf_type; + else + key << std::hex << ctf_type << '-' << ctf_cuname(dic); + return key.str(); +} + } // End of namespace ctf_reader } // End of namespace abigail diff --git a/src/abg-dwarf-reader.cc b/src/abg-dwarf-reader.cc index dc82cf3b..949161e2 100644 --- a/src/abg-dwarf-reader.cc +++ b/src/abg-dwarf-reader.cc @@ -14444,10 +14444,12 @@ read_debug_info_into_corpus(read_context& ctxt) // First set some mundane properties of the corpus gathered from // ELF. ctxt.current_corpus()->set_path(ctxt.elf_path()); + + uint32_t origin = corpus::DWARF_ORIGIN; if (is_linux_kernel(ctxt.elf_handle())) - ctxt.current_corpus()->set_origin(corpus::LINUX_KERNEL_BINARY_ORIGIN); - else - ctxt.current_corpus()->set_origin(corpus::DWARF_ORIGIN); + origin |= corpus::LINUX_KERNEL_BINARY_ORIGIN; + ctxt.current_corpus()->set_origin(origin); + ctxt.current_corpus()->set_soname(ctxt.dt_soname()); ctxt.current_corpus()->set_needed(ctxt.dt_needed()); ctxt.current_corpus()->set_architecture_name(ctxt.elf_architecture()); diff --git a/src/abg-ir.cc b/src/abg-ir.cc index 0ef5e8b2..4e907620 100644 --- a/src/abg-ir.cc +++ b/src/abg-ir.cc @@ -13498,7 +13498,7 @@ types_defined_same_linux_kernel_corpus_public(const type_base& t1, /// kernel corpus, let's move on. Otherwise bail out. if (!(t1_corpus && t2_corpus && t1_corpus == t2_corpus - && (t1_corpus->get_origin() == corpus::LINUX_KERNEL_BINARY_ORIGIN) + && (t1_corpus->get_origin() & corpus::LINUX_KERNEL_BINARY_ORIGIN) && (is_class_or_union_type(&t1) || is_enum_type(&t1)))) return false; diff --git a/src/abg-tools-utils.cc b/src/abg-tools-utils.cc index 1f0f6fa8..f30c3f1d 100644 --- a/src/abg-tools-utils.cc +++ b/src/abg-tools-utils.cc @@ -44,6 +44,9 @@ #include #include "abg-dwarf-reader.h" +#ifdef WITH_CTF +#include "abg-ctf-reader.h" +#endif #include "abg-internal.h" #include "abg-regex.h" @@ -2486,6 +2489,223 @@ get_binary_paths_from_kernel_dist(const string& dist_root, module_paths); } +/// If the @ref origin is DWARF_ORIGIN it build a @ref corpus_group +/// made of vmlinux kernel file and the linux kernel modules found +/// under @p root directory and under its sub-directories, recursively. +/// +/// @param origin the debug type information in vmlinux kernel and +/// the linux kernel modules to be used to build the corpora @p group. +/// +/// @param the group @ref corpus_group to be built. +/// +/// @param vmlinux the path to the vmlinux binary. +/// +/// @param modules a vector with the paths to the linux kernel +/// modules. +/// +/// @param root the path of the directory under which the kernel +/// kernel modules were found. +/// +/// @param di_root the directory in aboslute path which debug +/// info is to be found for binaries under director @p root +/// +/// @param suppr_paths the paths to the suppression specifications to +/// apply while loading the binaries. +/// +/// @param kabi_wl_path the paths to the kabi whitelist files to take +/// into account while loading the binaries. +/// +/// @param supprs the suppressions resulting from parsing the +/// suppression specifications at @p suppr_paths. This is set by this +/// function. +/// +/// @param verbose true if the function has to emit some verbose +/// messages. +/// +/// @param t time to trace time spent in each step. +/// +/// @param env the environment to create the corpus_group in. +static void +maybe_load_vmlinux_dwarf_corpus(corpus::origin origin, + corpus_group_sptr& group, + const string& vmlinux, + vector& modules, + const string& root, + vector& di_roots, + vector& suppr_paths, + vector& kabi_wl_paths, + suppressions_type& supprs, + bool verbose, + timer& t, + environment_sptr& env) +{ + if (!(origin & corpus::DWARF_ORIGIN)) + return; + + abigail::elf_reader::status status = abigail::elf_reader::STATUS_OK; + dwarf_reader::read_context_sptr ctxt; + ctxt = + dwarf_reader::create_read_context(vmlinux, di_roots, env.get(), + /*read_all_types=*/false, + /*linux_kernel_mode=*/true); + dwarf_reader::set_do_log(*ctxt, verbose); + + t.start(); + load_generate_apply_suppressions(*ctxt, suppr_paths, + kabi_wl_paths, supprs); + t.stop(); + + if (verbose) + std::cerr << "loaded white list and generated suppr spec in: " + << t + << "\n"; + + group.reset(new corpus_group(env.get(), root)); + + set_read_context_corpus_group(*ctxt, group); + + if (verbose) + std::cerr << "reading kernel binary '" + << vmlinux << "' ...\n" << std::flush; + + // Read the vmlinux corpus and add it to the group. + t.start(); + read_and_add_corpus_to_group_from_elf(*ctxt, *group, status); + t.stop(); + + if (verbose) + std::cerr << vmlinux + << " reading DONE:" + << t << "\n"; + + if (group->is_empty()) + return; + + // Now add the corpora of the modules to the corpus group. + int total_nb_modules = modules.size(); + int cur_module_index = 1; + for (vector::const_iterator m = modules.begin(); + m != modules.end(); + ++m, ++cur_module_index) + { + if (verbose) + std::cerr << "reading module '" + << *m << "' (" + << cur_module_index + << "/" << total_nb_modules + << ") ... " << std::flush; + + reset_read_context(ctxt, *m, di_roots, env.get(), + /*read_all_types=*/false, + /*linux_kernel_mode=*/true); + + load_generate_apply_suppressions(*ctxt, suppr_paths, + kabi_wl_paths, supprs); + + set_read_context_corpus_group(*ctxt, group); + + t.start(); + read_and_add_corpus_to_group_from_elf(*ctxt, + *group, status); + t.stop(); + if (verbose) + std::cerr << "module '" + << *m + << "' reading DONE: " + << t << "\n"; + } +} + +/// If the @ref origin is CTF_ORIGIN it build a @ref corpus_group +/// made of vmlinux kernel file and the linux kernel modules found +/// under @p root directory and under its sub-directories, recursively. +/// +/// @param origin the debug type information in vmlinux kernel and +/// the linux kernel modules to be used to build the corpora @p group. +/// +/// @param group the @ref corpus_group to be built. +/// +/// @param vmlinux the path to the vmlinux binary. +/// +/// @param modules a vector with the paths to the linux kernel +/// modules. +/// +/// @param root the path of the directory under which the kernel +/// kernel modules were found. +/// +/// @param verbose true if the function has to emit some verbose +/// messages. +/// +/// @param t time to trace time spent in each step. +/// +/// @param env the environment to create the corpus_group in. +#ifdef WITH_CTF +static void +maybe_load_vmlinux_ctf_corpus(corpus::origin origin, + corpus_group_sptr& group, + const string& vmlinux, + vector& modules, + const string& root, + bool verbose, + timer& t, + environment_sptr& env) +{ + if (!(origin & corpus::CTF_ORIGIN)) + return; + + abigail::elf_reader::status status = abigail::elf_reader::STATUS_OK; + ctf_reader::read_context_sptr ctxt; + ctxt = ctf_reader::create_read_context(vmlinux, env.get()); + group.reset(new corpus_group(env.get(), root)); + set_read_context_corpus_group(*ctxt, group); + + if (verbose) + std::cerr << "reading kernel binary '" + << vmlinux << "' ...\n" << std::flush; + + // Read the vmlinux corpus and add it to the group. + t.start(); + read_and_add_corpus_to_group_from_elf(ctxt.get(), *group, status); + t.stop(); + + if (verbose) + std::cerr << vmlinux + << " reading DONE:" + << t << "\n"; + + if (group->is_empty()) + return; + + // Now add the corpora of the modules to the corpus group. + int total_nb_modules = modules.size(); + int cur_module_index = 1; + for (vector::const_iterator m = modules.begin(); + m != modules.end(); + ++m, ++cur_module_index) + { + if (verbose) + std::cerr << "reading module '" + << *m << "' (" + << cur_module_index + << "/" << total_nb_modules + << ") ... " << std::flush; + + reset_read_context(ctxt, *m, env.get()); + set_read_context_corpus_group(*ctxt, group); + + t.start(); + read_and_add_corpus_to_group_from_elf(ctxt.get(), + *group, status); + t.stop(); + if (verbose) + std::cerr << "module '" + << *m + << "' reading DONE: " + << t << "\n"; + } +} +#endif + /// Walk a given directory and build an instance of @ref corpus_group /// from the vmlinux kernel binary and the linux kernel modules found /// under that directory and under its sub-directories, recursively. @@ -2493,6 +2713,11 @@ get_binary_paths_from_kernel_dist(const string& dist_root, /// The main corpus of the @ref corpus_group is made of the vmlinux /// binary. The other corpora are made of the linux kernel binaries. /// +/// Depending of the @ref origin it delegates the corpus build @p group +/// to: +/// @ref maybe_load_vmlinux_dwarf_corpus +/// @ref maybe_load_vmlinux_ctf_corpus +/// /// @param root the path of the directory under which the kernel /// kernel modules are to be found. The vmlinux can also be found /// somewhere under that directory, but if it's not in there, its path @@ -2528,10 +2753,11 @@ build_corpus_group_from_kernel_dist_under(const string& root, vector& kabi_wl_paths, suppressions_type& supprs, bool verbose, - environment_sptr& env) + environment_sptr& env, + corpus::origin origin) { string vmlinux = vmlinux_path; - corpus_group_sptr result; + corpus_group_sptr group; vector modules; if (verbose) @@ -2548,7 +2774,6 @@ build_corpus_group_from_kernel_dist_under(const string& root, if (verbose) std::cerr << "DONE: " << t << "\n"; - dwarf_reader::read_context_sptr ctxt; if (got_binary_paths) { shared_ptr di_root = @@ -2556,86 +2781,18 @@ build_corpus_group_from_kernel_dist_under(const string& root, char *di_root_ptr = di_root.get(); vector di_roots; di_roots.push_back(&di_root_ptr); - abigail::elf_reader::status status = abigail::elf_reader::STATUS_OK; - corpus_group_sptr group; - if (!vmlinux.empty()) - { - ctxt = - dwarf_reader::create_read_context(vmlinux, di_roots ,env.get(), - /*read_all_types=*/false, - /*linux_kernel_mode=*/true); - dwarf_reader::set_do_log(*ctxt, verbose); - - t.start(); - load_generate_apply_suppressions(*ctxt, suppr_paths, - kabi_wl_paths, supprs); - t.stop(); - - if (verbose) - std::cerr << "loaded white list and generated suppr spec in: " - << t - << "\n"; - - group.reset(new corpus_group(env.get(), root)); - - set_read_context_corpus_group(*ctxt, group); - - if (verbose) - std::cerr << "reading kernel binary '" - << vmlinux << "' ...\n" << std::flush; - - // Read the vmlinux corpus and add it to the group. - t.start(); - read_and_add_corpus_to_group_from_elf(*ctxt, *group, status); - t.stop(); - - if (verbose) - std::cerr << vmlinux - << " reading DONE:" - << t << "\n"; - } - if (!group->is_empty()) - { - // Now add the corpora of the modules to the corpus group. - int total_nb_modules = modules.size(); - int cur_module_index = 1; - for (vector::const_iterator m = modules.begin(); - m != modules.end(); - ++m, ++cur_module_index) - { - if (verbose) - std::cerr << "reading module '" - << *m << "' (" - << cur_module_index - << "/" << total_nb_modules - << ") ... " << std::flush; - - reset_read_context(ctxt, *m, di_roots, env.get(), - /*read_all_types=*/false, - /*linux_kernel_mode=*/true); - - load_generate_apply_suppressions(*ctxt, suppr_paths, - kabi_wl_paths, supprs); - - set_read_context_corpus_group(*ctxt, group); - - t.start(); - read_and_add_corpus_to_group_from_elf(*ctxt, - *group, status); - t.stop(); - if (verbose) - std::cerr << "module '" - << *m - << "' reading DONE: " - << t << "\n"; - } - - result = group; - } + maybe_load_vmlinux_dwarf_corpus(origin, group, vmlinux, + modules, root, di_roots, + suppr_paths, kabi_wl_paths, + supprs, verbose, t, env); +#ifdef WITH_CTF + maybe_load_vmlinux_ctf_corpus(origin, group, vmlinux, + modules, root, verbose, t, env); +#endif } - return result; + return group; } }//end namespace tools_utils diff --git a/tests/data/test-read-common/test-PR26568-2.o b/tests/data/test-read-common/test-PR26568-2.o index b192d52132766d4f95136e23f93de43a64d8cc2e..a6706e8cf508ab89927b0fccecd0268d27111ed9 100644 GIT binary patch delta 1078 zcmZuwJ#Q015Z%3R+XNC4+rf~Wtw;eB%$Z17uE08YF6d0W;^l$g=#T(agS?dAmD%_cwibb#Nn{HP|`IVN_-* zmVAHz`rg0aiWKFRqCj{BJsEYAeL@S+bhImI1kPB7YSeJW7>OZA(1_29CDoC}EE<7w zsFR{tqeK}CFQZ3+WdmmP^q!2X1vG>5tjC(Z*T_=XXAjt} z?^ap8>oM!4->O-aex+~uZQE(qtWMjpnr_9exDDHF=DRzcpk1?DR#58%`Hoe!1A7!B zTg^LutL1wmVsmS~c(+)}uitjUGkmTeaY###a!yds<{P6;&yUgdUU~i-FRVO%A=#X( z@LEpFGSh{RlZR`{0vyU3p8->G4^#sN$`0Q;=IRA5qc9}2U+_o!rrts&jvez*)(zKAJj1W*_wN`>n^!NiJ zq)4A4oum?*^gRLR*Bo;Ows%V+A&PrWJgIxNM)g7wIC4r-B9=BEzkjHoMKyJa; z<6cR+1v3%uzsx3bA#*GVy9i8=??;51BwYL=&_9GWa+XVTN^Gm4YI*z;4a-&P3c6OF z-$aI)G!!Gly?9nJ)bP(1&@Cd)j_$-e(ZWJ_ae b7pX1XF9?zz{8U!i*07e^pjUb2Nd)`?0Gw2F diff --git a/tests/data/test-read-ctf/PR27700/test-PR27700.abi b/tests/data/test-read-ctf/PR27700/test-PR27700.abi index 62df1017..fe3a897d 100644 --- a/tests/data/test-read-ctf/PR27700/test-PR27700.abi +++ b/tests/data/test-read-ctf/PR27700/test-PR27700.abi @@ -11,9 +11,8 @@ - - + diff --git a/tests/data/test-read-ctf/test-PR26568-1.o.abi b/tests/data/test-read-ctf/test-PR26568-1.o.abi index d62474ec..832dc1bd 100644 --- a/tests/data/test-read-ctf/test-PR26568-1.o.abi +++ b/tests/data/test-read-ctf/test-PR26568-1.o.abi @@ -7,47 +7,29 @@ - - - - - - - - - - - - - + - + - + - + - - + + - - - - - - - + - + diff --git a/tests/data/test-read-ctf/test-PR26568-2.o.abi b/tests/data/test-read-ctf/test-PR26568-2.o.abi index a934d15b..70e0772f 100644 --- a/tests/data/test-read-ctf/test-PR26568-2.o.abi +++ b/tests/data/test-read-ctf/test-PR26568-2.o.abi @@ -3,34 +3,28 @@ - + - + - + - + - - + + - - - - - - - + - + diff --git a/tests/data/test-read-ctf/test-PR27700.abi b/tests/data/test-read-ctf/test-PR27700.abi new file mode 100644 index 00000000..fe3a897d --- /dev/null +++ b/tests/data/test-read-ctf/test-PR27700.abi @@ -0,0 +1,21 @@ + + + + + + + + + + + + + + + + + + + + + diff --git a/tests/data/test-read-ctf/test-ambiguous-struct-A.o.hash.abi b/tests/data/test-read-ctf/test-ambiguous-struct-A.o.hash.abi index 922a1daf..12050a5b 100644 --- a/tests/data/test-read-ctf/test-ambiguous-struct-A.o.hash.abi +++ b/tests/data/test-read-ctf/test-ambiguous-struct-A.o.hash.abi @@ -30,7 +30,6 @@ - - + diff --git a/tests/data/test-read-ctf/test-ambiguous-struct-B.c b/tests/data/test-read-ctf/test-ambiguous-struct-B.c index 95a93469..e592529b 100644 --- a/tests/data/test-read-ctf/test-ambiguous-struct-B.c +++ b/tests/data/test-read-ctf/test-ambiguous-struct-B.c @@ -2,4 +2,4 @@ struct A; struct B { struct A *a; }; struct A { struct B b; int foo; struct B b2; }; -static struct A a __attribute__((__used__)); +struct A a __attribute__((__used__)); diff --git a/tests/data/test-read-ctf/test-ambiguous-struct-B.o b/tests/data/test-read-ctf/test-ambiguous-struct-B.o index 06bd0f502a874ad4a10c4beb5788eaace44c2bf9..40a11fcbd4ae19b78a943812f69606cdc6c9fc15 100644 GIT binary patch delta 20 ccmX@Wb%1Nb8%9RK&2Jg^GchtvUdu8W08_UIfB*mh delta 20 ccmX@Wb%1Nb8%9RP&2Jg^GchtxUdu8W08;!1aR2}S diff --git a/tests/data/test-read-ctf/test-ambiguous-struct-B.o.hash.abi b/tests/data/test-read-ctf/test-ambiguous-struct-B.o.hash.abi index 28291eb5..83d21919 100644 --- a/tests/data/test-read-ctf/test-ambiguous-struct-B.o.hash.abi +++ b/tests/data/test-read-ctf/test-ambiguous-struct-B.o.hash.abi @@ -1,4 +1,7 @@ + + + @@ -18,6 +21,6 @@ - + diff --git a/tests/data/test-read-ctf/test-anonymous-fields.o.abi b/tests/data/test-read-ctf/test-anonymous-fields.o.abi index 0419c29c..2134a86d 100644 --- a/tests/data/test-read-ctf/test-anonymous-fields.o.abi +++ b/tests/data/test-read-ctf/test-anonymous-fields.o.abi @@ -3,14 +3,14 @@ - + - + - + - + @@ -18,16 +18,16 @@ - + - + - + - - + + diff --git a/tests/data/test-read-ctf/test-array-of-pointers.abi b/tests/data/test-read-ctf/test-array-of-pointers.abi index 920da28b..195361df 100644 --- a/tests/data/test-read-ctf/test-array-of-pointers.abi +++ b/tests/data/test-read-ctf/test-array-of-pointers.abi @@ -31,6 +31,6 @@ - + diff --git a/tests/data/test-read-ctf/test-callback.abi b/tests/data/test-read-ctf/test-callback.abi index 704c4971..7f9b6c5f 100644 --- a/tests/data/test-read-ctf/test-callback.abi +++ b/tests/data/test-read-ctf/test-callback.abi @@ -4,28 +4,16 @@ - - - - - - - - - - - + + + + - - - - + + + + - - - - - - + diff --git a/tests/data/test-read-ctf/test-callback2.abi b/tests/data/test-read-ctf/test-callback2.abi index bdd4ad33..ddc3e493 100644 --- a/tests/data/test-read-ctf/test-callback2.abi +++ b/tests/data/test-read-ctf/test-callback2.abi @@ -11,7 +11,7 @@ - + diff --git a/tests/data/test-read-ctf/test-conflicting-type-syms-a.o.hash.abi b/tests/data/test-read-ctf/test-conflicting-type-syms-a.o.hash.abi index 03dd56b3..8d5c3e36 100644 --- a/tests/data/test-read-ctf/test-conflicting-type-syms-a.o.hash.abi +++ b/tests/data/test-read-ctf/test-conflicting-type-syms-a.o.hash.abi @@ -8,7 +8,7 @@ - - + + diff --git a/tests/data/test-read-ctf/test-conflicting-type-syms-b.o.hash.abi b/tests/data/test-read-ctf/test-conflicting-type-syms-b.o.hash.abi index 35bcac7a..992b669d 100644 --- a/tests/data/test-read-ctf/test-conflicting-type-syms-b.o.hash.abi +++ b/tests/data/test-read-ctf/test-conflicting-type-syms-b.o.hash.abi @@ -7,7 +7,7 @@ - - + + diff --git a/tests/data/test-read-ctf/test-dynamic-array.o.abi b/tests/data/test-read-ctf/test-dynamic-array.o.abi index 02b22811..a9849d49 100644 --- a/tests/data/test-read-ctf/test-dynamic-array.o.abi +++ b/tests/data/test-read-ctf/test-dynamic-array.o.abi @@ -21,7 +21,7 @@ - + diff --git a/tests/data/test-read-ctf/test-enum-ctf.o.abi b/tests/data/test-read-ctf/test-enum-ctf.o.abi new file mode 100644 index 00000000..f36f3fad --- /dev/null +++ b/tests/data/test-read-ctf/test-enum-ctf.o.abi @@ -0,0 +1,24 @@ + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/tests/data/test-read-ctf/test-enum-many-ctf.o.hash.abi b/tests/data/test-read-ctf/test-enum-many-ctf.o.hash.abi new file mode 100644 index 00000000..67a958a7 --- /dev/null +++ b/tests/data/test-read-ctf/test-enum-many-ctf.o.hash.abi @@ -0,0 +1,69 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/tests/data/test-read-ctf/test-enum-many.o.hash.abi b/tests/data/test-read-ctf/test-enum-many.o.hash.abi index 26bc048c..116325f9 100644 --- a/tests/data/test-read-ctf/test-enum-many.o.hash.abi +++ b/tests/data/test-read-ctf/test-enum-many.o.hash.abi @@ -61,9 +61,7 @@ - - - - + + diff --git a/tests/data/test-read-ctf/test-enum-symbol-ctf.o.hash.abi b/tests/data/test-read-ctf/test-enum-symbol-ctf.o.hash.abi new file mode 100644 index 00000000..fea6eb8b --- /dev/null +++ b/tests/data/test-read-ctf/test-enum-symbol-ctf.o.hash.abi @@ -0,0 +1,16 @@ + + + + + + + + + + + + + + + + diff --git a/tests/data/test-read-ctf/test-enum-symbol.o.hash.abi b/tests/data/test-read-ctf/test-enum-symbol.o.hash.abi index d128b5b1..f4911bc4 100644 --- a/tests/data/test-read-ctf/test-enum-symbol.o.hash.abi +++ b/tests/data/test-read-ctf/test-enum-symbol.o.hash.abi @@ -10,7 +10,6 @@ - - + diff --git a/tests/data/test-read-ctf/test-enum.o.abi b/tests/data/test-read-ctf/test-enum.o.abi index 88e6ad61..bd3a55be 100644 --- a/tests/data/test-read-ctf/test-enum.o.abi +++ b/tests/data/test-read-ctf/test-enum.o.abi @@ -16,9 +16,7 @@ - - - - + + diff --git a/tests/data/test-read-ctf/test-forward-type-decl.abi b/tests/data/test-read-ctf/test-forward-type-decl.abi index 21bb45c8..026e7d32 100644 --- a/tests/data/test-read-ctf/test-forward-type-decl.abi +++ b/tests/data/test-read-ctf/test-forward-type-decl.abi @@ -24,6 +24,6 @@ - + diff --git a/tests/data/test-read-ctf/test-functions-declaration.abi b/tests/data/test-read-ctf/test-functions-declaration.abi index 9ef05e44..7eb57d26 100644 --- a/tests/data/test-read-ctf/test-functions-declaration.abi +++ b/tests/data/test-read-ctf/test-functions-declaration.abi @@ -7,11 +7,11 @@ - + - + diff --git a/tests/data/test-read-ctf/test-list-struct.abi b/tests/data/test-read-ctf/test-list-struct.abi index 196a79a1..deb6a4a1 100644 --- a/tests/data/test-read-ctf/test-list-struct.abi +++ b/tests/data/test-read-ctf/test-list-struct.abi @@ -14,7 +14,7 @@ - - + + diff --git a/tests/data/test-read-ctf/test0 b/tests/data/test-read-ctf/test0 index aca7fbacdfff7ad2caf0142390710373a8da72a1..90aa8a39a2e1326a5cb5faaa9e854fc344caaa52 100755 GIT binary patch delta 1886 zcmah~L2MgE6rEkWZra+(ZUVNWHnG!6kSdbB4pp+Tz+NFxdrBKokUdjHSN zd;ibh8SlFBwsHP~(HKr_=EVc^gezjg69eK-2CcCEMV+OqRCA^GGli z>$IIInx!!1vl-J3)7k7`$O(J&cW80VdGA07&*>>?n$G1?dj~K!MU=jF68%x}p%rjS z05{6)U9bt^c>!xMeHzgay_Cwy+>9NtOp13EYtzZxal@iZxop2JP89yD`B7>5=r>fb zh1bE}QKfpQXEf3Ou)y|eu3azM=tbv6uyK*>U=M?}M4WuEG4X9jZ%4t}B0(RIW{o)g ztb7~FZu}sEUGyzq8AHp?loq0|C=(`#L0t= ziZq}62-7#gTIleuVkf!(1I*r0l@pXYCiitZD&s=c)13s1jE+0)4%m&AW2eoP6DW8M z#YTYVF^&Uo0S^IJ033+;4u}FRzy@vsG2nMVHgpf@15DJ|i#Q9wBJy?hfI+OoBJ$AE zfW>o!M+8J>1BIT%j+a|`ugcxKkG z@NnLrnX6Z8!FmlTwV7$AUiW7!^_kf!nG;V(UzuCX%-8(+YOTJ!;Z5u_=v3a?pA31b z!(@og=Eq_#Pt`|*rb2yd1#4go3VCqy`K6Uuq`pW}M4Nf*sJtrq(reWctCsk?m6WNl z2)<~Em8USe-){R_tv?$LRv%JYs5#2Z+HI4V$jjCF>%7}K=gV%S$-WIh=W`c_#dBDc zT3uj^3v6)#m(Jx=D=$IO-qj|)R9F$h;WtdTr=zZ>a7=LG}|x;{%$4|A?CKm#GDRmFz;%xK2~> zKT)&rXuoVnnyx1Q4W#`-Eog3$T`U^6X$t;NYQq0TE%+MHB?NX;F)iHf3#0R0<+&`XH#N z(X+7HWF#?*pbraz7G=z0T7=Llf`~Sev~ZQw#_8Pq&Mlh@@7#Cp?|;rc_r81Ih07v$ zQRF(B-Cp?H6r>;w3O2y{34c@fpbUuZSF_NeK15HRd0r~=v_CP~SN%(4YQLO`PfHC2 z))b`DgHVvzD%CR0Z0;czm#Y9qZ%2;=KSb0iFR-GCVZnQo?w zM4LdVfC|Qks25O=(WH)z)d!RzTs73i{^UMUhf~RT#D)ER|7+eGO#^~_0WX(y5ph|=Z!6qvKGJ# zX3?Kgw`)90e2dGB;s#tDZMemYF`NSzSNTYD%N*l%`)3(An{=ha|p{q4Cp{Y@X2Z zKvtWF|SZbxhr*YaTI5E z4Ajm9!%#>4BXm<;;w?@7v0jCP#lT8}D(v$MO4FtoK82j*aIk+F=h3YOPIRulgV-Na3b*bo>A*N-N5o z=4qx`Wq%CzhhekX{xHQauZYice4gVy;<}ZW8H~biI6-gvjM#)lPEh0o30B$y%GL?P znBC=zGdM@~Jc>SFT*3_5>zE~b2j|E>K;ir1F=oiV!mPhJVwYvJ=@Y110q02b4u$f? o2h5QDh*`3qagOXa6q%?P!TXsgs!WGy#gmz#s - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + diff --git a/tests/data/test-read-ctf/test0.c b/tests/data/test-read-ctf/test0.c index eb702312..fdefd57c 100644 --- a/tests/data/test-read-ctf/test0.c +++ b/tests/data/test-read-ctf/test0.c @@ -1,3 +1,9 @@ +/* + * ELF EXEC files must use -Wl,--ctf-variables -Bdynimic options + * to export an ABI and store CTF symbols information. + * + * ctf-gcc -gctf -Wl,--ctf-variables -Bdynamic test0.c -o test0 + */ #include diff --git a/tests/data/test-read-ctf/test0.hash.abi b/tests/data/test-read-ctf/test0.hash.abi index b58520ab..eff32228 100644 --- a/tests/data/test-read-ctf/test0.hash.abi +++ b/tests/data/test-read-ctf/test0.hash.abi @@ -39,16 +39,26 @@ - - - - + + + + + + + + + + + + + + diff --git a/tests/data/test-read-ctf/test1.so.abi b/tests/data/test-read-ctf/test1.so.abi index 5b3caf63..416bd39d 100644 --- a/tests/data/test-read-ctf/test1.so.abi +++ b/tests/data/test-read-ctf/test1.so.abi @@ -16,8 +16,13 @@ - - - + + + + + + + + diff --git a/tests/data/test-read-ctf/test1.so.hash.abi b/tests/data/test-read-ctf/test1.so.hash.abi index 8019cb54..b3620e75 100644 --- a/tests/data/test-read-ctf/test1.so.hash.abi +++ b/tests/data/test-read-ctf/test1.so.hash.abi @@ -16,8 +16,13 @@ - + + + + + + diff --git a/tests/data/test-read-ctf/test2.so.abi b/tests/data/test-read-ctf/test2.so.abi index eb6aff3e..67f802f9 100644 --- a/tests/data/test-read-ctf/test2.so.abi +++ b/tests/data/test-read-ctf/test2.so.abi @@ -20,6 +20,14 @@ + + + + + + + + diff --git a/tests/data/test-read-ctf/test2.so.hash.abi b/tests/data/test-read-ctf/test2.so.hash.abi index 6e57333b..6bbf347e 100644 --- a/tests/data/test-read-ctf/test2.so.hash.abi +++ b/tests/data/test-read-ctf/test2.so.hash.abi @@ -20,6 +20,14 @@ + + + + + + + + diff --git a/tests/data/test-read-ctf/test3.so.abi b/tests/data/test-read-ctf/test3.so.abi index 9d55fec5..3d2f6326 100644 --- a/tests/data/test-read-ctf/test3.so.abi +++ b/tests/data/test-read-ctf/test3.so.abi @@ -8,5 +8,9 @@ + + + + diff --git a/tests/data/test-read-ctf/test3.so.hash.abi b/tests/data/test-read-ctf/test3.so.hash.abi index 9d55fec5..1c69e2e1 100644 --- a/tests/data/test-read-ctf/test3.so.hash.abi +++ b/tests/data/test-read-ctf/test3.so.hash.abi @@ -8,5 +8,9 @@ + + + + diff --git a/tests/data/test-read-ctf/test4.so.abi b/tests/data/test-read-ctf/test4.so.abi index b8e0ead3..dc18e191 100644 --- a/tests/data/test-read-ctf/test4.so.abi +++ b/tests/data/test-read-ctf/test4.so.abi @@ -10,5 +10,11 @@ + + + + + + diff --git a/tests/data/test-read-ctf/test4.so.hash.abi b/tests/data/test-read-ctf/test4.so.hash.abi index dbe34d9c..dc428592 100644 --- a/tests/data/test-read-ctf/test4.so.hash.abi +++ b/tests/data/test-read-ctf/test4.so.hash.abi @@ -10,5 +10,11 @@ + + + + + + diff --git a/tests/data/test-read-ctf/test5.o.abi b/tests/data/test-read-ctf/test5.o.abi index eb30cf6a..f7bcdeb1 100644 --- a/tests/data/test-read-ctf/test5.o.abi +++ b/tests/data/test-read-ctf/test5.o.abi @@ -17,34 +17,32 @@ - - - - - - - - - - + + + + + + + + - + - + - + - + - + - + - + - + - + diff --git a/tests/data/test-read-ctf/test7.o.abi b/tests/data/test-read-ctf/test7.o.abi index b744fac5..a13af174 100644 --- a/tests/data/test-read-ctf/test7.o.abi +++ b/tests/data/test-read-ctf/test7.o.abi @@ -3,40 +3,31 @@ - - + - + - + - + - - - - - - - - - - - - - - - - - - + + + + + + + + + + - - + + - + diff --git a/tests/data/test-read-ctf/test8.o.abi b/tests/data/test-read-ctf/test8.o.abi index 68b3af06..b6996c29 100644 --- a/tests/data/test-read-ctf/test8.o.abi +++ b/tests/data/test-read-ctf/test8.o.abi @@ -4,7 +4,7 @@ - + diff --git a/tests/data/test-read-ctf/test9.o.abi b/tests/data/test-read-ctf/test9.o.abi index 746dd77b..08704d79 100644 --- a/tests/data/test-read-ctf/test9.o.abi +++ b/tests/data/test-read-ctf/test9.o.abi @@ -49,7 +49,7 @@ - + diff --git a/tests/test-read-ctf.cc b/tests/test-read-ctf.cc index fdf49e90..215ed8d6 100644 --- a/tests/test-read-ctf.cc +++ b/tests/test-read-ctf.cc @@ -269,20 +269,28 @@ static InOutSpec in_out_specs[] = "output/test-read-ctf/test-list-struct.abi", }, { - "data/test-read-ctf/test-callback2.o", + "data/test-read-common/test-PR26568-1.o", "", "", SEQUENCE_TYPE_ID_STYLE, - "data/test-read-ctf/test-callback2.abi", - "output/test-read-ctf/test-callback2.abi", + "data/test-read-ctf/test-PR26568-1.o.abi", + "output/test-read-ctf/test-PR26568-1.o.abi", }, { - "data/test-read-ctf/test-forward-undefine-type-decl.o", + "data/test-read-common/test-PR26568-2.o", "", "", SEQUENCE_TYPE_ID_STYLE, - "data/test-read-ctf/test-forward-undefine-type-decl.abi", - "output/test-read-ctf/test-forward-undefine-type-decl.abi", + "data/test-read-ctf/test-PR26568-2.o.abi", + "output/test-read-ctf/test-PR26568-2.o.abi", + }, + { + "data/test-read-ctf/test-callback2.o", + "", + "", + SEQUENCE_TYPE_ID_STYLE, + "data/test-read-ctf/test-callback2.abi", + "output/test-read-ctf/test-callback2.abi", }, // This should be the last entry. {NULL, NULL, NULL, SEQUENCE_TYPE_ID_STYLE, NULL, NULL} diff --git a/tools/abidw.cc b/tools/abidw.cc index f7a8937d..32d055f5 100644 --- a/tools/abidw.cc +++ b/tools/abidw.cc @@ -811,13 +811,18 @@ load_kernel_corpus_group_and_write_abixml(char* argv[], global_timer.start(); t.start(); +corpus::origin origin = +#ifdef WITH_CTF + opts.use_ctf ? corpus::CTF_ORIGIN : +#endif + corpus::DWARF_ORIGIN; corpus_group_sptr group = build_corpus_group_from_kernel_dist_under(opts.in_file_path, /*debug_info_root=*/"", opts.vmlinux, opts.suppression_paths, opts.kabi_whitelist_paths, - supprs, opts.do_log, env); + supprs, opts.do_log, env, origin); t.stop(); if (opts.do_log)