From patchwork Tue May 12 10:14:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Szabolcs Nagy X-Patchwork-Id: 38961 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id E26A5395443B; Tue, 12 May 2020 10:15:12 +0000 (GMT) X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-eopbgr130044.outbound.protection.outlook.com [40.107.13.44]) by sourceware.org (Postfix) with ESMTPS id B18E9383F846 for ; Tue, 12 May 2020 10:15:07 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org B18E9383F846 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=Szabolcs.Nagy@arm.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TXwxaKtNa5cfJD/yM5jtlVVPc+/uEMumxjRuCvBGLvY=; b=pDHs5f1C8O5jy31kbc0BNQEeBy8Rf9lD02pc199nU6DTLZmf67Fv2COerXVXCh9XyHDlgdorEc2RdsJnArE7kAhO/71x3L5CBBLmZszlfP5UDmFIR9zUrLpNwsXfEk/9j07Gmzml1nYxofbDgTKgCk/CopPfTUx2yNAJYQyP1v4= Received: from DB6PR0202CA0005.eurprd02.prod.outlook.com (2603:10a6:4:29::15) by AM0PR08MB3091.eurprd08.prod.outlook.com (2603:10a6:208:60::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.28; Tue, 12 May 2020 10:15:05 +0000 Received: from DB5EUR03FT019.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:29:cafe::51) by DB6PR0202CA0005.outlook.office365.com (2603:10a6:4:29::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.28 via Frontend Transport; Tue, 12 May 2020 10:15:05 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; sourceware.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; sourceware.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT019.mail.protection.outlook.com (10.152.20.163) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.27 via Frontend Transport; Tue, 12 May 2020 10:15:05 +0000 Received: ("Tessian outbound fb9de21a7e90:v54"); Tue, 12 May 2020 10:15:05 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 023b3ac4503c223d X-CR-MTA-TID: 64aa7808 Received: from 136d48774594.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 24E91E7E-A3D8-4A51-9C57-529D49D48D62.1; Tue, 12 May 2020 10:14:59 +0000 Received: from EUR04-DB3-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 136d48774594.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 12 May 2020 10:14:59 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=azSiPULyEL/mStlBjUeaJOvSviG76NlTtkbNbI3Lc3fFkF+It8s/uteOUfA0h6J3bUxPAt6+tFFS9tb173Lvg13RD25iwBoyljMZfylGFZUNcTTt4wsYLTiwmRelxyzofuCb48Dw5JBbKp77htjBidA5t8lrt+xG4qAT0rldlcSwP0yv2E8pJw8WCH9wSmAEb3sgOj7gfZhQAagg+XqBMOAgS4ymjmTTKlBZFAm0Rqm8OfIB+q23lF1EDoxC0naXxKy4GgYX/12gUJcR3CHHe8b1YlC1tKnWNsJFMBgYxLE3aZkCX9W8F97v0dFhCipmB5kPCvgEOQfJIH7fV4+Oqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TXwxaKtNa5cfJD/yM5jtlVVPc+/uEMumxjRuCvBGLvY=; b=fhqfRGa8mvDH4O8JM8YnewIJ4hhb4L7OgKc4A9Uyn2e11He9krpkUjPfjGovDhP5IbGrpseELMvR1GXMtuyNfX3RSa1loyniTjlBvSQLMZDN6oM9Pev4+cir6UfC7aZFoxhSaf6Hvk85aokfy52ykKSwJyXrBIJBYtWJrAEG3zrQJfZ0ajngy4XMuEP+WE+9d7NdIlczvxKAadYEHshLs+GFi5N0G7WiXgO5RmOxORV0WWiDJkGZLt9JNBE+agWX3gzkcuvGofC/JRZ7Z2w5lI369paPdH75le+lGSgWtCaLgAEtGYwsAjv2rhMc0L9EBsQS5kcr6Ehq+TNaZq8Cag== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TXwxaKtNa5cfJD/yM5jtlVVPc+/uEMumxjRuCvBGLvY=; b=pDHs5f1C8O5jy31kbc0BNQEeBy8Rf9lD02pc199nU6DTLZmf67Fv2COerXVXCh9XyHDlgdorEc2RdsJnArE7kAhO/71x3L5CBBLmZszlfP5UDmFIR9zUrLpNwsXfEk/9j07Gmzml1nYxofbDgTKgCk/CopPfTUx2yNAJYQyP1v4= Authentication-Results-Original: sourceware.org; dkim=none (message not signed) header.d=none;sourceware.org; dmarc=none action=none header.from=arm.com; Received: from AM6PR08MB3047.eurprd08.prod.outlook.com (2603:10a6:209:4c::23) by AM6PR08MB2982.eurprd08.prod.outlook.com (2603:10a6:209:43::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.28; Tue, 12 May 2020 10:14:58 +0000 Received: from AM6PR08MB3047.eurprd08.prod.outlook.com ([fe80::49fd:6ded:4da7:8862]) by AM6PR08MB3047.eurprd08.prod.outlook.com ([fe80::49fd:6ded:4da7:8862%7]) with mapi id 15.20.2979.033; Tue, 12 May 2020 10:14:58 +0000 From: Szabolcs Nagy To: libc-alpha@sourceware.org Subject: [PATCH v2 05/13] aarch64: Add BTI landing pads to assembly code Date: Tue, 12 May 2020 11:14:41 +0100 Message-Id: <14aaaaef4986ecda5edc2659d87934dcd8cc7933.1589277641.git.szabolcs.nagy@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: X-ClientProxiedBy: CWLP123CA0056.GBRP123.PROD.OUTLOOK.COM (2603:10a6:401:59::20) To AM6PR08MB3047.eurprd08.prod.outlook.com (2603:10a6:209:4c::23) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (217.140.106.55) by CWLP123CA0056.GBRP123.PROD.OUTLOOK.COM (2603:10a6:401:59::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.27 via Frontend Transport; Tue, 12 May 2020 10:14:57 +0000 X-Mailer: git-send-email 2.17.1 X-Originating-IP: [217.140.106.55] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 89c0bb54-ed19-40fb-abe0-08d7f65d5723 X-MS-TrafficTypeDiagnostic: AM6PR08MB2982:|AM6PR08MB2982:|AM0PR08MB3091: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Oob-TLC-OOBClassifiers: OLM:9508;OLM:9508; X-Forefront-PRVS: 0401647B7F X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: lPGTHtSYv+ibwM5mveSGGKQkygswgknGeYt9K5xNe0pB+7mp3chAleII4EPtGHt59QT1ZQZZw6wne25J8iEYVBhbbqOxp67Ebu+ZDZDS3R7k4wj0yDNYFo0zE+vNiFqDF+ZsXOpemZpCt2J32IxcyGu524YRLhsZJWgA9+CzeY/jd7Ioc0sZeJEH6EJy/DaCiuQU0vYUPIUg7nHApdTVLBcR/FhcdRDoDZdbvVEzitRHt7psBcd/MZDVsq7iTI8LInpjIJbsTC+89uAXJFXmwrBFmYkK9bxLhgex+G/c41zY7eVSgoh2MJEqEWMKG1iXJO6Bm/dSEHg9O5OxceVG7eyzPUZMS7RmdPkAyK8NZj03aCg+dSTnhNknrkPEiHJFeK8QDuHb1esGk6OpBTNzgvTHrVyukhcL3hPSC3dZwYfMuc+++PY8EWxWl15xp69hl2q7/59BfLg3TVp/rWU3wZMiNwny2FYTf+4CKUwSElWqNKOmRmgz67hq/fXAFp2P7q5izorx8ZzLXpgONZMPRJ8rRGhPVAzrqgkZhcwp2fT7qfW5BsTu7jc7O5C948dwu+vVj1/WnJClHa/55Qt/rnaRetvjubXBjF8EfP4111xmNR3utjMnF5S/SOIwQ69ubXCcwGTpbqCu1Mi3iiW7lPCL11NkarxlhL1ZSTx6vCAtg3h3SvVKCPhZiFAZ6tWf X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM6PR08MB3047.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(136003)(376002)(396003)(346002)(366004)(33430700001)(6916009)(16526019)(66556008)(66476007)(30864003)(36756003)(44832011)(5660300002)(52116002)(86362001)(956004)(8676002)(66946007)(69590400007)(2906002)(6486002)(186003)(316002)(2616005)(4326008)(33440700001)(6666004)(478600001)(6506007)(6512007)(8936002)(26005)(136400200001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: PQJsibyFafOuiBWfHSdfHQpl0aqFhKqPEH1+D0NN3P3me+R+7R10HvEWn5E4LhlkmUy66Tya3QMFXSwyN/SE4qHZNTMAbPzuGPkj/2NpPlpDYN6ewJBm8YW1t8FpPoiNO7QuzH5DOZkR98/i3Orfigh/FawIO5YOIbwnSBZLsZcQ3uLkzMEdK4TcuXSnmeEHyvsl004BAJjnl8hFGpg+CBNJkXVvpdz1opkttk3ADL+QFUQkfKBP0cTlJhVTGF1G4hRgPoT9jjl0ma+edQWyijR5jF/X8d84f6VO8OAx41xjkAqk3Cc5ca5wVQUE39yEJQazWTItc4E2G9j165g+aBHe6HSjgaT6myI+ET0KShB+wVR4yMWVbne0oavYT/5q9v6VWYuyZv1DpcdPktt9BU6CVZsMaXPqhAMtAyjcwlwb6eUCHDUoy7FqUFU8wmtWkN/39NfoMhPBeJ/10qQYAK1HBE+4B1llv4zkh1o0cvsTNSKdyl/csCDq2hQRX2lR X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB2982 Original-Authentication-Results: sourceware.org; dkim=none (message not signed) header.d=none; sourceware.org; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT019.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFTY:; SFS:(4636009)(396003)(39860400002)(136003)(346002)(376002)(46966005)(33430700001)(70206006)(33440700001)(336012)(6486002)(6512007)(2906002)(8676002)(316002)(44832011)(186003)(16526019)(70586007)(81166007)(6506007)(356005)(82740400003)(36756003)(82310400002)(69590400007)(2616005)(5660300002)(6666004)(47076004)(26005)(30864003)(86362001)(956004)(8936002)(6916009)(4326008)(478600001)(136400200001); DIR:OUT; SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: 61c0dea0-d8a4-4984-cc73-08d7f65d52cf X-Forefront-PRVS: 0401647B7F X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: MIDdVaRVNUePQIDfHQrIn4flAPOsw1a5alb7fnzLxqNAKlLYYTligzaC3821shsYocdIYGGn9nd1ZNRAJM/Ik8OeKTdXnXHwPg0CUmge8A+6Ylo/1cvhNdeI3R9WAugn9XhBSDatmZWbfiOk5uGo17I8trY7C6HCebrYThZVllzo8eQHfGQiFS5UG3MkR0XUglTktIIwZ1KBiEH9shJhZPOpo+2NzmUlQfzcoNQndt9p0LAEEv69j6TBzItB2Ta3CKLKxNolw+piQHgRBvrtJJreYeq0VKawfknPjN1wogm8oTWkyQMxEYLSe2l6HRpL+5WGgDrER4nvfgdJuhwsk1GzjPtgVqa+xfbk+kYfctUGYFbA5p8VRgaq5l/XdQuTXcqxMmKai5OyplO2OFT0pOXnMIyn5ylCAo53Tg24LOE3IZF6bVjBCSUy79NaQYpnfQcjKvGJf53nfy/gz2BFF9XNnnwRHiiUBXwnnIVUuJ+EpbOWEVQWUh6+EtbDZvIZk2TH1Y8e4jakKl5lPiAYukI+5S+Fqs6tAUXtlh1toEGY6HiqXUK/NxF3pfsZHbd1/QJvFwY/D8GYdS0Xx610tDYjmqsi2GhCikPiTY1kNKSTzFLtClO/wg1pPi090WXwAqFMow+rMb/ViONtfgPvIg/2WsFjL1Y/Xx1DD7oW3enb0mjZ0LbulhdawUtRunTe4/aAysuRQwrQbno1aSnHRF5OFFGLD0nE3/ENaaYJF6M= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2020 10:15:05.0461 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 89c0bb54-ed19-40fb-abe0-08d7f65d5723 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3091 X-Spam-Status: No, score=-19.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_SHORT, KAM_STOCKGEN, MSGID_FROM_MTA_HEADER, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Sudakshina Das Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" From: Sudakshina Das To enable building glibc with branch protection, assembly code needs BTI landing pads and ELF object file markings in the form of a GNU property note. The landing pads are unconditionally added to all functions that may be indirectly called. When the code segment is not mapped with PROT_BTI these instructions are nops. They are kept in the code when BTI is not supported so that the layout of performance critical code is unchanged across configurations. The GNU property notes are only added when there is support for BTI in the toolchain, because old binutils does not handle the notes right. (Does not know how to merge them nor to put them in PT_GNU_PROPERTY segment instead of PT_NOTE, and some versions of binutils emit warnings about the unknown GNU property. In such cases the produced libc binaries would not have the right ELF marking so BTI would not be enabled.) Note: functions using ENTRY or ENTRY_ALIGN now start with an additional BTI c, so alignment of the following code changes, but ENTRY_ALIGN_AND_PAD was fixed so there is no change to the existing code layout. Some string functions may need to be tuned for optimal performance after this commit. Co-authored-by: Szabolcs Nagy TODO: how to add property notes to asm: use END_FILE macro or other macro (with descriptive name) or add them in sysdep.h --- sysdeps/aarch64/__longjmp.S | 1 + sysdeps/aarch64/crti.S | 4 +++ sysdeps/aarch64/crtn.S | 4 +++ sysdeps/aarch64/dl-tlsdesc.S | 5 +++ sysdeps/aarch64/dl-trampoline.S | 4 +++ sysdeps/aarch64/memchr.S | 1 + sysdeps/aarch64/memcmp.S | 1 + sysdeps/aarch64/memcpy.S | 1 + sysdeps/aarch64/memrchr.S | 1 + sysdeps/aarch64/memset.S | 1 + sysdeps/aarch64/multiarch/memchr_nosimd.S | 1 + sysdeps/aarch64/multiarch/memcpy_falkor.S | 1 + sysdeps/aarch64/multiarch/memcpy_thunderx.S | 1 + sysdeps/aarch64/multiarch/memcpy_thunderx2.S | 1 + sysdeps/aarch64/multiarch/memmove_falkor.S | 1 + sysdeps/aarch64/multiarch/memset_base64.S | 1 + sysdeps/aarch64/multiarch/memset_kunpeng.S | 1 + sysdeps/aarch64/multiarch/strlen_asimd.S | 1 + sysdeps/aarch64/rawmemchr.S | 1 + sysdeps/aarch64/setjmp.S | 1 + sysdeps/aarch64/start.S | 2 ++ sysdeps/aarch64/strchr.S | 1 + sysdeps/aarch64/strchrnul.S | 1 + sysdeps/aarch64/strcmp.S | 1 + sysdeps/aarch64/strcpy.S | 1 + sysdeps/aarch64/strlen.S | 1 + sysdeps/aarch64/strncmp.S | 1 + sysdeps/aarch64/strnlen.S | 1 + sysdeps/aarch64/strrchr.S | 1 + sysdeps/aarch64/sysdep.h | 32 ++++++++++++++++++- sysdeps/unix/sysv/linux/aarch64/__read_tp.S | 1 + sysdeps/unix/sysv/linux/aarch64/clone.S | 1 + sysdeps/unix/sysv/linux/aarch64/getcontext.S | 1 + sysdeps/unix/sysv/linux/aarch64/ioctl.S | 1 + .../unix/sysv/linux/aarch64/libc-__read_tp.S | 1 + sysdeps/unix/sysv/linux/aarch64/setcontext.S | 1 + sysdeps/unix/sysv/linux/aarch64/syscall.S | 1 + sysdeps/unix/sysv/linux/aarch64/vfork.S | 1 + 38 files changed, 82 insertions(+), 1 deletion(-) diff --git a/sysdeps/aarch64/__longjmp.S b/sysdeps/aarch64/__longjmp.S index f9060776b4..362171cdb8 100644 --- a/sysdeps/aarch64/__longjmp.S +++ b/sysdeps/aarch64/__longjmp.S @@ -116,3 +116,4 @@ ENTRY (__longjmp) /* Use br instead of ret because ret is guaranteed to mispredict */ br x30 END (__longjmp) +END_FILE diff --git a/sysdeps/aarch64/crti.S b/sysdeps/aarch64/crti.S index 1728eac37a..89a9e25f5b 100644 --- a/sysdeps/aarch64/crti.S +++ b/sysdeps/aarch64/crti.S @@ -75,6 +75,7 @@ call_weak_fn: .hidden _init .type _init, %function _init: + BTI_C stp x29, x30, [sp, -16]! mov x29, sp #if PREINIT_FUNCTION_WEAK @@ -89,5 +90,8 @@ _init: .hidden _fini .type _fini, %function _fini: + BTI_C stp x29, x30, [sp, -16]! mov x29, sp + +END_FILE diff --git a/sysdeps/aarch64/crtn.S b/sysdeps/aarch64/crtn.S index c3e97cc449..94a6f970ef 100644 --- a/sysdeps/aarch64/crtn.S +++ b/sysdeps/aarch64/crtn.S @@ -37,6 +37,8 @@ /* crtn.S puts function epilogues in the .init and .fini sections corresponding to the prologues in crti.S. */ +#include + .section .init,"ax",%progbits ldp x29, x30, [sp], 16 RET @@ -44,3 +46,5 @@ .section .fini,"ax",%progbits ldp x29, x30, [sp], 16 RET + +END_FILE diff --git a/sysdeps/aarch64/dl-tlsdesc.S b/sysdeps/aarch64/dl-tlsdesc.S index 557ad1d505..d55e0443aa 100644 --- a/sysdeps/aarch64/dl-tlsdesc.S +++ b/sysdeps/aarch64/dl-tlsdesc.S @@ -74,6 +74,7 @@ cfi_startproc .align 2 _dl_tlsdesc_return: + BTI_C DELOUSE (0) ldr PTR_REG (0), [x0, #PTR_SIZE] RET @@ -95,6 +96,7 @@ _dl_tlsdesc_return: cfi_startproc .align 2 _dl_tlsdesc_undefweak: + BTI_C str x1, [sp, #-16]! cfi_adjust_cfa_offset (16) DELOUSE (0) @@ -142,6 +144,7 @@ _dl_tlsdesc_undefweak: cfi_startproc .align 2 _dl_tlsdesc_dynamic: + BTI_C DELOUSE (0) /* Save just enough registers to support fast path, if we fall @@ -235,3 +238,5 @@ _dl_tlsdesc_dynamic: .size _dl_tlsdesc_dynamic, .-_dl_tlsdesc_dynamic # undef NSAVEXREGPAIRS #endif + +END_FILE diff --git a/sysdeps/aarch64/dl-trampoline.S b/sysdeps/aarch64/dl-trampoline.S index 94e965c096..fba5689d09 100644 --- a/sysdeps/aarch64/dl-trampoline.S +++ b/sysdeps/aarch64/dl-trampoline.S @@ -35,6 +35,7 @@ cfi_startproc .align 2 _dl_runtime_resolve: + BTI_C /* AArch64 we get called with: ip0 &PLTGOT[2] ip1 temp(dl resolver entry point) @@ -126,6 +127,7 @@ _dl_runtime_resolve: cfi_startproc .align 2 _dl_runtime_profile: + BTI_C /* AArch64 we get called with: ip0 &PLTGOT[2] ip1 temp(dl resolver entry point) @@ -298,3 +300,5 @@ _dl_runtime_profile: .size _dl_runtime_profile, .-_dl_runtime_profile #endif .previous + +END_FILE diff --git a/sysdeps/aarch64/memchr.S b/sysdeps/aarch64/memchr.S index 85c65cbfca..c67a31223f 100644 --- a/sysdeps/aarch64/memchr.S +++ b/sysdeps/aarch64/memchr.S @@ -159,3 +159,4 @@ L(zero_length): END (MEMCHR) weak_alias (MEMCHR, memchr) libc_hidden_builtin_def (memchr) +END_FILE diff --git a/sysdeps/aarch64/memcmp.S b/sysdeps/aarch64/memcmp.S index 827f54f99e..c6e07f9287 100644 --- a/sysdeps/aarch64/memcmp.S +++ b/sysdeps/aarch64/memcmp.S @@ -178,3 +178,4 @@ END (memcmp) #undef bcmp weak_alias (memcmp, bcmp) libc_hidden_builtin_def (memcmp) +END_FILE diff --git a/sysdeps/aarch64/memcpy.S b/sysdeps/aarch64/memcpy.S index e0b4c4502f..543d9417f3 100644 --- a/sysdeps/aarch64/memcpy.S +++ b/sysdeps/aarch64/memcpy.S @@ -282,3 +282,4 @@ L(copy64_from_start): END (MEMMOVE) libc_hidden_builtin_def (MEMMOVE) +END_FILE diff --git a/sysdeps/aarch64/memrchr.S b/sysdeps/aarch64/memrchr.S index ace5a94e8f..f35a68d14d 100644 --- a/sysdeps/aarch64/memrchr.S +++ b/sysdeps/aarch64/memrchr.S @@ -163,3 +163,4 @@ L(zero_length): END (__memrchr) weak_alias (__memrchr, memrchr) libc_hidden_builtin_def (memrchr) +END_FILE diff --git a/sysdeps/aarch64/memset.S b/sysdeps/aarch64/memset.S index ac577f1660..7cdae20563 100644 --- a/sysdeps/aarch64/memset.S +++ b/sysdeps/aarch64/memset.S @@ -189,3 +189,4 @@ L(zva_other): END (MEMSET) libc_hidden_builtin_def (MEMSET) +END_FILE diff --git a/sysdeps/aarch64/multiarch/memchr_nosimd.S b/sysdeps/aarch64/multiarch/memchr_nosimd.S index 41ce10eb32..6d7d38d5bb 100644 --- a/sysdeps/aarch64/multiarch/memchr_nosimd.S +++ b/sysdeps/aarch64/multiarch/memchr_nosimd.S @@ -221,3 +221,4 @@ L(none_chr): END (MEMCHR) libc_hidden_builtin_def (MEMCHR) +END_FILE diff --git a/sysdeps/aarch64/multiarch/memcpy_falkor.S b/sysdeps/aarch64/multiarch/memcpy_falkor.S index 35a1fae1b9..999aa48b16 100644 --- a/sysdeps/aarch64/multiarch/memcpy_falkor.S +++ b/sysdeps/aarch64/multiarch/memcpy_falkor.S @@ -188,4 +188,5 @@ L(last64): END (__memcpy_falkor) libc_hidden_builtin_def (__memcpy_falkor) +END_FILE #endif diff --git a/sysdeps/aarch64/multiarch/memcpy_thunderx.S b/sysdeps/aarch64/multiarch/memcpy_thunderx.S index e9407571b5..e6e36a6633 100644 --- a/sysdeps/aarch64/multiarch/memcpy_thunderx.S +++ b/sysdeps/aarch64/multiarch/memcpy_thunderx.S @@ -318,5 +318,6 @@ L(move_long): END (MEMCPY) libc_hidden_builtin_def (MEMCPY) +END_FILE #endif diff --git a/sysdeps/aarch64/multiarch/memcpy_thunderx2.S b/sysdeps/aarch64/multiarch/memcpy_thunderx2.S index 68e99455c8..fde4c7198c 100644 --- a/sysdeps/aarch64/multiarch/memcpy_thunderx2.S +++ b/sysdeps/aarch64/multiarch/memcpy_thunderx2.S @@ -474,4 +474,5 @@ L(ext_table): .word L(ext_size_15) -. libc_hidden_builtin_def (MEMCPY) +END_FILE #endif diff --git a/sysdeps/aarch64/multiarch/memmove_falkor.S b/sysdeps/aarch64/multiarch/memmove_falkor.S index 35fc1fdd41..d8cc992d27 100644 --- a/sysdeps/aarch64/multiarch/memmove_falkor.S +++ b/sysdeps/aarch64/multiarch/memmove_falkor.S @@ -223,3 +223,4 @@ L(move_long): END (__memmove_falkor) libc_hidden_builtin_def (__memmove_falkor) +END_FILE diff --git a/sysdeps/aarch64/multiarch/memset_base64.S b/sysdeps/aarch64/multiarch/memset_base64.S index 8f85cd1caf..ee0b832ef9 100644 --- a/sysdeps/aarch64/multiarch/memset_base64.S +++ b/sysdeps/aarch64/multiarch/memset_base64.S @@ -184,3 +184,4 @@ L(zva_64): END (MEMSET) libc_hidden_builtin_def (MEMSET) +END_FILE diff --git a/sysdeps/aarch64/multiarch/memset_kunpeng.S b/sysdeps/aarch64/multiarch/memset_kunpeng.S index 8e051d4fd1..aa7ab62fdd 100644 --- a/sysdeps/aarch64/multiarch/memset_kunpeng.S +++ b/sysdeps/aarch64/multiarch/memset_kunpeng.S @@ -111,3 +111,4 @@ L(set_long): END (MEMSET) libc_hidden_builtin_def (MEMSET) #endif +END_FILE diff --git a/sysdeps/aarch64/multiarch/strlen_asimd.S b/sysdeps/aarch64/multiarch/strlen_asimd.S index 236a2c96a6..c28aa0ca8d 100644 --- a/sysdeps/aarch64/multiarch/strlen_asimd.S +++ b/sysdeps/aarch64/multiarch/strlen_asimd.S @@ -176,3 +176,4 @@ L(page_cross): END (__strlen_asimd) weak_alias (__strlen_asimd, strlen_asimd) libc_hidden_builtin_def (strlen_asimd) +END_FILE diff --git a/sysdeps/aarch64/rawmemchr.S b/sysdeps/aarch64/rawmemchr.S index 5c7a664fb4..4ad614a169 100644 --- a/sysdeps/aarch64/rawmemchr.S +++ b/sysdeps/aarch64/rawmemchr.S @@ -40,3 +40,4 @@ L(do_strlen): END (__rawmemchr) weak_alias (__rawmemchr, rawmemchr) libc_hidden_builtin_def (__rawmemchr) +END_FILE diff --git a/sysdeps/aarch64/setjmp.S b/sysdeps/aarch64/setjmp.S index 28fdd3f46a..9a0201ef70 100644 --- a/sysdeps/aarch64/setjmp.S +++ b/sysdeps/aarch64/setjmp.S @@ -73,3 +73,4 @@ ENTRY (__sigsetjmp) #endif END (__sigsetjmp) hidden_def (__sigsetjmp) +END_FILE diff --git a/sysdeps/aarch64/start.S b/sysdeps/aarch64/start.S index d96cf57e2d..e6c0393c20 100644 --- a/sysdeps/aarch64/start.S +++ b/sysdeps/aarch64/start.S @@ -46,6 +46,7 @@ .globl _start .type _start,#function _start: + BTI_C /* Create an initial frame with 0 LR and FP */ mov x29, #0 mov x30, #0 @@ -110,3 +111,4 @@ __data_start: .long 0 .weak data_start data_start = __data_start +END_FILE diff --git a/sysdeps/aarch64/strchr.S b/sysdeps/aarch64/strchr.S index 4a75e73945..e1f98aa42c 100644 --- a/sysdeps/aarch64/strchr.S +++ b/sysdeps/aarch64/strchr.S @@ -137,3 +137,4 @@ L(tail): END (strchr) libc_hidden_builtin_def (strchr) weak_alias (strchr, index) +END_FILE diff --git a/sysdeps/aarch64/strchrnul.S b/sysdeps/aarch64/strchrnul.S index a65be6cba8..a9ccc54205 100644 --- a/sysdeps/aarch64/strchrnul.S +++ b/sysdeps/aarch64/strchrnul.S @@ -129,3 +129,4 @@ L(tail): END(__strchrnul) weak_alias (__strchrnul, strchrnul) +END_FILE diff --git a/sysdeps/aarch64/strcmp.S b/sysdeps/aarch64/strcmp.S index d044c29e9b..a7bee697c0 100644 --- a/sysdeps/aarch64/strcmp.S +++ b/sysdeps/aarch64/strcmp.S @@ -182,3 +182,4 @@ L(done): RET END(strcmp) libc_hidden_builtin_def (strcmp) +END_FILE diff --git a/sysdeps/aarch64/strcpy.S b/sysdeps/aarch64/strcpy.S index 548130e413..631da4a358 100644 --- a/sysdeps/aarch64/strcpy.S +++ b/sysdeps/aarch64/strcpy.S @@ -321,3 +321,4 @@ libc_hidden_builtin_def (stpcpy) #else libc_hidden_builtin_def (strcpy) #endif +END_FILE diff --git a/sysdeps/aarch64/strlen.S b/sysdeps/aarch64/strlen.S index e01fab7c2a..3af25de4b8 100644 --- a/sysdeps/aarch64/strlen.S +++ b/sysdeps/aarch64/strlen.S @@ -222,3 +222,4 @@ L(page_cross): END (STRLEN) weak_alias (STRLEN, strlen) libc_hidden_builtin_def (strlen) +END_FILE diff --git a/sysdeps/aarch64/strncmp.S b/sysdeps/aarch64/strncmp.S index c5141fab8a..d289d4a3ba 100644 --- a/sysdeps/aarch64/strncmp.S +++ b/sysdeps/aarch64/strncmp.S @@ -270,3 +270,4 @@ L(ret0): END (strncmp) libc_hidden_builtin_def (strncmp) +END_FILE diff --git a/sysdeps/aarch64/strnlen.S b/sysdeps/aarch64/strnlen.S index 5981247dd9..964536dba9 100644 --- a/sysdeps/aarch64/strnlen.S +++ b/sysdeps/aarch64/strnlen.S @@ -213,3 +213,4 @@ END (__strnlen) libc_hidden_def (__strnlen) weak_alias (__strnlen, strnlen) libc_hidden_def (strnlen) +END_FILE diff --git a/sysdeps/aarch64/strrchr.S b/sysdeps/aarch64/strrchr.S index 94da08d351..9f6d956f21 100644 --- a/sysdeps/aarch64/strrchr.S +++ b/sysdeps/aarch64/strrchr.S @@ -164,3 +164,4 @@ L(null_search): END(strrchr) weak_alias (strrchr, rindex) libc_hidden_builtin_def (strrchr) +END_FILE diff --git a/sysdeps/aarch64/sysdep.h b/sysdeps/aarch64/sysdep.h index 604c489170..161d63c782 100644 --- a/sysdeps/aarch64/sysdep.h +++ b/sysdeps/aarch64/sysdep.h @@ -41,12 +41,40 @@ #define ASM_SIZE_DIRECTIVE(name) .size name,.-name +/* Branch Target Identitication support. */ +#define BTI_C hint 34 +#define BTI_J hint 36 + +#define FEATURE_1_BTI 1 +#define FEATURE_1_PAC 2 + +/* Add a GNU_PROPERTY_AARCH64_FEATURE_1_AND note. */ +#define GNU_PROPERTY(features) \ + .section .note.gnu.property, "a"; \ + .p2align 3; \ + .word 4; \ + .word 16; \ + .word 5; \ + .asciz "GNU"; \ + .word 0xc0000000; \ + .word 4; \ + .word features; \ + .word 0; + +/* GNU property note with the supported features. */ +#ifdef HAVE_AARCH64_BTI +# define END_FILE GNU_PROPERTY (FEATURE_1_BTI) +#else +# define END_FILE +#endif + /* Define an entry point visible from C. */ #define ENTRY(name) \ .globl C_SYMBOL_NAME(name); \ .type C_SYMBOL_NAME(name),%function; \ .align 4; \ C_LABEL(name) \ + BTI_C; \ cfi_startproc; \ CALL_MCOUNT @@ -56,6 +84,7 @@ .type C_SYMBOL_NAME(name),%function; \ .p2align align; \ C_LABEL(name) \ + BTI_C; \ cfi_startproc; \ CALL_MCOUNT @@ -68,10 +97,11 @@ .globl C_SYMBOL_NAME(name); \ .type C_SYMBOL_NAME(name),%function; \ .p2align align; \ - .rep padding; \ + .rep padding - 1; /* -1 for bti c. */ \ nop; \ .endr; \ C_LABEL(name) \ + BTI_C; \ cfi_startproc; \ CALL_MCOUNT diff --git a/sysdeps/unix/sysv/linux/aarch64/__read_tp.S b/sysdeps/unix/sysv/linux/aarch64/__read_tp.S index 12e1131fe7..7825aa5e1b 100644 --- a/sysdeps/unix/sysv/linux/aarch64/__read_tp.S +++ b/sysdeps/unix/sysv/linux/aarch64/__read_tp.S @@ -23,3 +23,4 @@ ENTRY (__read_tp) mrs x0, tpidr_el0 RET END (__read_tp) +END_FILE diff --git a/sysdeps/unix/sysv/linux/aarch64/clone.S b/sysdeps/unix/sysv/linux/aarch64/clone.S index 2b14106fd1..15b5a7b217 100644 --- a/sysdeps/unix/sysv/linux/aarch64/clone.S +++ b/sysdeps/unix/sysv/linux/aarch64/clone.S @@ -85,3 +85,4 @@ thread_start: libc_hidden_def (__clone) weak_alias (__clone, clone) +END_FILE diff --git a/sysdeps/unix/sysv/linux/aarch64/getcontext.S b/sysdeps/unix/sysv/linux/aarch64/getcontext.S index 8571556189..95f61d5f28 100644 --- a/sysdeps/unix/sysv/linux/aarch64/getcontext.S +++ b/sysdeps/unix/sysv/linux/aarch64/getcontext.S @@ -107,3 +107,4 @@ ENTRY(__getcontext) PSEUDO_END (__getcontext) weak_alias (__getcontext, getcontext) +END_FILE diff --git a/sysdeps/unix/sysv/linux/aarch64/ioctl.S b/sysdeps/unix/sysv/linux/aarch64/ioctl.S index ed36e309c4..19abf43726 100644 --- a/sysdeps/unix/sysv/linux/aarch64/ioctl.S +++ b/sysdeps/unix/sysv/linux/aarch64/ioctl.S @@ -30,3 +30,4 @@ PSEUDO_END (__ioctl) libc_hidden_def (__ioctl) weak_alias (__ioctl, ioctl) +END_FILE diff --git a/sysdeps/unix/sysv/linux/aarch64/libc-__read_tp.S b/sysdeps/unix/sysv/linux/aarch64/libc-__read_tp.S index 6de259ed83..df5eec8599 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libc-__read_tp.S +++ b/sysdeps/unix/sysv/linux/aarch64/libc-__read_tp.S @@ -17,3 +17,4 @@ . */ #include <__read_tp.S> +END_FILE diff --git a/sysdeps/unix/sysv/linux/aarch64/setcontext.S b/sysdeps/unix/sysv/linux/aarch64/setcontext.S index 61fb813db3..d9d6c0f364 100644 --- a/sysdeps/unix/sysv/linux/aarch64/setcontext.S +++ b/sysdeps/unix/sysv/linux/aarch64/setcontext.S @@ -128,3 +128,4 @@ ENTRY (__startcontext) cbnz x0, __setcontext 1: b HIDDEN_JUMPTARGET (exit) END (__startcontext) +END_FILE diff --git a/sysdeps/unix/sysv/linux/aarch64/syscall.S b/sysdeps/unix/sysv/linux/aarch64/syscall.S index 993e307fcf..10e59ece58 100644 --- a/sysdeps/unix/sysv/linux/aarch64/syscall.S +++ b/sysdeps/unix/sysv/linux/aarch64/syscall.S @@ -42,3 +42,4 @@ ENTRY (syscall) 1: b SYSCALL_ERROR PSEUDO_END (syscall) +END_FILE diff --git a/sysdeps/unix/sysv/linux/aarch64/vfork.S b/sysdeps/unix/sysv/linux/aarch64/vfork.S index 9c8ea48c5a..0306003a22 100644 --- a/sysdeps/unix/sysv/linux/aarch64/vfork.S +++ b/sysdeps/unix/sysv/linux/aarch64/vfork.S @@ -40,3 +40,4 @@ libc_hidden_def (__vfork) weak_alias (__vfork, vfork) strong_alias (__vfork, __libc_vfork) +END_FILE