From patchwork Thu Mar 17 19:28:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Florian Weimer X-Patchwork-Id: 52061 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id E42CC3842423 for ; Thu, 17 Mar 2022 19:28:29 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org E42CC3842423 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1647545309; bh=tcs8TfuaXpSq6pqkar4KDnagc7p6TTQJnjFBwVeCgh8=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=nNXdgml9gzH79odcDVMValAQAFsniSU9jEVpiZ6dm0PCqrVoKJ+fE0OfFOOIcjnu6 NU283sONE7svs5tqTQp7xlrrPdw9n546fGqpcUs4SAhOeaFslcbHoEUZXKPCLNKf0b yN0dzjZwMWE45KhACFB67zMW2VtZP04TwJVwsVXM= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id 32D89385801E for ; Thu, 17 Mar 2022 19:28:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 32D89385801E Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-607--lzxIYnjPn6XX-k5eLwJ6g-1; Thu, 17 Mar 2022 15:28:05 -0400 X-MC-Unique: -lzxIYnjPn6XX-k5eLwJ6g-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 63F0480159B for ; Thu, 17 Mar 2022 19:28:05 +0000 (UTC) Received: from oldenburg.str.redhat.com (unknown [10.39.192.88]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9118C1400E73 for ; Thu, 17 Mar 2022 19:28:04 +0000 (UTC) To: libc-alpha@sourceware.org Subject: [PATCH 00/26] vfprintf rework to remove vtables X-From-Line: 9bed5c50fe3e6c5872c94c451413ca50a2051ef5 Mon Sep 17 00:00:00 2001 Message-Id: Date: Thu, 17 Mar 2022 20:28:02 +0100 User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-5.1 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Florian Weimer via Libc-alpha From: Florian Weimer Reply-To: Florian Weimer Errors-To: libc-alpha-bounces+patchwork=sourceware.org@sourceware.org Sender: "Libc-alpha" This started out as an attempt to fix bug 27124, the vtable hardening bypass involving obstack_printf. The new buffers are modeled after the write pointer/end pointer abstraction used for inline putc_unlocked. They have considerably lower setup overhead than the internal stdio-compatible streams used by sprintf. Performance seems to be at least on par with the old implementation, and a bit better for floating point formatting. The new internal interfaces are added in commit “stdio-common: Introduce buffers for implementing printf”. The bulk of the conversions happen in “stdio-common: Convert vfprintf and related functions to buffers”. Initially, I converted things piece by piece, but that required implementing additional bounce buffers for translation, so this one commit converts the majority of the functions. The commit message of this large commit has some suggestions for review order. Tested on i686-linux-gnu and x86_64-linux-gnu. Built with build-many-glibcs.py. Thanks, Florian Florian Weimer (26): libio: Convert tst_swprintf to the test framework libio: Flush-only _IO_str_overflow must not return EOF (bug 28949) stdio-common: Add wide stream coverage to tst-vfprintf-user-type stdio-common: Add tst-printf-width-i18n to cover numeric field width vfprintf: Move argument processing into vfprintf-process-arg.c vfprintf: Consolidate some multibyte/wide character processing __printf_fphex always uses LC_NUMERIC stdio-common: Add tst-memstream-string for open_memstream overflow stdio-common: Add printf specifier registry to stdio-common: Move union printf_arg int stdio-common: Simplify printf_unknown interface in vfprintf-internal.c locale: Call _nl_unload_locale from _nl_archive_subfreeres locale: Remove cleanup function pointer from struct __localedata locale: Remove private union from struct __locale_data locale: Add more cached data to LC_CTYPE locale: Implement struct grouping_iterator stdio-common: Introduce buffers for implementing printf stdio-common: Add __printf_function_invoke stdio-common: Add __translated_number_width stdio-common: Convert vfprintf and related functions to buffers stdio-common: Add lock optimization to vfprintf and vfwprintf libio: Convert __vsprintf_internal to buffers libio: Convert __vasprintf_internal to buffers libio: Convert __vdprintf_internal to buffers libio: Convert __obstack_vprintf_internal to buffers (bug 27124) libio: Convert __vswprintf_internal to buffers (bug 27857) include/printf.h | 79 +- include/printf_buffer.h | 332 +++++ libio/iovdprintf.c | 69 +- libio/iovsprintf.c | 70 +- libio/obprintf.c | 170 +-- libio/strops.c | 5 +- libio/tst-vtables-common.c | 9 +- libio/tst_swprintf.c | 110 +- libio/vasprintf.c | 141 +- libio/vsnprintf.c | 131 +- libio/vswprintf.c | 100 +- libio/wstrops.c | 5 +- locale/C-address.c | 2 +- locale/C-collate.c | 2 +- locale/C-ctype.c | 9 +- locale/C-identification.c | 2 +- locale/C-measurement.c | 2 +- locale/C-messages.c | 2 +- locale/C-monetary.c | 2 +- locale/C-name.c | 2 +- locale/C-numeric.c | 2 +- locale/C-paper.c | 2 +- locale/C-telephone.c | 2 +- locale/C-time.c | 2 +- locale/findlocale.c | 2 +- locale/loadarchive.c | 8 +- locale/loadlocale.c | 89 +- locale/localeinfo.h | 48 +- locale/setlocale.c | 2 +- manual/stdio.texi | 7 +- stdio-common/Makefile | 29 +- stdio-common/Xprintf_buffer_done.c | 40 + stdio-common/Xprintf_buffer_flush.c | 72 + stdio-common/Xprintf_buffer_pad_1.c | 44 + stdio-common/Xprintf_buffer_putc_1.c | 29 + stdio-common/Xprintf_buffer_puts_1.c | 37 + stdio-common/Xprintf_buffer_write.c | 43 + stdio-common/Xprintf_function_invoke.c | 42 + stdio-common/grouping_iterator.c | 125 ++ stdio-common/grouping_iterator.h | 65 + stdio-common/printf-parse.h | 23 - stdio-common/printf_buffer-char.h | 24 + stdio-common/printf_buffer-wchar_t.h | 24 + stdio-common/printf_buffer_as_file.c | 148 ++ stdio-common/printf_buffer_as_file.h | 87 ++ stdio-common/printf_buffer_done.c | 21 + stdio-common/printf_buffer_flush.c | 81 ++ stdio-common/printf_buffer_pad_1.c | 21 + stdio-common/printf_buffer_putc_1.c | 21 + stdio-common/printf_buffer_puts_1.c | 21 + stdio-common/printf_buffer_to_file.c | 122 ++ stdio-common/printf_buffer_to_file.h | 57 + stdio-common/printf_buffer_write.c | 21 + stdio-common/printf_fp.c | 736 +++++----- stdio-common/printf_fphex.c | 271 ++-- stdio-common/printf_function_invoke.c | 22 + stdio-common/reg-printf.c | 7 - stdio-common/translated_number_width.c | 42 + stdio-common/tst-grouping_iterator.c | 262 ++++ stdio-common/tst-memstream-string.c | 85 ++ stdio-common/tst-printf-width-i18n.c | 95 ++ stdio-common/tst-vfprintf-user-type.c | 128 +- stdio-common/vfprintf-internal.c | 1189 +++-------------- stdio-common/vfprintf-process-arg.c | 489 +++++++ stdio-common/wprintf_buffer_as_file.c | 153 +++ stdio-common/wprintf_buffer_done.c | 21 + stdio-common/wprintf_buffer_flush.c | 42 + stdio-common/wprintf_buffer_pad_1.c | 21 + stdio-common/wprintf_buffer_putc_1.c | 21 + stdio-common/wprintf_buffer_puts_1.c | 21 + stdio-common/wprintf_buffer_to_file.c | 55 + stdio-common/wprintf_buffer_write.c | 21 + stdio-common/wprintf_function_invoke.c | 22 + stdlib/strfmon_l.c | 196 ++- stdlib/strfrom-skeleton.c | 38 +- sysdeps/ia64/fpu/printf_fphex.c | 8 +- .../ieee754/ldbl-128/printf_fphex_macros.h | 36 +- sysdeps/ieee754/ldbl-128ibm/printf_fphex.c | 36 +- sysdeps/ieee754/ldbl-96/printf_fphex.c | 22 +- sysdeps/x86_64/fpu/printf_fphex.c | 21 +- time/alt_digit.c | 49 +- time/era.c | 49 +- time/lc-time-cleanup.c | 5 +- wcsmbs/wcsmbsload.c | 27 +- wcsmbs/wcsmbsload.h | 11 +- 85 files changed, 4275 insertions(+), 2531 deletions(-) create mode 100644 include/printf_buffer.h create mode 100644 stdio-common/Xprintf_buffer_done.c create mode 100644 stdio-common/Xprintf_buffer_flush.c create mode 100644 stdio-common/Xprintf_buffer_pad_1.c create mode 100644 stdio-common/Xprintf_buffer_putc_1.c create mode 100644 stdio-common/Xprintf_buffer_puts_1.c create mode 100644 stdio-common/Xprintf_buffer_write.c create mode 100644 stdio-common/Xprintf_function_invoke.c create mode 100644 stdio-common/grouping_iterator.c create mode 100644 stdio-common/grouping_iterator.h create mode 100644 stdio-common/printf_buffer-char.h create mode 100644 stdio-common/printf_buffer-wchar_t.h create mode 100644 stdio-common/printf_buffer_as_file.c create mode 100644 stdio-common/printf_buffer_as_file.h create mode 100644 stdio-common/printf_buffer_done.c create mode 100644 stdio-common/printf_buffer_flush.c create mode 100644 stdio-common/printf_buffer_pad_1.c create mode 100644 stdio-common/printf_buffer_putc_1.c create mode 100644 stdio-common/printf_buffer_puts_1.c create mode 100644 stdio-common/printf_buffer_to_file.c create mode 100644 stdio-common/printf_buffer_to_file.h create mode 100644 stdio-common/printf_buffer_write.c create mode 100644 stdio-common/printf_function_invoke.c create mode 100644 stdio-common/translated_number_width.c create mode 100644 stdio-common/tst-grouping_iterator.c create mode 100644 stdio-common/tst-memstream-string.c create mode 100644 stdio-common/tst-printf-width-i18n.c create mode 100644 stdio-common/vfprintf-process-arg.c create mode 100644 stdio-common/wprintf_buffer_as_file.c create mode 100644 stdio-common/wprintf_buffer_done.c create mode 100644 stdio-common/wprintf_buffer_flush.c create mode 100644 stdio-common/wprintf_buffer_pad_1.c create mode 100644 stdio-common/wprintf_buffer_putc_1.c create mode 100644 stdio-common/wprintf_buffer_puts_1.c create mode 100644 stdio-common/wprintf_buffer_to_file.c create mode 100644 stdio-common/wprintf_buffer_write.c create mode 100644 stdio-common/wprintf_function_invoke.c base-commit: e297d33c73aa22c86228170343c7b2f9bfac6f1e