From patchwork Thu Jan 8 18:09:46 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Packard X-Patchwork-Id: 127654 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from vm01.sourceware.org (localhost [127.0.0.1]) by sourceware.org (Postfix) with ESMTP id EB8434BA2E21 for ; Thu, 8 Jan 2026 18:11:44 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org EB8434BA2E21 Authentication-Results: sourceware.org; dkim=fail reason="signature verification failed" (2048-bit key, unprotected) header.d=keithp.com header.i=@keithp.com header.a=rsa-sha256 header.s=mail header.b=oqyNt6Nq; dkim=fail reason="signature verification failed" (2048-bit key) header.d=keithp.com header.i=@keithp.com header.a=rsa-sha256 header.s=mail header.b=va9JXIf/ X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from elaine.keithp.com (home.keithp.com [63.227.221.253]) by sourceware.org (Postfix) with ESMTPS id 5BA8B4BA2E04; Thu, 8 Jan 2026 18:10:00 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5BA8B4BA2E04 Authentication-Results: sourceware.org; dmarc=pass (p=quarantine dis=none) header.from=keithp.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=keithp.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 5BA8B4BA2E04 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=63.227.221.253 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1767895800; cv=none; b=u6F6ZckCVAYKXxbE8AFXkXchGeG1L/mOX8J5u80X/5Y0FIO5kxz9pN42EnGc6fqMkfvq4cD32QUoLT8uxwphqzRAzy800IqmrlrARCkKW51Nc8GYGNvve7UN5byP4BUE/TwbDXoRoTTt/me8dvWOA2QU3vzUXiNfGvUSxBUoeLc= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1767895800; c=relaxed/simple; bh=BfWJO2LqqLOtf3fsYYDZ1r+2HmBAi5GwSfwsxfkPK44=; h=DKIM-Signature:DKIM-Signature:From:To:Subject:Date:Message-ID: MIME-Version; b=fWXmeAJ4CgWNKWHq3sx5ty9yLkHsORyTNwJdcJGgwuYFKsvCr+NI34j2uYDVwKDFUCHRrxRNQZEWMxiA2X23TnF0liGPwhreHS/MagTCeFmyxivflv1MZRm8ptVqYcemkDXTIixoxDvbEwLVWWHOJGN7bRYWSHQNBiVgFVlCQCM= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5BA8B4BA2E04 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=keithp.com; s=mail; t=1767895798; bh=BfWJO2LqqLOtf3fsYYDZ1r+2HmBAi5GwSfwsxfkPK44=; h=From:To:Cc:Subject:Date:From; b=oqyNt6Nq8j13Ue1ALPiFr54OgC4QFY01RnpkKJBHbovdVcFRozmUyVJMyop7WIDZL Nhg4FWdjgFvo8/hVLV48lVxHTWUdoThklFMCRhoLtwoave+F+a5sn42f6WYNvLamjK 4ZF1ii4WsgNoE247EDf2xHH3aQNMwGCQgkBRX9Px8ejoLo4TyP/X3HQ9ae5CREDQws 8ovyd7FuTHNX7QoWFBIE68HvlnaOz9h33BNcBLXpIC2IM31qIRZSfTRo4SQ3u7uY58 h7xmKu+eKllN/a2ygLZQFi2YFL8tLIbXTPPSBCXYGK4VED+ceSnSwgGp2ynJ8ffVLY HaantnDLHWVtQ== Received: from localhost (localhost [127.0.0.1]) by elaine.keithp.com (Postfix) with ESMTP id D3D3B3F28708; Thu, 8 Jan 2026 10:09:58 -0800 (PST) X-Virus-Scanned: Debian amavis at keithp.com Received: from elaine.keithp.com ([127.0.0.1]) by localhost (elaine.keithp.com [127.0.0.1]) (amavis, port 10024) with LMTP id Etjh2-FS5p6s; Thu, 8 Jan 2026 10:09:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=keithp.com; s=mail; t=1767895797; bh=BfWJO2LqqLOtf3fsYYDZ1r+2HmBAi5GwSfwsxfkPK44=; h=From:To:Cc:Subject:Date:From; b=va9JXIf/sTl9EHacZkZ5WKzMY8LtQgT2HLKYCYjTQQfeUzbG5O7PP1Am9TCfw0Eh7 V+O7CRpjjkrBfWvBGMfv/Nvud+Uxdvhvp1fy6YN4pVuyTdXaUtOqDHjTdXlH/K8Njx tIUDkihVUOOdVwi7yaEZX+L6C2Gs/wMg3L8aQhKiVRzuzC7pEoKuNeSWIkEhFVR3M3 T8fNnS+T7+mEzIZ2tbjUmOoZ+W9srmGRYMQiElL8yyukB0TYKnbE2F3SWZnio3W60s 9YAEVOTVPHJAQyXXiPMEKIlvDS7WjIUgAa+D2M/2fhgin/g8aFn5RzUN0MjqLyP0WT tPZdPN60wKrOw== Received: from koto.keithp.com (koto.keithp.com [192.168.11.2]) by elaine.keithp.com (Postfix) with ESMTPSA id D850D3F26FDE; Thu, 8 Jan 2026 10:09:57 -0800 (PST) Received: by koto.keithp.com (Postfix, from userid 1000) id CDF743409F86; Thu, 08 Jan 2026 10:09:57 -0800 (PST) From: Keith Packard To: gcc-patches@gcc.gnu.org, libstdc++@gcc.gnu.org Cc: Keith Packard Subject: [PATCH] libstdc++: Allow std::ctype_base masks to contain multiple set bits Date: Thu, 8 Jan 2026 10:09:46 -0800 Message-ID: <20260108180946.1298387-1-keithp@keithp.com> X-Mailer: git-send-email 2.51.0 MIME-Version: 1.0 X-Spam-Status: No, score=-10.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_VALIDITY_RPBL_BLOCKED, RCVD_IN_VALIDITY_SAFE_BLOCKED, SPF_HELO_NONE, SPF_PASS, TXREP, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~patchwork=sourceware.org@gcc.gnu.org The C++ standard does not require that each std::ctype_base mask value have only one bit set; in fact the expository values presented in the text explicitly include two multi-bit values: // numeric values are for exposition only. static const mask space = 1 << 0; static const mask print = 1 << 1; static const mask cntrl = 1 << 2; static const mask upper = 1 << 3; static const mask lower = 1 << 4; static const mask alpha = 1 << 5; static const mask digit = 1 << 6; static const mask punct = 1 << 7; static const mask xdigit = 1 << 8; static const mask blank = 1 << 9; static const mask alnum = alpha | digit; static const mask graph = alnum | punct; Adapt the code to handle this possibility in three places. 1. In _M_initialize_ctype use explicit assignments from the std::ctype_base mask values into _M_bit array instead of assuming that iterating over values from 1<<0 to 1<<15 will match all of the possible std::ctype_base mask values. 2. In do_is(mask, char_type) change the condition testing __m against _M_bit[__bitcur] to require *all* bits in _M_bit[__bitcur] to be present in __m before calling iswctype with the matching _M_wmask value. For a C library where alpha = (upper|lower) when __m is upper the previous version would have done iswctype(*__lo, wctype("alpha")) iswctype(*__lo, wctype("lower")) the new version will do iswctype(*__lo, wctype("upper")) 3. In do_is(const wchar_t *, const wchar_t *, mask *), replace the code setting _M_bit values when iswctype returns true to code which clears _M_bit values when iswctype returns false. For a C library where alpha = (upper|lower), when when iswctype(wctype("alpha")) returns true iswctype(wctype("upper")) returns false iswctype(wctype("lower")) returns true the old version would have done __m = 0 __m |= alpha __m |= lower return value = (upper|lower) the new version will do __m = upper|lower __m &= ~upper return value = (lower) These changes should not introduce significant additional complexity into these computations nor affect results with C libraries which use independent values for the std::ctype_base masks. These changes do reduce the range of mask values used in iterations from 16 to 12, so there may actually be a tiny performance improvement. Signed-off-by: Keith Packard --- .../config/locale/generic/ctype_members.cc | 81 +++++++++++++++---- 1 file changed, 64 insertions(+), 17 deletions(-) diff --git a/libstdc++-v3/config/locale/generic/ctype_members.cc b/libstdc++-v3/config/locale/generic/ctype_members.cc index 2d61347feb1..21c25a68915 100644 --- a/libstdc++-v3/config/locale/generic/ctype_members.cc +++ b/libstdc++-v3/config/locale/generic/ctype_members.cc @@ -53,6 +53,15 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION { } #ifdef _GLIBCXX_USE_WCHAR_T + + // Values which index into _M_bit and _M_wmask for each classifier + enum { + _id_alnum, _id_alpha, _id_blank, _id_cntrl, + _id_digit, _id_graph, _id_lower, _id_print, + _id_punct, _id_space, _id_upper, _id_xdigit, + _id_num, + }; + ctype::__wmask_type ctype::_M_convert_to_wmask(const mask __m) const throw() { @@ -138,11 +147,20 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION do_is(mask __m, char_type __c) const { bool __ret = false; - // Generically, 15 (instead of 11) since we don't know the numerical - // encoding of the various categories in /usr/include/ctype.h. - const size_t __bitmasksize = 15; - for (size_t __bitcur = 0; __bitcur <= __bitmasksize; ++__bitcur) - if (__m & _M_bit[__bitcur] + + // Each _M_bit may have more than one bit set. Only check tests for which + // all of the matching _M_bit values are set. This skips tests with some + // bits which are not set. For example: + // + // alpha = lower|upper + // __c = 'a' + // __m = upper + // + // We evaluate iswupper (which returns false), but we skip iswalpha (which + // would return true) because __m is missing the lower bit, so the return + // value is false. + for (size_t __bitcur = 0; __bitcur <= _id_num; ++__bitcur) + if ((__m & _M_bit[__bitcur]) == _M_bit[__bitcur] && iswctype(__c, _M_wmask[__bitcur])) { __ret = true; @@ -157,13 +175,29 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION { for (;__lo < __hi; ++__vec, ++__lo) { - // Generically, 15 (instead of 11) since we don't know the numerical - // encoding of the various categories in /usr/include/ctype.h. - const size_t __bitmasksize = 15; - mask __m = 0; - for (size_t __bitcur = 0; __bitcur <= __bitmasksize; ++__bitcur) - if (iswctype(*__lo, _M_wmask[__bitcur])) - __m |= _M_bit[__bitcur]; + + // Start with all bits set. Iterate over all wctypes, clearing bits for + // non-matching values. For example: + // + // alpha == lower|upper + // *__lo == 'a' + // + // __m = + // iswalpha and iswlower match + // __m left alone + // iswupper does not match + // __m &= ~upper + // + // The return value will satisfy the following conditions: + // + // (__m & alpha) != 0 + // (__m & lower) != 0 + // (__m & upper) == 0 + mask __m = (alnum|alpha|blank|cntrl|digit|graph| + lower|print|punct|space|upper|xdigit); + for (size_t __bitcur = 0; __bitcur < _id_num; ++__bitcur) + if (!iswctype(*__lo, _M_wmask[__bitcur])) + __m &= ~_M_bit[__bitcur]; *__vec = __m; } return __hi; @@ -264,11 +298,24 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION __i < sizeof(_M_widen) / sizeof(wint_t); ++__i) _M_widen[__i] = btowc(__i); - for (size_t __i = 0; __i <= 15; ++__i) - { - _M_bit[__i] = static_cast(1 << __i); - _M_wmask[__i] = _M_convert_to_wmask(_M_bit[__i]); - } + // Set these directly from the OS provided values. + // We can't simply iterate over bit values because + // they may have more than one bit set + _M_bit[_id_alnum] = alnum; + _M_bit[_id_alpha] = alpha; + _M_bit[_id_blank] = blank; + _M_bit[_id_cntrl] = cntrl; + _M_bit[_id_digit] = digit; + _M_bit[_id_graph] = graph; + _M_bit[_id_lower] = lower; + _M_bit[_id_print] = print; + _M_bit[_id_punct] = punct; + _M_bit[_id_space] = space; + _M_bit[_id_upper] = upper; + _M_bit[_id_xdigit] = xdigit; + + for (size_t __i = 0; __i <= _id_num; __i++) + _M_wmask[__i] = _M_convert_to_wmask(_M_bit[__i]); } #endif // _GLIBCXX_USE_WCHAR_T