[2/3] : C++20 P0482R6 and C2X N2653: Implement mbrtoc8, c8rtomb, char8_t
Checks
| Context |
Check |
Description |
| dj/TryBot-apply_patch |
success
|
Patch applied to master at the time it was sent
|
Commit Message
This patch provides implementations for the mbrtoc8 and c8rtomb
functions adopted for C++20 via WG21 P0482R6 [1] and for C2X via WG14
N2653 [2]. It also provides the char8_t typedef adopted for C2X via WG14
N2653 [2].
The mbrtoc8 and c8rtomb functions are declared in uchar.h in C2X mode or
when the _GNU_SOURCE macro or C++20 __cpp_char8_t feature test macro is
defined.
The char8_t typedef is declared in uchar.h in C2X mode or when the
_GNU_SOURCE macro is defined and the C++20 __cpp_char8_t feature test
macro is not defined (if __cpp_char8_t is defined, then char8_t is a
builtin type).
Tested on Linux x86_64.
Tom.
[1]: WG21 P0482R6
"char8_t: A type for UTF-8 characters and strings (Revision 6)"
https://wg21.link/p0482r6
[2]: WG14 N2653
"char8_t: A type for UTF-8 characters and strings (Revision 1)"
http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2653.htm
Comments
On Sun, 27 Feb 2022, Tom Honermann via Libc-alpha wrote:
> This patch provides implementations for the mbrtoc8 and c8rtomb functions
> adopted for C++20 via WG21 P0482R6 [1] and for C2X via WG14 N2653 [2]. It also
> provides the char8_t typedef adopted for C2X via WG14 N2653 [2].
>
> The mbrtoc8 and c8rtomb functions are declared in uchar.h in C2X mode or when
> the _GNU_SOURCE macro or C++20 __cpp_char8_t feature test macro is defined.
>
> The char8_t typedef is declared in uchar.h in C2X mode or when the _GNU_SOURCE
> macro is defined and the C++20 __cpp_char8_t feature test macro is not defined
> (if __cpp_char8_t is defined, then char8_t is a builtin type).
You should almost never have "defined __USE_GNU || ..." in header
conditionals, because _GNU_SOURCE implies the other feature test macros.
On 2/28/22 6:01 PM, Joseph Myers wrote:
> On Sun, 27 Feb 2022, Tom Honermann via Libc-alpha wrote:
>
>> This patch provides implementations for the mbrtoc8 and c8rtomb functions
>> adopted for C++20 via WG21 P0482R6 [1] and for C2X via WG14 N2653 [2]. It also
>> provides the char8_t typedef adopted for C2X via WG14 N2653 [2].
>>
>> The mbrtoc8 and c8rtomb functions are declared in uchar.h in C2X mode or when
>> the _GNU_SOURCE macro or C++20 __cpp_char8_t feature test macro is defined.
>>
>> The char8_t typedef is declared in uchar.h in C2X mode or when the _GNU_SOURCE
>> macro is defined and the C++20 __cpp_char8_t feature test macro is not defined
>> (if __cpp_char8_t is defined, then char8_t is a builtin type).
> You should almost never have "defined __USE_GNU || ..." in header
> conditionals, because _GNU_SOURCE implies the other feature test macros.
Ah, right. Thank you, a corrected patch is attached.
Tom.
On 2/28/22 22:40, Tom Honermann via Libc-alpha wrote:
>
> On 2/28/22 6:01 PM, Joseph Myers wrote:
>> On Sun, 27 Feb 2022, Tom Honermann via Libc-alpha wrote:
>>
>>> This patch provides implementations for the mbrtoc8 and c8rtomb functions
>>> adopted for C++20 via WG21 P0482R6 [1] and for C2X via WG14 N2653 [2]. It also
>>> provides the char8_t typedef adopted for C2X via WG14 N2653 [2].
>>>
>>> The mbrtoc8 and c8rtomb functions are declared in uchar.h in C2X mode or when
>>> the _GNU_SOURCE macro or C++20 __cpp_char8_t feature test macro is defined.
>>>
>>> The char8_t typedef is declared in uchar.h in C2X mode or when the _GNU_SOURCE
>>> macro is defined and the C++20 __cpp_char8_t feature test macro is not defined
>>> (if __cpp_char8_t is defined, then char8_t is a builtin type).
>> You should almost never have "defined __USE_GNU || ..." in header
>> conditionals, because _GNU_SOURCE implies the other feature test macros.
>
> Ah, right. Thank you, a corrected patch is attached.
Tom,
Could you please repost a v2 with Joseph's change incorporated?
Your subject is also a bit odd with a leading colon.
My suggestion is to use git send-email to post the series with a cover-letter
to the list.
The cover-letter can explain anything that the commit message doesn't include.
On 01/03/2022 00:40, Tom Honermann via Libc-alpha wrote:
> Subject:
> Re: [PATCH 2/3]: C++20 P0482R6 and C2X N2653: Implement mbrtoc8, c8rtomb, char8_t
> From:
> Tom Honermann via Libc-alpha <libc-alpha@sourceware.org>
> Date:
> 01/03/2022 00:40
>
> To:
> Joseph Myers <joseph@codesourcery.com>
> CC:
> libc-alpha <libc-alpha@sourceware.org>
>
>
>
> On 2/28/22 6:01 PM, Joseph Myers wrote:
>> On Sun, 27 Feb 2022, Tom Honermann via Libc-alpha wrote:
>>
>>> This patch provides implementations for the mbrtoc8 and c8rtomb functions
>>> adopted for C++20 via WG21 P0482R6 [1] and for C2X via WG14 N2653 [2]. It also
>>> provides the char8_t typedef adopted for C2X via WG14 N2653 [2].
I think you have missed the links for [1] and [2] (they are on previous message).
Usually I put the link references after the remarks.
>>>
>>> The mbrtoc8 and c8rtomb functions are declared in uchar.h in C2X mode or when
>>> the _GNU_SOURCE macro or C++20 __cpp_char8_t feature test macro is defined.
>>>
>>> The char8_t typedef is declared in uchar.h in C2X mode or when the _GNU_SOURCE
>>> macro is defined and the C++20 __cpp_char8_t feature test macro is not defined
>>> (if __cpp_char8_t is defined, then char8_t is a builtin type).
>> You should almost never have "defined __USE_GNU || ..." in header
>> conditionals, because _GNU_SOURCE implies the other feature test macros.
>
> Ah, right. Thank you, a corrected patch is attached.
Patch looks ok, some comments below.
>
> Tom.
> Attachments:
>
> n2653-2.patch 28,4 KB
As before, please send the patch inline instead of attachment (even if it is large
as this one).
On a next version, use git send-mail or similar and put the initial message as
the cover letter.
>
> commit f163269acc109dc41bbf09ea56135ae94dc1e963
> Author: Tom Honermann <tom@honermann.net>
> Date: Sun Feb 27 10:28:58 2022 -0500
>
> Implement mbrtoc8(), c8rtomb(), and the char8_t typedef.
>
> This change provides implementations for the mbrtoc8 and c8rtomb
> functions adopted for C++20 via WG21 P0482R6 and for C2X via WG14
> N2653. It also provides the char8_t typedef from WG14 N2653.
>
> The mbrtoc8 and c8rtomb functions are declared in uchar.h in C2X
> mode or when the _GNU_SOURCE macro or C++20 __cpp_char8_t feature
> test macro is defined.
>
> The char8_t typedef is declared in uchar.h in C2X mode or when the
> _GNU_SOURCE macro is defined and the C++20 __cpp_char8_t feature
> test macro is not defined (if __cpp_char8_t is defined, then char8_t
> is a builtin type).
>
> diff --git a/NEWS b/NEWS
> index 626eeabf5d..3c4b597ab3 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -9,6 +9,15 @@ Version 2.36
>
> Major new features:
>
> +* Support for the mbrtoc8 and c8rtomb multibyte/UTF-8 character conversion
> + functions has been added per the ISO C2X N2653 and C++20 P0482R6 proposals.
> + Support for the char8_t typedef has been added per the ISO C2X N2653
> + proposal. The functions are declared in uchar.h in C2X mode or when the
Double space after period.
> + _GNU_SOURCE macro or C++20 __cpp_char8_t feature test macro is defined.
> + The char8_t typedef is declared in uchar.h in C2X mode or when the
> + _GNU_SOURCE macro is defined and the C++20 __cpp_char8_t feature test macro
> + is not defined (if __cpp_char8_t is defined, then char8_t is a builtin type).
> +
> [Add new features here]
Once a new feature is added the '[...]' mark can be removed.
>
> Deprecated and removed features, and other changes affecting compatibility:
> diff --git a/sysdeps/mach/hurd/i386/libc.abilist b/sysdeps/mach/hurd/i386/libc.abilist
> index 4dc87e9061..66fb0e28fa 100644
> --- a/sysdeps/mach/hurd/i386/libc.abilist
> +++ b/sysdeps/mach/hurd/i386/libc.abilist
> @@ -2289,6 +2289,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 close_range F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 __confstr_chk F
> GLIBC_2.4 __fgets_chk F
> GLIBC_2.4 __fgets_unlocked_chk F
> diff --git a/sysdeps/unix/sysv/linux/aarch64/libc.abilist b/sysdeps/unix/sysv/linux/aarch64/libc.abilist
> index 1b63d9e447..82dfae3421 100644
> --- a/sysdeps/unix/sysv/linux/aarch64/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/aarch64/libc.abilist
> @@ -2616,3 +2616,5 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> diff --git a/sysdeps/unix/sysv/linux/alpha/libc.abilist b/sysdeps/unix/sysv/linux/alpha/libc.abilist
> index e7e4cf7d2a..56d9094a58 100644
> --- a/sysdeps/unix/sysv/linux/alpha/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/alpha/libc.abilist
> @@ -2713,6 +2713,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 _IO_fprintf F
> GLIBC_2.4 _IO_printf F
> GLIBC_2.4 _IO_sprintf F
> diff --git a/sysdeps/unix/sysv/linux/arc/libc.abilist b/sysdeps/unix/sysv/linux/arc/libc.abilist
> index bc3d228e31..fb7fc117f3 100644
> --- a/sysdeps/unix/sysv/linux/arc/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/arc/libc.abilist
> @@ -2377,3 +2377,5 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> diff --git a/sysdeps/unix/sysv/linux/arm/be/libc.abilist b/sysdeps/unix/sysv/linux/arm/be/libc.abilist
> index db7039c4ab..f7c3b87bc0 100644
> --- a/sysdeps/unix/sysv/linux/arm/be/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/arm/be/libc.abilist
> @@ -496,6 +496,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 _Exit F
> GLIBC_2.4 _IO_2_1_stderr_ D 0xa0
> GLIBC_2.4 _IO_2_1_stdin_ D 0xa0
> diff --git a/sysdeps/unix/sysv/linux/arm/le/libc.abilist b/sysdeps/unix/sysv/linux/arm/le/libc.abilist
> index d2add4fb49..19f44510ee 100644
> --- a/sysdeps/unix/sysv/linux/arm/le/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/arm/le/libc.abilist
> @@ -493,6 +493,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 _Exit F
> GLIBC_2.4 _IO_2_1_stderr_ D 0xa0
> GLIBC_2.4 _IO_2_1_stdin_ D 0xa0
> diff --git a/sysdeps/unix/sysv/linux/csky/libc.abilist b/sysdeps/unix/sysv/linux/csky/libc.abilist
> index 355d72a30c..fce3ee63ea 100644
> --- a/sysdeps/unix/sysv/linux/csky/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/csky/libc.abilist
> @@ -2652,3 +2652,5 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> diff --git a/sysdeps/unix/sysv/linux/hppa/libc.abilist b/sysdeps/unix/sysv/linux/hppa/libc.abilist
> index 3df39bb28c..307b77c25b 100644
> --- a/sysdeps/unix/sysv/linux/hppa/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/hppa/libc.abilist
> @@ -2601,6 +2601,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 __confstr_chk F
> GLIBC_2.4 __fgets_chk F
> GLIBC_2.4 __fgets_unlocked_chk F
> diff --git a/sysdeps/unix/sysv/linux/i386/libc.abilist b/sysdeps/unix/sysv/linux/i386/libc.abilist
> index c4da358f80..99f3b0dde1 100644
> --- a/sysdeps/unix/sysv/linux/i386/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/i386/libc.abilist
> @@ -2785,6 +2785,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 __confstr_chk F
> GLIBC_2.4 __fgets_chk F
> GLIBC_2.4 __fgets_unlocked_chk F
> diff --git a/sysdeps/unix/sysv/linux/ia64/libc.abilist b/sysdeps/unix/sysv/linux/ia64/libc.abilist
> index 241bac70ea..3fe18c1bea 100644
> --- a/sysdeps/unix/sysv/linux/ia64/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/ia64/libc.abilist
> @@ -2551,6 +2551,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 __confstr_chk F
> GLIBC_2.4 __fgets_chk F
> GLIBC_2.4 __fgets_unlocked_chk F
> diff --git a/sysdeps/unix/sysv/linux/m68k/coldfire/libc.abilist b/sysdeps/unix/sysv/linux/m68k/coldfire/libc.abilist
> index 78bf372b72..aee8cb576b 100644
> --- a/sysdeps/unix/sysv/linux/m68k/coldfire/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/m68k/coldfire/libc.abilist
> @@ -497,6 +497,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 _Exit F
> GLIBC_2.4 _IO_2_1_stderr_ D 0x98
> GLIBC_2.4 _IO_2_1_stdin_ D 0x98
> diff --git a/sysdeps/unix/sysv/linux/m68k/m680x0/libc.abilist b/sysdeps/unix/sysv/linux/m68k/m680x0/libc.abilist
> index 00df5c901f..9b204c5cf9 100644
> --- a/sysdeps/unix/sysv/linux/m68k/m680x0/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/m68k/m680x0/libc.abilist
> @@ -2728,6 +2728,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 __confstr_chk F
> GLIBC_2.4 __fgets_chk F
> GLIBC_2.4 __fgets_unlocked_chk F
> diff --git a/sysdeps/unix/sysv/linux/microblaze/be/libc.abilist b/sysdeps/unix/sysv/linux/microblaze/be/libc.abilist
> index e8118569c3..6ec81952e1 100644
> --- a/sysdeps/unix/sysv/linux/microblaze/be/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/microblaze/be/libc.abilist
> @@ -2701,3 +2701,5 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> diff --git a/sysdeps/unix/sysv/linux/microblaze/le/libc.abilist b/sysdeps/unix/sysv/linux/microblaze/le/libc.abilist
> index c0d2373e64..7b484a8419 100644
> --- a/sysdeps/unix/sysv/linux/microblaze/le/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/microblaze/le/libc.abilist
> @@ -2698,3 +2698,5 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> diff --git a/sysdeps/unix/sysv/linux/mips/mips32/fpu/libc.abilist b/sysdeps/unix/sysv/linux/mips/mips32/fpu/libc.abilist
> index 2d0fd04f54..26e4321f80 100644
> --- a/sysdeps/unix/sysv/linux/mips/mips32/fpu/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/mips/mips32/fpu/libc.abilist
> @@ -2693,6 +2693,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 __confstr_chk F
> GLIBC_2.4 __fgets_chk F
> GLIBC_2.4 __fgets_unlocked_chk F
> diff --git a/sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc.abilist b/sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc.abilist
> index e39ccfb312..2c5e66e2c0 100644
> --- a/sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/mips/mips32/nofpu/libc.abilist
> @@ -2691,6 +2691,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 __confstr_chk F
> GLIBC_2.4 __fgets_chk F
> GLIBC_2.4 __fgets_unlocked_chk F
> diff --git a/sysdeps/unix/sysv/linux/mips/mips64/n32/libc.abilist b/sysdeps/unix/sysv/linux/mips/mips64/n32/libc.abilist
> index 1e900f86e4..f2e0a4818a 100644
> --- a/sysdeps/unix/sysv/linux/mips/mips64/n32/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/mips/mips64/n32/libc.abilist
> @@ -2699,6 +2699,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 __confstr_chk F
> GLIBC_2.4 __fgets_chk F
> GLIBC_2.4 __fgets_unlocked_chk F
> diff --git a/sysdeps/unix/sysv/linux/mips/mips64/n64/libc.abilist b/sysdeps/unix/sysv/linux/mips/mips64/n64/libc.abilist
> index 9145ba7931..e0505441f6 100644
> --- a/sysdeps/unix/sysv/linux/mips/mips64/n64/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/mips/mips64/n64/libc.abilist
> @@ -2602,6 +2602,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 __confstr_chk F
> GLIBC_2.4 __fgets_chk F
> GLIBC_2.4 __fgets_unlocked_chk F
> diff --git a/sysdeps/unix/sysv/linux/nios2/libc.abilist b/sysdeps/unix/sysv/linux/nios2/libc.abilist
> index e95d60d926..d0260261e2 100644
> --- a/sysdeps/unix/sysv/linux/nios2/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/nios2/libc.abilist
> @@ -2740,3 +2740,5 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> diff --git a/sysdeps/unix/sysv/linux/or1k/libc.abilist b/sysdeps/unix/sysv/linux/or1k/libc.abilist
> index ca934e374b..506853ee25 100644
> --- a/sysdeps/unix/sysv/linux/or1k/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/or1k/libc.abilist
> @@ -2123,3 +2123,5 @@ GLIBC_2.35 wprintf F
> GLIBC_2.35 write F
> GLIBC_2.35 writev F
> GLIBC_2.35 wscanf F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> diff --git a/sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc.abilist b/sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc.abilist
> index 3820b9f235..0096aa7a97 100644
> --- a/sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/powerpc/powerpc32/fpu/libc.abilist
> @@ -2755,6 +2755,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 _IO_fprintf F
> GLIBC_2.4 _IO_printf F
> GLIBC_2.4 _IO_sprintf F
> diff --git a/sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/libc.abilist b/sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/libc.abilist
> index 464dc27fcd..5869941ec6 100644
> --- a/sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/powerpc/powerpc32/nofpu/libc.abilist
> @@ -2788,6 +2788,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 _IO_fprintf F
> GLIBC_2.4 _IO_printf F
> GLIBC_2.4 _IO_sprintf F
> diff --git a/sysdeps/unix/sysv/linux/powerpc/powerpc64/be/libc.abilist b/sysdeps/unix/sysv/linux/powerpc/powerpc64/be/libc.abilist
> index 2f7e58747f..dc6277f1d8 100644
> --- a/sysdeps/unix/sysv/linux/powerpc/powerpc64/be/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/powerpc/powerpc64/be/libc.abilist
> @@ -2510,6 +2510,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 _IO_fprintf F
> GLIBC_2.4 _IO_printf F
> GLIBC_2.4 _IO_sprintf F
> diff --git a/sysdeps/unix/sysv/linux/powerpc/powerpc64/le/libc.abilist b/sysdeps/unix/sysv/linux/powerpc/powerpc64/le/libc.abilist
> index 4f3043d913..fd8adceee6 100644
> --- a/sysdeps/unix/sysv/linux/powerpc/powerpc64/le/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/powerpc/powerpc64/le/libc.abilist
> @@ -2812,3 +2812,5 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> diff --git a/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist b/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist
> index 84b6ac815a..2f0b08fe30 100644
> --- a/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/riscv/rv32/libc.abilist
> @@ -2379,3 +2379,5 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> diff --git a/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist b/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist
> index 4d5c19c56a..ad49621c76 100644
> --- a/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/riscv/rv64/libc.abilist
> @@ -2579,3 +2579,5 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> diff --git a/sysdeps/unix/sysv/linux/s390/s390-32/libc.abilist b/sysdeps/unix/sysv/linux/s390/s390-32/libc.abilist
> index 7c5ee8d569..ab25e964f1 100644
> --- a/sysdeps/unix/sysv/linux/s390/s390-32/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/s390/s390-32/libc.abilist
> @@ -2753,6 +2753,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 _IO_fprintf F
> GLIBC_2.4 _IO_printf F
> GLIBC_2.4 _IO_sprintf F
> diff --git a/sysdeps/unix/sysv/linux/s390/s390-64/libc.abilist b/sysdeps/unix/sysv/linux/s390/s390-64/libc.abilist
> index 50de0b46cf..3851645690 100644
> --- a/sysdeps/unix/sysv/linux/s390/s390-64/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/s390/s390-64/libc.abilist
> @@ -2547,6 +2547,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 _IO_fprintf F
> GLIBC_2.4 _IO_printf F
> GLIBC_2.4 _IO_sprintf F
> diff --git a/sysdeps/unix/sysv/linux/sh/be/libc.abilist b/sysdeps/unix/sysv/linux/sh/be/libc.abilist
> index 66fba013ca..b606bad5dc 100644
> --- a/sysdeps/unix/sysv/linux/sh/be/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/sh/be/libc.abilist
> @@ -2608,6 +2608,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 __confstr_chk F
> GLIBC_2.4 __fgets_chk F
> GLIBC_2.4 __fgets_unlocked_chk F
> diff --git a/sysdeps/unix/sysv/linux/sh/le/libc.abilist b/sysdeps/unix/sysv/linux/sh/le/libc.abilist
> index 38703f8aa0..9b7e03a290 100644
> --- a/sysdeps/unix/sysv/linux/sh/le/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/sh/le/libc.abilist
> @@ -2605,6 +2605,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 __confstr_chk F
> GLIBC_2.4 __fgets_chk F
> GLIBC_2.4 __fgets_unlocked_chk F
> diff --git a/sysdeps/unix/sysv/linux/sparc/sparc32/libc.abilist b/sysdeps/unix/sysv/linux/sparc/sparc32/libc.abilist
> index 6df55eb765..04651211d7 100644
> --- a/sysdeps/unix/sysv/linux/sparc/sparc32/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/sparc/sparc32/libc.abilist
> @@ -2748,6 +2748,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 _IO_fprintf F
> GLIBC_2.4 _IO_printf F
> GLIBC_2.4 _IO_sprintf F
> diff --git a/sysdeps/unix/sysv/linux/sparc/sparc64/libc.abilist b/sysdeps/unix/sysv/linux/sparc/sparc64/libc.abilist
> index b90569d881..5ce2c90903 100644
> --- a/sysdeps/unix/sysv/linux/sparc/sparc64/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/sparc/sparc64/libc.abilist
> @@ -2574,6 +2574,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 __confstr_chk F
> GLIBC_2.4 __fgets_chk F
> GLIBC_2.4 __fgets_unlocked_chk F
> diff --git a/sysdeps/unix/sysv/linux/x86_64/64/libc.abilist b/sysdeps/unix/sysv/linux/x86_64/64/libc.abilist
> index e88b0f101f..85a7fb3152 100644
> --- a/sysdeps/unix/sysv/linux/x86_64/64/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/x86_64/64/libc.abilist
> @@ -2525,6 +2525,8 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> GLIBC_2.4 __confstr_chk F
> GLIBC_2.4 __fgets_chk F
> GLIBC_2.4 __fgets_unlocked_chk F
> diff --git a/sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist b/sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist
> index e0755272eb..975e78eaa5 100644
> --- a/sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist
> +++ b/sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist
> @@ -2631,3 +2631,5 @@ GLIBC_2.35 __memcmpeq F
> GLIBC_2.35 _dl_find_object F
> GLIBC_2.35 epoll_pwait2 F
> GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
> +GLIBC_2.36 c8rtomb F
> +GLIBC_2.36 mbrtoc8 F
> diff --git a/wcsmbs/Makefile b/wcsmbs/Makefile
> index df9a85f4a9..bda281ad70 100644
> --- a/wcsmbs/Makefile
> +++ b/wcsmbs/Makefile
> @@ -42,7 +42,7 @@ routines := wcscat wcschr wcscmp wcscpy wcscspn wcsdup wcslen wcsncat \
> wcsmbsload mbsrtowcs_l \
> isoc99_wscanf isoc99_vwscanf isoc99_fwscanf isoc99_vfwscanf \
> isoc99_swscanf isoc99_vswscanf \
> - mbrtoc16 c16rtomb mbrtoc32 c32rtomb
> + mbrtoc8 c8rtomb mbrtoc16 c16rtomb mbrtoc32 c32rtomb
>
> strop-tests := wcscmp wcsncmp wmemcmp wcslen wcschr wcsrchr wcscpy wcsnlen \
> wcpcpy wcsncpy wcpncpy wcscat wcsncat wcschrnul wcsspn wcspbrk \
> diff --git a/wcsmbs/Versions b/wcsmbs/Versions
> index 0b31c1b940..ec28acfb73 100644
> --- a/wcsmbs/Versions
> +++ b/wcsmbs/Versions
> @@ -49,4 +49,7 @@ libc {
> wcstof32; wcstof64; wcstof32x;
> wcstof32_l; wcstof64_l; wcstof32x_l;
> }
> + GLIBC_2.36 {
> + c8rtomb; mbrtoc8;
> + }
> }
> diff --git a/wcsmbs/c8rtomb.c b/wcsmbs/c8rtomb.c
> new file mode 100644
> index 0000000000..8a5ffeab23
> --- /dev/null
> +++ b/wcsmbs/c8rtomb.c
> @@ -0,0 +1,132 @@
> +/* UTF-8 to multibyte conversion.
> + Copyright (C) 2022 Free Software Foundation, Inc.
> + This file is part of the GNU C Library.
> +
> + The GNU C Library is free software; you can redistribute it and/or
> + modify it under the terms of the GNU Lesser General Public
> + License as published by the Free Software Foundation; either
> + version 2.1 of the License, or (at your option) any later version.
> +
> + The GNU C Library is distributed in the hope that it will be useful,
> + but WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + Lesser General Public License for more details.
> +
> + You should have received a copy of the GNU Lesser General Public
> + License along with the GNU C Library; if not, see
> + <https://www.gnu.org/licenses/>. */
> +
> +#include <errno.h>
> +#include <uchar.h>
> +#include <wchar.h>
> +
> +
> +/* This is the private state used if PS is NULL. */
> +static mbstate_t state;
Although it was done for other convertion interfaces, I wonder if we should
keep supporting this mt-unsafe usage for newer ones. It was not clear from
n2653 how the function should behave for PS equal to NULL, so I think it
would be conformant to either define PS as nonull or return -1/EINVAL.
> +
> +size_t
> +c8rtomb (char *s, char8_t c8, mbstate_t *ps)
> +{
> + /* This implementation depends on the converter invoked by wcrtomb not
> + needing to retain state in either the top most bit of ps->__count or
> + in ps->__value between invocations. This implementation uses the
> + top most bit of ps->__count to indicate that trailing code units are
> + expected and uses ps->__value to store previously seen code units. */
> +
> + wchar_t wc;
> +
> + if (ps == NULL)
> + ps = &state;
> +
> + if (s == NULL)
> + {
> + /* if 's' is a null pointer, behave as if u8'\0' was passed as 'c8'. If
> + this occurs for an incomplete code unit sequence, then an error will
> + be reported below. */
> + c8 = u8""[0];
> + }
> +
> + if (! (ps->__count & 0x80000000))
> + {
> + /* Initial state. */
> + if ((c8 >= 0x80 && c8 <= 0xC1) || c8 >= 0xF5)
> + {
> + /* An invalid lead code unit. */
> + __set_errno (EILSEQ);
> + return -1;
> + }
> + if (c8 >= 0xC2)
> + {
> + /* A valid lead code unit. */
> + ps->__count |= 0x80000000;
> + ps->__value.__wchb[0] = c8;
> + ps->__value.__wchb[3] = 1;
Indentation seems off here (you used tabs and this one was done with whitespaces).
> + return 0;
> + }
> + /* A single byte (ASCII) code unit. */
> + wc = c8;
> + }
> + else
> + {
> + char8_t cu1 = ps->__value.__wchb[0];
> + if (ps->__value.__wchb[3] == 1)
> + {
> + /* A single lead code unit was previously seen. */
> + if ((c8 < 0x80 || c8 > 0xBF) ||
> + (cu1 == 0xE0 && c8 < 0xA0) ||
> + (cu1 == 0xED && c8 > 0x9F) ||
> + (cu1 == 0xF0 && c8 < 0x90) ||
> + (cu1 == 0xF4 && c8 > 0x8F))
I think the GNU style place the comparison on a newer line:
if ((c8 < 0x80 || c8 > 0xBF)
|| (cu1 == 0xE0 && c8 < 0xA0)
[...]
> + {
> + /* An invalid second code unit. */
> + __set_errno (EILSEQ);
> + return -1;
> + }
> + if (cu1 >= 0xE0)
> + {
> + /* A three or four code unit sequence. */
> + ps->__value.__wchb[1] = c8;
> + ++ps->__value.__wchb[3];
> + return 0;
> + }
> + wc = ((cu1 & 0x1F) << 6) +
> + (c8 & 0x3F);
Same as before for the addition:
wc = ((cu1 & 0x1F) << 64)
+ (c8 & 0x3F);
> + }
> + else
> + {
> + char8_t cu2 = ps->__value.__wchb[1];
> + /* A three or four byte code unit sequence. */
> + if (c8 < 0x80 || c8 > 0xBF)
> + {
> + /* An invalid third or fourth code unit. */
> + __set_errno (EILSEQ);
> + return -1;
> + }
> + if (ps->__value.__wchb[3] == 2 && cu1 >= 0xF0)
> + {
> + /* A four code unit sequence. */
> + ps->__value.__wchb[2] = c8;
> + ++ps->__value.__wchb[3];
> + return 0;
> + }
> + if (cu1 < 0xF0)
> + {
> + wc = ((cu1 & 0x0F) << 12) +
> + ((cu2 & 0x3F) << 6) +
> + (c8 & 0x3F);
Same as before.
> + }
> + else
> + {
> + char8_t cu3 = ps->__value.__wchb[2];
> + wc = ((cu1 & 0x07) << 18) +
> + ((cu2 & 0x3F) << 12) +
> + ((cu3 & 0x3F) << 6) +
> + (c8 & 0x3F);
Same as before.
> + }
> + }
> + ps->__count &= 0x7fffffff;
> + ps->__value.__wch = 0;
> + }
> +
> + return wcrtomb (s, wc, ps);
> +}
The UTF-8 decoders seems to follow the RFC 3629 ABNF definition.
> diff --git a/wcsmbs/mbrtoc8.c b/wcsmbs/mbrtoc8.c
> new file mode 100644
> index 0000000000..8ca457088d
> --- /dev/null
> +++ b/wcsmbs/mbrtoc8.c
> @@ -0,0 +1,126 @@
> +/* Multibyte to UTF-8 conversion.
> + Copyright (C) 2022 Free Software Foundation, Inc.
> + This file is part of the GNU C Library.
> +
> + The GNU C Library is free software; you can redistribute it and/or
> + modify it under the terms of the GNU Lesser General Public
> + License as published by the Free Software Foundation; either
> + version 2.1 of the License, or (at your option) any later version.
> +
> + The GNU C Library is distributed in the hope that it will be useful,
> + but WITHOUT ANY WARRANTY; without even the implied warranty of
> + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + Lesser General Public License for more details.
> +
> + You should have received a copy of the GNU Lesser General Public
> + License along with the GNU C Library; if not, see
> + <https://www.gnu.org/licenses/>. */
> +
> +#include <assert.h>
> +#include <dlfcn.h>
> +#include <errno.h>
> +#include <gconv.h>
> +#include <uchar.h>
> +#include <wcsmbsload.h>
> +
> +#include <sysdep.h>
> +
> +#ifndef EILSEQ
> +# define EILSEQ EINVAL
> +#endif
> +
> +
> +/* This is the private state used if PS is NULL. */
> +static mbstate_t state;
> +
> +size_t
> +mbrtoc8 (char8_t *pc8, const char *s, size_t n, mbstate_t *ps)
> +{
> + /* This implementation depends on the converter invoked by mbrtowc() not
> + needing to retain state in either the top most bit of ps->__count or
> + in ps->__value between invocations. This implementation uses the
> + top most bit of ps->__count to indicate that trailing code units are
> + yet to be written and uses ps->__value to store those code units. */
> +
> + if (ps == NULL)
> + ps = &state;
> +
> + /* If state indicates that trailing code units are yet to be written, write
> + those first regardless of whether 's' is a null pointer. */
> + if (ps->__count & 0x80000000)
> + {
> + /* ps->__value.__wchb[3] stores the index of the next code unit to
> + write. Code units are stored in reverse order. */
> + size_t i = ps->__value.__wchb[3];
> + if (pc8 != NULL)
> + {
> + *pc8 = ps->__value.__wchb[i];
> + }
> + if (i == 0)
> + {
> + ps->__count &= 0x7fffffff;
> + ps->__value.__wch = 0;
> + }
> + else
> + --ps->__value.__wchb[3];
> + return -3;
> + }
> +
> + if (s == NULL)
> + {
> + /* if 's' is a null pointer, behave as if a null pointer was passed for
> + 'pc8', an empty string was passed for 's', and 1 passed for 'n'. */
> + pc8 = NULL;
> + s = "";
> + n = 1;
> + }
> +
> + wchar_t wc;
> + size_t result;
> +
> + result = mbrtowc(&wc, s, n, ps);
Missing space before '('.
> + if (result <= n)
> + {
> + if (wc <= 0x7F)
> + {
> + if (pc8 != NULL)
> + *pc8 = wc;
> + }
> + else if (wc <= 0x7FF)
> + {
> + if (pc8 != NULL)
> + *pc8 = 0xC0 + ((wc >> 6) & 0x1F);
> + ps->__value.__wchb[0] = 0x80 + (wc & 0x3F);
> + ps->__value.__wchb[3] = 0;
> + ps->__count |= 0x80000000;
> + }
> + else if (wc <= 0xFFFF)
> + {
> + if (pc8 != NULL)
> + *pc8 = 0xE0 + ((wc >> 12) & 0x0F);
> + ps->__value.__wchb[1] = 0x80 + ((wc >> 6) & 0x3F);
> + ps->__value.__wchb[0] = 0x80 + (wc & 0x3F);
> + ps->__value.__wchb[3] = 1;
> + ps->__count |= 0x80000000;
> + }
> + else if (wc <= 0x10FFFF)
> + {
> + if (pc8 != NULL)
> + *pc8 = 0xF0 + ((wc >> 18) & 0x07);
> + ps->__value.__wchb[2] = 0x80 + ((wc >> 12) & 0x3F);
> + ps->__value.__wchb[1] = 0x80 + ((wc >> 6) & 0x3F);
> + ps->__value.__wchb[0] = 0x80 + (wc & 0x3F);
> + ps->__value.__wchb[3] = 2;
> + ps->__count |= 0x80000000;
> + }
> + }
> + if (result == 0 && wc != 0)
> + {
> + /* mbrtowc() never returns -3. When a MB sequence converts to multiple
> + WCs, no input is consumed when writing the subsequent WCs resulting
> + in a result of 0 even if a null character wasn't written. */
> + result = -3;
> + }
> +
> + return result;
> +}
Ok.
> diff --git a/wcsmbs/uchar.h b/wcsmbs/uchar.h
> index 051cdcbeb5..c37e8619a0 100644
> --- a/wcsmbs/uchar.h
> +++ b/wcsmbs/uchar.h
> @@ -31,6 +31,13 @@
> #include <bits/types.h>
> #include <bits/types/mbstate_t.h>
>
> +/* Declare the C2x char8_t typedef in C2x modes, but only if the C++
> + __cpp_char8_t feature test macro is not defined. */
> +#if __GLIBC_USE (ISOC2X) && !defined __cpp_char8_t
> +/* Define the 8-bit character type. */
> +typedef unsigned char char8_t;
> +#endif
> +
> #ifndef __USE_ISOCXX11
> /* Define the 16-bit and 32-bit character types. */
> typedef __uint_least16_t char16_t;
> @@ -40,6 +47,20 @@ typedef __uint_least32_t char32_t;
>
> __BEGIN_DECLS
>
> +/* Declare the C2x mbrtoc8() and c8rtomb() functions in C2x modes or if
> + the C++ __cpp_char8_t feature test macro is defined. */
> +#if __GLIBC_USE (ISOC2X) || defined __cpp_char8_t
> +/* Write char8_t representation of multibyte character pointed
> + to by S to PC8. */
> +extern size_t mbrtoc8 (char8_t *__restrict __pc8,
> + const char *__restrict __s, size_t __n,
> + mbstate_t *__restrict __p) __THROW;
> +
> +/* Write multibyte representation of char8_t C8 to S. */
> +extern size_t c8rtomb (char *__restrict __s, char8_t __c8,
> + mbstate_t *__restrict __ps) __THROW;
> +#endif
> +
> /* Write char16_t representation of multibyte character pointed
> to by S to PC16. */
> extern size_t mbrtoc16 (char16_t *__restrict __pc16,
Ok.
On Tue, 17 May 2022, Adhemerval Zanella wrote:
> > +/* This is the private state used if PS is NULL. */
> > +static mbstate_t state;
>
> Although it was done for other convertion interfaces, I wonder if we should
> keep supporting this mt-unsafe usage for newer ones. It was not clear from
In C23 it's implementation-defined whether the internal state for such
functions has static or thread storage duration (see the general
introduction to the uchar.h functions).
On 17/05/2022 15:12, Joseph Myers wrote:
> On Tue, 17 May 2022, Adhemerval Zanella wrote:
>
>>> +/* This is the private state used if PS is NULL. */
>>> +static mbstate_t state;
>>
>> Although it was done for other convertion interfaces, I wonder if we should
>> keep supporting this mt-unsafe usage for newer ones. It was not clear from
>
> In C23 it's implementation-defined whether the internal state for such
> functions has static or thread storage duration (see the general
> introduction to the uchar.h functions).
>
Right, so glibc still need to support either mode.
* Adhemerval Zanella via Libc-alpha:
> On 17/05/2022 15:12, Joseph Myers wrote:
>> On Tue, 17 May 2022, Adhemerval Zanella wrote:
>>
>>>> +/* This is the private state used if PS is NULL. */
>>>> +static mbstate_t state;
>>>
>>> Although it was done for other convertion interfaces, I wonder if we should
>>> keep supporting this mt-unsafe usage for newer ones. It was not clear from
>>
>> In C23 it's implementation-defined whether the internal state for such
>> functions has static or thread storage duration (see the general
>> introduction to the uchar.h functions).
>>
>
> Right, so glibc still need to support either mode.
No, I think we can and should switch. Maybe with new symbol versions
(but the same implementation) if we want to play it conservative.
The intent in POSIX and C has been for a long time that thread-local
state is permitted for these functions, only the wording did not
technically allow it. The phrase was “not required to avoid data
races”. The problem is that it's possible to tell the difference
without data races (with external synchronization). Many libcs already
use thread-local state for these functions, without any apparent ill
effects.
Thanks,
Florian
On 5/17/22 5:33 PM, Florian Weimer wrote:
> * Adhemerval Zanella via Libc-alpha:
>
>> On 17/05/2022 15:12, Joseph Myers wrote:
>>> On Tue, 17 May 2022, Adhemerval Zanella wrote:
>>>
>>>>> +/* This is the private state used if PS is NULL. */
>>>>> +static mbstate_t state;
>>>> Although it was done for other convertion interfaces, I wonder if we should
>>>> keep supporting this mt-unsafe usage for newer ones. It was not clear from
>>> In C23 it's implementation-defined whether the internal state for such
>>> functions has static or thread storage duration (see the general
>>> introduction to the uchar.h functions).
>>>
>> Right, so glibc still need to support either mode.
> No, I think we can and should switch. Maybe with new symbol versions
> (but the same implementation) if we want to play it conservative.
>
> The intent in POSIX and C has been for a long time that thread-local
> state is permitted for these functions, only the wording did not
> technically allow it. The phrase was “not required to avoid data
> races”. The problem is that it's possible to tell the difference
> without data races (with external synchronization). Many libcs already
> use thread-local state for these functions, without any apparent ill
> effects.
I would err on the side of maintaining consistency across these
functions for now and then transition them all at once if there is a
desire to do so.
Tom.
>
> Thanks,
> Florian
>
On 18/05/2022 12:32, Tom Honermann wrote:
> On 5/17/22 5:33 PM, Florian Weimer wrote:
>> * Adhemerval Zanella via Libc-alpha:
>>
>>> On 17/05/2022 15:12, Joseph Myers wrote:
>>>> On Tue, 17 May 2022, Adhemerval Zanella wrote:
>>>>
>>>>>> +/* This is the private state used if PS is NULL. */
>>>>>> +static mbstate_t state;
>>>>> Although it was done for other convertion interfaces, I wonder if we should
>>>>> keep supporting this mt-unsafe usage for newer ones. It was not clear from
>>>> In C23 it's implementation-defined whether the internal state for such
>>>> functions has static or thread storage duration (see the general
>>>> introduction to the uchar.h functions).
>>>>
>>> Right, so glibc still need to support either mode.
>> No, I think we can and should switch. Maybe with new symbol versions
>> (but the same implementation) if we want to play it conservative.
>>
>> The intent in POSIX and C has been for a long time that thread-local
>> state is permitted for these functions, only the wording did not
>> technically allow it. The phrase was “not required to avoid data
>> races”. The problem is that it's possible to tell the difference
>> without data races (with external synchronization). Many libcs already
>> use thread-local state for these functions, without any apparent ill
>> effects.
>
> I would err on the side of maintaining consistency across these functions for now and then transition them all at once if there is a desire to do so.
Do we need to keep distinct states for each one or can use the a shared
one?
On 5/18/22 12:17 PM, Adhemerval Zanella wrote:
>
> On 18/05/2022 12:32, Tom Honermann wrote:
>> On 5/17/22 5:33 PM, Florian Weimer wrote:
>>> * Adhemerval Zanella via Libc-alpha:
>>>
>>>> On 17/05/2022 15:12, Joseph Myers wrote:
>>>>> On Tue, 17 May 2022, Adhemerval Zanella wrote:
>>>>>
>>>>>>> +/* This is the private state used if PS is NULL. */
>>>>>>> +static mbstate_t state;
>>>>>> Although it was done for other convertion interfaces, I wonder if we should
>>>>>> keep supporting this mt-unsafe usage for newer ones. It was not clear from
>>>>> In C23 it's implementation-defined whether the internal state for such
>>>>> functions has static or thread storage duration (see the general
>>>>> introduction to the uchar.h functions).
>>>>>
>>>> Right, so glibc still need to support either mode.
>>> No, I think we can and should switch. Maybe with new symbol versions
>>> (but the same implementation) if we want to play it conservative.
>>>
>>> The intent in POSIX and C has been for a long time that thread-local
>>> state is permitted for these functions, only the wording did not
>>> technically allow it. The phrase was “not required to avoid data
>>> races”. The problem is that it's possible to tell the difference
>>> without data races (with external synchronization). Many libcs already
>>> use thread-local state for these functions, without any apparent ill
>>> effects.
>> I would err on the side of maintaining consistency across these functions for now and then transition them all at once if there is a desire to do so.
> Do we need to keep distinct states for each one or can use the a shared
> one?
Each is required to maintain its own state. The relevant wording from
C17 7.28.1 (Restartable multibyte/wide character conversion functions)
paragraph 1 states:
These functions have a parameter, ps, of type pointer to mbstate_t that
points to an object that can completely describe the current conversion
state of the associated multibyte character sequence, which the
functions alter as necessary. *If ps is a null pointer, each function
uses its own internal **mbstate_t**object instead, which is initialized
at program startup to the initial conversion state; the****functions are
not required to avoid data races with other calls to the same function
in this case. The **implementation behaves as if no library function
calls these functions with a null pointer for ps.*
Tom.
On 18/05/2022 14:26, Tom Honermann wrote:
> On 5/18/22 12:17 PM, Adhemerval Zanella wrote:
>> On 18/05/2022 12:32, Tom Honermann wrote:
>>> On 5/17/22 5:33 PM, Florian Weimer wrote:
>>>> * Adhemerval Zanella via Libc-alpha:
>>>>
>>>>> On 17/05/2022 15:12, Joseph Myers wrote:
>>>>>> On Tue, 17 May 2022, Adhemerval Zanella wrote:
>>>>>>
>>>>>>>> +/* This is the private state used if PS is NULL. */
>>>>>>>> +static mbstate_t state;
>>>>>>> Although it was done for other convertion interfaces, I wonder if we should
>>>>>>> keep supporting this mt-unsafe usage for newer ones. It was not clear from
>>>>>> In C23 it's implementation-defined whether the internal state for such
>>>>>> functions has static or thread storage duration (see the general
>>>>>> introduction to the uchar.h functions).
>>>>>>
>>>>> Right, so glibc still need to support either mode.
>>>> No, I think we can and should switch. Maybe with new symbol versions
>>>> (but the same implementation) if we want to play it conservative.
>>>>
>>>> The intent in POSIX and C has been for a long time that thread-local
>>>> state is permitted for these functions, only the wording did not
>>>> technically allow it. The phrase was “not required to avoid data
>>>> races”. The problem is that it's possible to tell the difference
>>>> without data races (with external synchronization). Many libcs already
>>>> use thread-local state for these functions, without any apparent ill
>>>> effects.
>>> I would err on the side of maintaining consistency across these functions for now and then transition them all at once if there is a desire to do so.
>> Do we need to keep distinct states for each one or can use the a shared
>> one?
>
> Each is required to maintain its own state. The relevant wording from C17 7.28.1 (Restartable multibyte/wide character conversion functions) paragraph 1 states:
>
> These functions have a parameter, ps, of type pointer to mbstate_t that points to an object that can completely describe the current conversion state of the associated multibyte character sequence, which the functions alter as necessary. *If ps is a null pointer, each function uses its own internal **mbstate_t**object instead, which is initialized at program startup to the initial conversion state; the****functions are not required to avoid data races with other calls to the same function in this case. The **implementation behaves as if no library function calls these functions with a null pointer for ps.*
Yeah, I forgot that I just read this very sentence yesterday reviewing you patch.
So it it currently 88 bytes per thread (104 considering the mbrtoc8, c8rtomb)
per thread state if we make it thread-local. We can maybe lazy allocate the
whole wcsmbs state, so we just keep a pointer per thread.
We already have the internal scheme to allocate ion tcb (tls-internal.h) so
it does not exhaust static tls allocation.
* Adhemerval Zanella:
> Do we need to keep distinct states for each one or can use the a shared
> one?
I think per-function state is required. We can allocate it on demand
because these functions may return ENOMEM.
Thanks,
Florian
On 18/05/2022 14:40, Florian Weimer wrote:
> * Adhemerval Zanella:
>
>> Do we need to keep distinct states for each one or can use the a shared
>> one?
>
> I think per-function state is required. We can allocate it on demand
> because these functions may return ENOMEM.
That was I suggested to Tom as well. In any case I think this is orthogonal
to the mbrtoc8/c8rtomb patch.
commit d29ff21403913f7014fe3deebe65c9294e6c1d44
Author: Tom Honermann <tom@honermann.net>
Date: Sun Feb 27 10:28:58 2022 -0500
Implement mbrtoc8(), c8rtomb(), and the char8_t typedef.
This change provides implementations for the mbrtoc8 and c8rtomb
functions adopted for C++20 via WG21 P0482R6 and for C2X via WG14
N2653. It also provides the char8_t typedef from WG14 N2653.
The mbrtoc8 and c8rtomb functions are declared in uchar.h in C2X
mode or when the _GNU_SOURCE macro or C++20 __cpp_char8_t feature
test macro is defined.
The char8_t typedef is declared in uchar.h in C2X mode or when the
_GNU_SOURCE macro is defined and the C++20 __cpp_char8_t feature
test macro is not defined (if __cpp_char8_t is defined, then char8_t
is a builtin type).
@@ -9,6 +9,15 @@ Version 2.36
Major new features:
+* Support for the mbrtoc8 and c8rtomb multibyte/UTF-8 character conversion
+ functions has been added per the ISO C2X N2653 and C++20 P0482R6 proposals.
+ Support for the char8_t typedef has been added per the ISO C2X N2653
+ proposal. The functions are declared in uchar.h in C2X mode or when the
+ _GNU_SOURCE macro or C++20 __cpp_char8_t feature test macro is defined.
+ The char8_t typedef is declared in uchar.h in C2X mode or when the
+ _GNU_SOURCE macro is defined and the C++20 __cpp_char8_t feature test macro
+ is not defined (if __cpp_char8_t is defined, then char8_t is a builtin type).
+
[Add new features here]
Deprecated and removed features, and other changes affecting compatibility:
@@ -2289,6 +2289,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 close_range F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 __confstr_chk F
GLIBC_2.4 __fgets_chk F
GLIBC_2.4 __fgets_unlocked_chk F
@@ -2616,3 +2616,5 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
@@ -2713,6 +2713,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 _IO_fprintf F
GLIBC_2.4 _IO_printf F
GLIBC_2.4 _IO_sprintf F
@@ -2377,3 +2377,5 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
@@ -496,6 +496,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 _Exit F
GLIBC_2.4 _IO_2_1_stderr_ D 0xa0
GLIBC_2.4 _IO_2_1_stdin_ D 0xa0
@@ -493,6 +493,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 _Exit F
GLIBC_2.4 _IO_2_1_stderr_ D 0xa0
GLIBC_2.4 _IO_2_1_stdin_ D 0xa0
@@ -2652,3 +2652,5 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
@@ -2601,6 +2601,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 __confstr_chk F
GLIBC_2.4 __fgets_chk F
GLIBC_2.4 __fgets_unlocked_chk F
@@ -2785,6 +2785,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 __confstr_chk F
GLIBC_2.4 __fgets_chk F
GLIBC_2.4 __fgets_unlocked_chk F
@@ -2551,6 +2551,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 __confstr_chk F
GLIBC_2.4 __fgets_chk F
GLIBC_2.4 __fgets_unlocked_chk F
@@ -497,6 +497,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 _Exit F
GLIBC_2.4 _IO_2_1_stderr_ D 0x98
GLIBC_2.4 _IO_2_1_stdin_ D 0x98
@@ -2728,6 +2728,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 __confstr_chk F
GLIBC_2.4 __fgets_chk F
GLIBC_2.4 __fgets_unlocked_chk F
@@ -2701,3 +2701,5 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
@@ -2698,3 +2698,5 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
@@ -2693,6 +2693,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 __confstr_chk F
GLIBC_2.4 __fgets_chk F
GLIBC_2.4 __fgets_unlocked_chk F
@@ -2691,6 +2691,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 __confstr_chk F
GLIBC_2.4 __fgets_chk F
GLIBC_2.4 __fgets_unlocked_chk F
@@ -2699,6 +2699,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 __confstr_chk F
GLIBC_2.4 __fgets_chk F
GLIBC_2.4 __fgets_unlocked_chk F
@@ -2602,6 +2602,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 __confstr_chk F
GLIBC_2.4 __fgets_chk F
GLIBC_2.4 __fgets_unlocked_chk F
@@ -2740,3 +2740,5 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
@@ -2123,3 +2123,5 @@ GLIBC_2.35 wprintf F
GLIBC_2.35 write F
GLIBC_2.35 writev F
GLIBC_2.35 wscanf F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
@@ -2755,6 +2755,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 _IO_fprintf F
GLIBC_2.4 _IO_printf F
GLIBC_2.4 _IO_sprintf F
@@ -2788,6 +2788,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 _IO_fprintf F
GLIBC_2.4 _IO_printf F
GLIBC_2.4 _IO_sprintf F
@@ -2510,6 +2510,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 _IO_fprintf F
GLIBC_2.4 _IO_printf F
GLIBC_2.4 _IO_sprintf F
@@ -2812,3 +2812,5 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
@@ -2379,3 +2379,5 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
@@ -2579,3 +2579,5 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
@@ -2753,6 +2753,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 _IO_fprintf F
GLIBC_2.4 _IO_printf F
GLIBC_2.4 _IO_sprintf F
@@ -2547,6 +2547,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 _IO_fprintf F
GLIBC_2.4 _IO_printf F
GLIBC_2.4 _IO_sprintf F
@@ -2608,6 +2608,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 __confstr_chk F
GLIBC_2.4 __fgets_chk F
GLIBC_2.4 __fgets_unlocked_chk F
@@ -2605,6 +2605,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 __confstr_chk F
GLIBC_2.4 __fgets_chk F
GLIBC_2.4 __fgets_unlocked_chk F
@@ -2748,6 +2748,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 _IO_fprintf F
GLIBC_2.4 _IO_printf F
GLIBC_2.4 _IO_sprintf F
@@ -2574,6 +2574,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 __confstr_chk F
GLIBC_2.4 __fgets_chk F
GLIBC_2.4 __fgets_unlocked_chk F
@@ -2525,6 +2525,8 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
GLIBC_2.4 __confstr_chk F
GLIBC_2.4 __fgets_chk F
GLIBC_2.4 __fgets_unlocked_chk F
@@ -2631,3 +2631,5 @@ GLIBC_2.35 __memcmpeq F
GLIBC_2.35 _dl_find_object F
GLIBC_2.35 epoll_pwait2 F
GLIBC_2.35 posix_spawn_file_actions_addtcsetpgrp_np F
+GLIBC_2.36 c8rtomb F
+GLIBC_2.36 mbrtoc8 F
@@ -42,7 +42,7 @@ routines := wcscat wcschr wcscmp wcscpy wcscspn wcsdup wcslen wcsncat \
wcsmbsload mbsrtowcs_l \
isoc99_wscanf isoc99_vwscanf isoc99_fwscanf isoc99_vfwscanf \
isoc99_swscanf isoc99_vswscanf \
- mbrtoc16 c16rtomb mbrtoc32 c32rtomb
+ mbrtoc8 c8rtomb mbrtoc16 c16rtomb mbrtoc32 c32rtomb
strop-tests := wcscmp wcsncmp wmemcmp wcslen wcschr wcsrchr wcscpy wcsnlen \
wcpcpy wcsncpy wcpncpy wcscat wcsncat wcschrnul wcsspn wcspbrk \
@@ -49,4 +49,7 @@ libc {
wcstof32; wcstof64; wcstof32x;
wcstof32_l; wcstof64_l; wcstof32x_l;
}
+ GLIBC_2.36 {
+ c8rtomb; mbrtoc8;
+ }
}
new file mode 100644
@@ -0,0 +1,132 @@
+/* UTF-8 to multibyte conversion.
+ Copyright (C) 2022 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <https://www.gnu.org/licenses/>. */
+
+#include <errno.h>
+#include <uchar.h>
+#include <wchar.h>
+
+
+/* This is the private state used if PS is NULL. */
+static mbstate_t state;
+
+size_t
+c8rtomb (char *s, char8_t c8, mbstate_t *ps)
+{
+ /* This implementation depends on the converter invoked by wcrtomb not
+ needing to retain state in either the top most bit of ps->__count or
+ in ps->__value between invocations. This implementation uses the
+ top most bit of ps->__count to indicate that trailing code units are
+ expected and uses ps->__value to store previously seen code units. */
+
+ wchar_t wc;
+
+ if (ps == NULL)
+ ps = &state;
+
+ if (s == NULL)
+ {
+ /* if 's' is a null pointer, behave as if u8'\0' was passed as 'c8'. If
+ this occurs for an incomplete code unit sequence, then an error will
+ be reported below. */
+ c8 = u8""[0];
+ }
+
+ if (! (ps->__count & 0x80000000))
+ {
+ /* Initial state. */
+ if ((c8 >= 0x80 && c8 <= 0xC1) || c8 >= 0xF5)
+ {
+ /* An invalid lead code unit. */
+ __set_errno (EILSEQ);
+ return -1;
+ }
+ if (c8 >= 0xC2)
+ {
+ /* A valid lead code unit. */
+ ps->__count |= 0x80000000;
+ ps->__value.__wchb[0] = c8;
+ ps->__value.__wchb[3] = 1;
+ return 0;
+ }
+ /* A single byte (ASCII) code unit. */
+ wc = c8;
+ }
+ else
+ {
+ char8_t cu1 = ps->__value.__wchb[0];
+ if (ps->__value.__wchb[3] == 1)
+ {
+ /* A single lead code unit was previously seen. */
+ if ((c8 < 0x80 || c8 > 0xBF) ||
+ (cu1 == 0xE0 && c8 < 0xA0) ||
+ (cu1 == 0xED && c8 > 0x9F) ||
+ (cu1 == 0xF0 && c8 < 0x90) ||
+ (cu1 == 0xF4 && c8 > 0x8F))
+ {
+ /* An invalid second code unit. */
+ __set_errno (EILSEQ);
+ return -1;
+ }
+ if (cu1 >= 0xE0)
+ {
+ /* A three or four code unit sequence. */
+ ps->__value.__wchb[1] = c8;
+ ++ps->__value.__wchb[3];
+ return 0;
+ }
+ wc = ((cu1 & 0x1F) << 6) +
+ (c8 & 0x3F);
+ }
+ else
+ {
+ char8_t cu2 = ps->__value.__wchb[1];
+ /* A three or four byte code unit sequence. */
+ if (c8 < 0x80 || c8 > 0xBF)
+ {
+ /* An invalid third or fourth code unit. */
+ __set_errno (EILSEQ);
+ return -1;
+ }
+ if (ps->__value.__wchb[3] == 2 && cu1 >= 0xF0)
+ {
+ /* A four code unit sequence. */
+ ps->__value.__wchb[2] = c8;
+ ++ps->__value.__wchb[3];
+ return 0;
+ }
+ if (cu1 < 0xF0)
+ {
+ wc = ((cu1 & 0x0F) << 12) +
+ ((cu2 & 0x3F) << 6) +
+ (c8 & 0x3F);
+ }
+ else
+ {
+ char8_t cu3 = ps->__value.__wchb[2];
+ wc = ((cu1 & 0x07) << 18) +
+ ((cu2 & 0x3F) << 12) +
+ ((cu3 & 0x3F) << 6) +
+ (c8 & 0x3F);
+ }
+ }
+ ps->__count &= 0x7fffffff;
+ ps->__value.__wch = 0;
+ }
+
+ return wcrtomb (s, wc, ps);
+}
new file mode 100644
@@ -0,0 +1,126 @@
+/* Multibyte to UTF-8 conversion.
+ Copyright (C) 2022 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <https://www.gnu.org/licenses/>. */
+
+#include <assert.h>
+#include <dlfcn.h>
+#include <errno.h>
+#include <gconv.h>
+#include <uchar.h>
+#include <wcsmbsload.h>
+
+#include <sysdep.h>
+
+#ifndef EILSEQ
+# define EILSEQ EINVAL
+#endif
+
+
+/* This is the private state used if PS is NULL. */
+static mbstate_t state;
+
+size_t
+mbrtoc8 (char8_t *pc8, const char *s, size_t n, mbstate_t *ps)
+{
+ /* This implementation depends on the converter invoked by mbrtowc() not
+ needing to retain state in either the top most bit of ps->__count or
+ in ps->__value between invocations. This implementation uses the
+ top most bit of ps->__count to indicate that trailing code units are
+ yet to be written and uses ps->__value to store those code units. */
+
+ if (ps == NULL)
+ ps = &state;
+
+ /* If state indicates that trailing code units are yet to be written, write
+ those first regardless of whether 's' is a null pointer. */
+ if (ps->__count & 0x80000000)
+ {
+ /* ps->__value.__wchb[3] stores the index of the next code unit to
+ write. Code units are stored in reverse order. */
+ size_t i = ps->__value.__wchb[3];
+ if (pc8 != NULL)
+ {
+ *pc8 = ps->__value.__wchb[i];
+ }
+ if (i == 0)
+ {
+ ps->__count &= 0x7fffffff;
+ ps->__value.__wch = 0;
+ }
+ else
+ --ps->__value.__wchb[3];
+ return -3;
+ }
+
+ if (s == NULL)
+ {
+ /* if 's' is a null pointer, behave as if a null pointer was passed for
+ 'pc8', an empty string was passed for 's', and 1 passed for 'n'. */
+ pc8 = NULL;
+ s = "";
+ n = 1;
+ }
+
+ wchar_t wc;
+ size_t result;
+
+ result = mbrtowc(&wc, s, n, ps);
+ if (result <= n)
+ {
+ if (wc <= 0x7F)
+ {
+ if (pc8 != NULL)
+ *pc8 = wc;
+ }
+ else if (wc <= 0x7FF)
+ {
+ if (pc8 != NULL)
+ *pc8 = 0xC0 + ((wc >> 6) & 0x1F);
+ ps->__value.__wchb[0] = 0x80 + (wc & 0x3F);
+ ps->__value.__wchb[3] = 0;
+ ps->__count |= 0x80000000;
+ }
+ else if (wc <= 0xFFFF)
+ {
+ if (pc8 != NULL)
+ *pc8 = 0xE0 + ((wc >> 12) & 0x0F);
+ ps->__value.__wchb[1] = 0x80 + ((wc >> 6) & 0x3F);
+ ps->__value.__wchb[0] = 0x80 + (wc & 0x3F);
+ ps->__value.__wchb[3] = 1;
+ ps->__count |= 0x80000000;
+ }
+ else if (wc <= 0x10FFFF)
+ {
+ if (pc8 != NULL)
+ *pc8 = 0xF0 + ((wc >> 18) & 0x07);
+ ps->__value.__wchb[2] = 0x80 + ((wc >> 12) & 0x3F);
+ ps->__value.__wchb[1] = 0x80 + ((wc >> 6) & 0x3F);
+ ps->__value.__wchb[0] = 0x80 + (wc & 0x3F);
+ ps->__value.__wchb[3] = 2;
+ ps->__count |= 0x80000000;
+ }
+ }
+ if (result == 0 && wc != 0)
+ {
+ /* mbrtowc() never returns -3. When a MB sequence converts to multiple
+ WCs, no input is consumed when writing the subsequent WCs resulting
+ in a result of 0 even if a null character wasn't written. */
+ result = -3;
+ }
+
+ return result;
+}
@@ -31,6 +31,13 @@
#include <bits/types.h>
#include <bits/types/mbstate_t.h>
+/* Declare the C2x char8_t typedef in _GNU_SOURCE and C2x modes, but only
+ if the C++ __cpp_char8_t feature test macro is not defined. */
+#if (defined __USE_GNU || __GLIBC_USE (ISOC2X)) && !defined __cpp_char8_t
+/* Define the 8-bit character type. */
+typedef unsigned char char8_t;
+#endif
+
#ifndef __USE_ISOCXX11
/* Define the 16-bit and 32-bit character types. */
typedef __uint_least16_t char16_t;
@@ -40,6 +47,20 @@ typedef __uint_least32_t char32_t;
__BEGIN_DECLS
+/* Declare the C2x mbrtoc8() and c8rtomb() functions in _GNU_SOURCE and C2x
+ modes or if the C++ __cpp_char8_t feature test macro is defined. */
+#if defined __USE_GNU || __GLIBC_USE (ISOC2X) || defined __cpp_char8_t
+/* Write char8_t representation of multibyte character pointed
+ to by S to PC8. */
+extern size_t mbrtoc8 (char8_t *__restrict __pc8,
+ const char *__restrict __s, size_t __n,
+ mbstate_t *__restrict __p) __THROW;
+
+/* Write multibyte representation of char8_t C8 to S. */
+extern size_t c8rtomb (char *__restrict __s, char8_t __c8,
+ mbstate_t *__restrict __ps) __THROW;
+#endif
+
/* Write char16_t representation of multibyte character pointed
to by S to PC16. */
extern size_t mbrtoc16 (char16_t *__restrict __pc16,