mbox series

[v8,0/2] C.UTF-8

Message ID 20210825133437.2529307-1-carlos@redhat.com
Headers show
Series C.UTF-8 | expand

Message

Carlos O'Donell Aug. 25, 2021, 1:34 p.m. UTC
The following changes implement a minimally sized C.UTF-8.
First we implement the 'codepoint_collation' directive.
Then we implement C.UTF-8 with an LC_COLLATE that uses the
'codepoint_collation' directive to support using strcmp
or wcscmp for collation i.e. code point sorting. The final
C.UTF-8 is only ~396KiB with the largest ~346KiB in
LC_CTYPE for all of Unicode.

v8 includes a NEWS entry for the updated C.UTF-8.
v7 fixed the regressions detected in Fedora Rawhide
here: https://bugzilla.redhat.com/show_bug.cgi?id=1986421,
but does so by generating identity tables for 
_NL_COLLATE_COLLSEQMB, and _NL_COLLATE_COLLSEQWC to provide
mappings for ASCII characters. This ensures that static
applications using the new C.UTF-8 have a functioning
fnmatch, regcomp, and regexec for ASCII ranges. This raises
the size of LC_COLLATE from 92 to 1406 bytes. Valgrind
reports no errors using the tables with C.UTF-8 under
tst-fnmatch. v7 also corrected collation sequence byte
ordering on BE targets, and I verified this by building
crossed locales with localedef --big-endian and confirming
that s390x built native C.UTF-8 is the same as an x86_64
C.UTF-8 built wtih --big-endian.

The fixes that were in v4 for nrules == 0 will be included
in the next release of glibc, and when those are proven
correct they can be backported to provide dyanmic
or newly compiled static applications with the ability
to use all code points in ranges.

Carlos O'Donell (2):
  Add 'codepoint_collation' support for LC_COLLATE.
  Add generic C.UTF-8 locale (Bug 17318)

 NEWS                             |  10 +-
 iconv/Makefile                   |  22 +-
 iconv/tst-iconv9.c               |  87 +++++
 locale/C-collate-seq.c           | 101 ++++++
 locale/C-collate.c               |  78 +----
 locale/programs/ld-collate.c     |  36 +-
 locale/programs/locfile-kw.gperf |   1 +
 locale/programs/locfile-kw.h     | 299 ++++++++---------
 locale/programs/locfile-token.h  |   1 +
 localedata/C.UTF-8.in            | 157 +++++++++
 localedata/Makefile              |   2 +
 localedata/SUPPORTED             |   1 +
 localedata/locales/C             | 194 +++++++++++
 posix/Makefile                   |  16 +-
 posix/bug-regex1.c               |  20 ++
 posix/bug-regex19.c              |  22 +-
 posix/bug-regex4.c               |  25 ++
 posix/bug-regex6.c               |   2 +-
 posix/transbug.c                 |  22 +-
 posix/tst-fnmatch.input          | 549 ++++++++++++++++++++++++++++++-
 posix/tst-regcomp-truncated.c    |   1 +
 posix/tst-regex.c                |  25 +-
 22 files changed, 1413 insertions(+), 258 deletions(-)
 create mode 100644 iconv/tst-iconv9.c
 create mode 100644 locale/C-collate-seq.c
 create mode 100644 localedata/C.UTF-8.in
 create mode 100644 localedata/locales/C