mbox series

[0/5] Assume UTF-8 encoding for localedef input files

Message ID cover.1652994079.git.fweimer@redhat.com
Headers
Series Assume UTF-8 encoding for localedef input files |

Message

Florian Weimer May 19, 2022, 9:06 p.m. UTC 
  This is a backwards-compatible change because of two localedef bugs that
cause bytes outside the ASCII range to produce unpredictable results:

  If char is signed, conversion from the assumed ISO-8859-1 input format
  to a UCS-4 codepoint does not produce the correct result.

  If the output character set is not overlapping ISO-8859-1 in the
  characters used in the locale, the required character set conversion
  is not applied.

This is why I think we can switch to UTF-8 without impacting backwards
compatibility, and there is no need for an option to restore the old
behavior.

Tested on i686-linux-gnu and x86_64-linux-gnu.

Thanks,
Florian

Florian Weimer (5):
  locale: Turn ADDC and ADDS into functions in linereader.c
  locale: Fix signed char bug in lr_getc
  locale: Introduce translate_unicode_codepoint into linereader.c
  locale: localdef input files are now encoded in UTF-8
  de_DE: Convert to UTF-8

 NEWS                         |   4 +
 locale/programs/linereader.c | 504 ++++++++++++++++++++++-------------
 locale/programs/linereader.h |   2 +-
 localedata/locales/de_DE     |  32 +--
 4 files changed, 338 insertions(+), 204 deletions(-)


base-commit: 2d5ec6692f5746ccb11db60976a6481ef8e9d74f