From patchwork Sun Jun 21 14:11:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Carlos O'Donell X-Patchwork-Id: 39713 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id BC91D388C036; Sun, 21 Jun 2020 14:11:49 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org BC91D388C036 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1592748709; bh=5iycqapTbpXeeAJK7qcdoZWwm6mEmpBZ44IyZewNZ8w=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=Bn1QdGiCF1zjI/xZfaM8UPivewaOU/pGh2+SxNpNuxF5OZYxQ+AuDmLagVZPlSC2W rb1Ee6Q+dwR6Akw2tZdfJxlfI8MNGZIxiuMi0EKmFlrzNkA0eT/IOgk/D9y9CxAFW1 boF1Aag0Hl6ifD9F9S5Asrq7U7Db+fqpaXgTqHgY= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by sourceware.org (Postfix) with ESMTP id A8BF9388C013 for ; Sun, 21 Jun 2020 14:11:43 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org A8BF9388C013 Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-149-K6S0R0PgMdyYPNsaGC9c3w-1; Sun, 21 Jun 2020 10:11:20 -0400 X-MC-Unique: K6S0R0PgMdyYPNsaGC9c3w-1 Received: by mail-qt1-f200.google.com with SMTP id s30so8218286qts.18 for ; Sun, 21 Jun 2020 07:11:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:organization:message-id:date :user-agent:mime-version:content-language:content-transfer-encoding; bh=5iycqapTbpXeeAJK7qcdoZWwm6mEmpBZ44IyZewNZ8w=; b=ozuT8KJ/LrJ5M4q/jvUncrg9P7QBF1CxbUoRnccYqSnqcwBnSDSe+1z0dOhD2/IKbB bkv44oB96c5X9MPbMBdT8Ng9iEN+/JOQxDbSEnk2n6MJz7l2C1wcUPgIou7aQjZIXfIu 1jINKHCXU842tOCw3BwRDN0WhU+IuIcds7eM36STI2LC08+k+0r/+oKahWpkR6ckuA89 MsPb70c0bPrD3Vjig0fNDSPfKqja2NzArPH3SVPL/O+lLXxyRnOLX5NkWPwv4zmtfkxJ 7OOvphhPP3LToeedsoR5dtqrDVzLgRq7PgpOTRlmqZgrxXa0mdF8NB97LQs9JGfqTA7P FxZg== X-Gm-Message-State: AOAM533olJ53TWPnHZjgRNen2npNFe9KDx5/fTNZNWk/+AYoLoFSXdQJ z/ssXAKE/HX6HeVcvlZ8coUXoTpKx59Fc8AZrFeeey9R1lR355C8xcQbSpfsASqb5VZjIfHtUs1 hCxbOppA0sCv4gWEwri82 X-Received: by 2002:a0c:b787:: with SMTP id l7mr8309188qve.179.1592748678408; Sun, 21 Jun 2020 07:11:18 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwVhiHb5r1+4TW7sqTeQnRvlx9g6l19eb+SEJMz5qD60yst+A9RdH0j2EmytnoWWeAZ0yUong== X-Received: by 2002:a0c:b787:: with SMTP id l7mr8309092qve.179.1592748677053; Sun, 21 Jun 2020 07:11:17 -0700 (PDT) Received: from [192.168.1.4] (198-84-170-103.cpe.teksavvy.com. [198.84.170.103]) by smtp.gmail.com with ESMTPSA id j16sm5635816qtp.92.2020.06.21.07.11.15 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 21 Jun 2020 07:11:16 -0700 (PDT) To: libc-alpha , Florian Weimer , Rafal Luzynski , Mike Fabian Subject: [RFC] iconvdata/, localedata/: Fix TSCII and document tests. Organization: Red Hat Message-ID: <103199fc-7c6c-93ce-4702-2689b06b67b7@redhat.com> Date: Sun, 21 Jun 2020 10:11:14 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 Content-Language: en-US X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-10.7 required=5.0 tests=BAYES_40, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Carlos O'Donell via Libc-alpha From: Carlos O'Donell Reply-To: Carlos O'Donell Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" This is a general cleanup of the testing infrastructure around the iconv tests. I'm posting it now to get any feedback about general direction with the cleanup. Should I pass arguments to the script instead of environment variables? I have one test failure in test-tables which I need to fix which has to do with the regexps there. 8< --- 8< --- 8< The TSCII charmap was completely broken and is fixed by the removal of the many-to-one mappings, not needed WIDTH specification, invalid identical encodings, and wrong mb_cur_max. The %IRREVERSIBLE% markup is documented, and that documentation is extended to all locales that have such markup. TSCII was not explicitly being excluded from testing, but it should have been, and this led me to cleanup run-iconv-test.sh for TSCII. The iconvdata/run-iconv-test.sh had two implicit inputs, TESTS and TESTS2, the latter which was not included in the dependencies, along with iconv-test.yyy which was not included in generated. We rename the tests to iconv-test1.in and iconv-test2.in because that makes it clearer they are a part of the iconv-test special test, and we pass the test data names via env vars. We add iconv-test2.in to the dependencies, and iconv-test.yyy to generated to ensure the test is rebuilt if the test data changes and that iconv-test.yyy is removed. Further to this we extensively comment run-iconv-test.sh and explain what each step of the test is doing. We remove the cryptic grep for excluding charmaps that have data that cannot be handled by iconv with charmaps as input and we explicitly name those charmaps which cannot be tested in "Test 1c: Convert using charmaps." and why. We incidentally fix the BIG5-HKSCS <-> BIG5HKSCS alias problem even if we don't test it, we might in the future, and not it will work. In reviewing iconvdata/tst-table-charmap.sh it was shown that the %IRREVERSIBLE% markup in the charmaps is used both by the table generation for testing *and* the iconv table generation. Thus as part of the additional documentation effort we add comments to all of the %IRREVERSIBLE% markup to note where it is used for testing or where it is used for iconv table generation. For BIG5-HKSCS we add additional comments for HKSCS-2016, which was quite helpful during review. --- iconvdata/Makefile | 10 +- iconvdata/{TESTS => iconv-test1.in} | 0 iconvdata/{TESTS2 => iconv-test2.in} | 0 iconvdata/run-iconv-test.sh | 123 +++++++- localedata/charmaps/BIG5 | 7 + localedata/charmaps/BIG5-HKSCS | 32 +++ localedata/charmaps/EUC-JP-MS | 7 + localedata/charmaps/EUC-TW | 1 + localedata/charmaps/IBM1132 | 1 + localedata/charmaps/IBM1133 | 1 + localedata/charmaps/IBM1160 | 4 + localedata/charmaps/IBM1161 | 2 + localedata/charmaps/TSCII | 409 ++++++++++++++------------- localedata/charmaps/WINDOWS-31J | 7 +- 14 files changed, 394 insertions(+), 210 deletions(-) rename iconvdata/{TESTS => iconv-test1.in} (100%) rename iconvdata/{TESTS2 => iconv-test2.in} (100%) diff --git a/iconvdata/Makefile b/iconvdata/Makefile index 4ec2741cdc..6a02f227e0 100644 --- a/iconvdata/Makefile +++ b/iconvdata/Makefile @@ -178,8 +178,9 @@ generated-modules := $(gen-8bit-modules) $(gen-8bit-gap-modules) \ $(gen-special-modules) generated += $(generated-modules:=.h) $(generated-modules:=.stmp) \ - iconv-test.out iconv-rules tst-loading.mtrace \ - mtrace-tst-loading.out tst-tables.out iconv-test.xxx + iconv-test.out iconv-rules tst-loading.mtrace \ + mtrace-tst-loading.out tst-tables.out iconv-test.xxx \ + iconv-test.yyy ifdef objpfx generated += gconv-modules endif @@ -324,8 +325,11 @@ $(objpfx)bug-iconv12.out: $(objpfx)gconv-modules \ $(objpfx)iconv-test.out: run-iconv-test.sh $(objpfx)gconv-modules \ $(addprefix $(objpfx),$(modules.so)) \ - $(common-objdir)/iconv/iconv_prog TESTS + $(common-objdir)/iconv/iconv_prog iconv-test1.in \ + iconv-test2.in iconv_modules="$(modules)" \ + iconv_test1="iconv-test1.in" \ + iconv_test2="iconv-test2.in" \ $(SHELL) $< $(common-objdir) '$(test-wrapper-env)' \ '$(run-program-env)' > $@; \ $(evaluate-test) diff --git a/iconvdata/TESTS b/iconvdata/iconv-test1.in similarity index 100% rename from iconvdata/TESTS rename to iconvdata/iconv-test1.in diff --git a/iconvdata/TESTS2 b/iconvdata/iconv-test2.in similarity index 100% rename from iconvdata/TESTS2 rename to iconvdata/iconv-test2.in diff --git a/iconvdata/run-iconv-test.sh b/iconvdata/run-iconv-test.sh index 56b6630a6d..e337146af7 100755 --- a/iconvdata/run-iconv-test.sh +++ b/iconvdata/run-iconv-test.sh @@ -23,6 +23,10 @@ set -e codir=$1 test_wrapper_env="$2" run_program_env="$3" +# Additionally we get as input the following environment variables: +# iconv_modules: The list of modules to test. +# iconv_test1: The input data for the first test. +# iconv_test2: The input data for the second test. # We use always the same temporary file. temp1=$codir/iconvdata/iconv-test.xxx @@ -45,9 +49,10 @@ else ac_n= ac_c='\c' ac_t= fi -# We read the file named TESTS. All non-empty lines not starting with +# We read the file named ICONV_TEST1. All non-empty lines not starting with # `#' are interpreted as commands. failed=0 +echo "Reading $iconv_tests and running tests:" while read from to subset targets; do # Ignore empty and comment lines. if test -z "$subset" || test "$from" = '#'; then continue; fi @@ -57,91 +62,179 @@ while read from to subset targets; do if test -n "$targets"; then for t in $targets; do + # Test 1a: Convert data using iconv internally. if test -f testdata/$from; then echo $ac_n " test data: $from -> $t $ac_c" + + # Convert the test data using iconv DSOs. $PROG -f $from -t $t testdata/$from < /dev/null > $temp1 || { if test $? -gt 128; then exit 1; fi echo "FAILED"; failed=1; continue; } + + # Conversion succeeded. echo $ac_n "OK$ac_c" + + # Compare converted data to expected data. if test -s testdata/$from..$t; then LC_ALL=C cmp $temp1 testdata/$from..$t > /dev/null 2>&1 || { echo "/FAILED"; failed=1; continue; } echo $ac_n "/OK$ac_c" fi + + # Conversion *and* comparison succeeded. echo $ac_n " -> $from $ac_c" + + # Convert in the opposite direction. $PROG -f $t -t $to -o $temp2 $temp1 < /dev/null || { if test $? -gt 128; then exit 1; fi echo "FAILED"; failed=1; continue; } + + # Conversion succeeded. echo $ac_n "OK$ac_c" + + # Compare converted data to expected data. test -s $temp1 && LC_ALL=C cmp testdata/$from $temp2 > /dev/null 2>&1 || { echo "/FAILED"; failed=1; continue; } + + # Conversion *and* comparison succeeded in both directions. echo "/OK" + # Cleanup test. rm -f $temp1 $temp2 + fi - # Now test some bigger text, entirely in ASCII. If ASCII is no subset + # Test 1b: Convert "The Art of War" (larger text) [ASCII subset] + # + # Now test some bigger text, entirely in ASCII. If ASCII is not a subset # of the coded character set we convert the text to this coded character # set. Otherwise we convert to all the TARGETS. if test $subset = Y; then echo $ac_n " suntzu: $from -> $t -> $to $ac_c" + + # Convert test data to target and back. $PROG -f $from -t $t testdata/suntzus < /dev/null | $PROG -f $t -t $to > $temp1 || { if test $? -gt 128; then exit 1; fi echo "FAILED"; failed=1; continue; } + + # Conversion succeeded. echo $ac_n "OK$ac_c" + + # Compare against expected data. LC_ALL=C cmp testdata/suntzus $temp1 || { echo "/FAILED"; failed=1; continue; } + + # Conversion *and* comparison succeeded. echo "/OK" + # Cleanup test. + rm -f $temp1 fi - rm -f $temp1 - - # And tests where iconv(1) has to handle charmaps. - if test "$t" = UTF8; then tc=UTF-8; else tc="$t"; fi - if test -f ../localedata/charmaps/$from && - test -f ../localedata/charmaps/$tc && - test -f testdata/$from && - ! grep '' ../localedata/charmaps/$from > /dev/null; then + + # Test 1c: Convert using charmaps. + # + # And tests where iconv(1) has to handle charmaps: + # We exclude charmaps that have many-to-one mappings, which today is + # TSCII, SHIFT_SJISX0213, BIG5HKSCS, and EUC-JISX0213. We exclude them + # because such mappings make the there-and-back conversion tests + # difficult. All of these charmaps have commented out entries or + # valid entries for their many-to-one conversions e.g. + # '% /x83/xa4 TAMIL GLYPH JU' (TSCII) + # or '% /x82/xf5' (SHIFT_JISX0213). + # We could test them but it would require splitting the data files out + # into those that can be handled by charmap conversions versus those + # that can only be handled by iconv conversion (the larger set). + + # Some charsets need to be converted to special charmap names. + tc="$t"; + fromc="$from"; + if test "$t" = UTF8; then tc=UTF-8; fi + if test "$from" = BIG5HKSCS; then fromc=BIG5-HKSCS; fi + + # Run the charmap tests: + if test -f ../localedata/charmaps/$fromc \ + && test -f ../localedata/charmaps/$tc \ + && test -f testdata/$from \ + && test "$from" != "BIG5HKSCS" \ + && test "$from" != "EUC-JISX0213" \ + && test "$from" != "SHIFT_JISX0213" \ + && test "$from" != "TSCII"; then + + # Identify the the FROM and TO charmaps. echo $ac_n "test charmap: $from -> $t $ac_c" - $PROG -f ../localedata/charmaps/$from -t ../localedata/charmaps/$tc \ - testdata/$from < /dev/null > $temp1 || + + # Convert from FROM to TO using the initial data in testdata/FROM + # and storing in $temp1. + $PROG -f ../localedata/charmaps/$fromc -t ../localedata/charmaps/$tc \ + testdata/$from < /dev/null > $temp1 || { if test $? -gt 128; then exit 1; fi echo "FAILED"; failed=1; continue; } + + # The conversion from FROM to TO of the testdata/FROM succeeded. echo $ac_n "OK$ac_c" + + # If testdata/FROM..TO exists then we compare the generated output + # in $temp1 to the the expected output in FROM..TO. if test -s testdata/$from..$t; then LC_ALL=C cmp $temp1 testdata/$from..$t > /dev/null 2>&1 || { echo "/FAILED"; failed=1; continue; } + + # The comparison to expected output succeeded. echo $ac_n "/OK$ac_c" fi + + # Conversion *and* the comparison succeeded. echo $ac_n " -> $from $ac_c" + + # Run the conversion from TO to FROM (backwards) storing in $temp2. $PROG -t ../localedata/charmaps/$from -f ../localedata/charmaps/$tc \ -o $temp2 $temp1 < /dev/null || { if test $? -gt 128; then exit 1; fi echo "FAILED"; failed=1; continue; } + + # Conversion succeeded. echo $ac_n "OK$ac_c" + + # Compare the expected data with the converted data. test -s $temp1 && LC_ALL=C cmp testdata/$from $temp2 > /dev/null 2>&1 || { echo "/FAILED"; failed=1; continue; } + + # We succeeded in converting to the target charmap and back + # again and both directions worked and the data matched. echo "/OK" + + # Cleanup test. rm -f $temp1 $temp2 fi done fi + # Test 1b: Convert "The Art of War" (larger text) [Non-ASCII subset] if test "$subset" = N; then echo $ac_n " suntzu: ASCII -> $to -> ASCII $ac_c" + + # Convert only from ASCII to target and back to ASCII. $PROG -f ASCII -t $to testdata/suntzus < /dev/null | $PROG -f $to -t ASCII > $temp1 || { if test $? -gt 128; then exit 1; fi echo "FAILED"; failed=1; continue; } + + # Conversion succeeded. echo $ac_n "OK$ac_c" + + # Compare against expected ASCII data. LC_ALL=C cmp testdata/suntzus $temp1 || { echo "/FAILED"; failed=1; continue; } + + # Conversion *and* comparison succeeded. echo "/OK" + # Cleanup test. + rm -f $temp1 fi -done < TESTS +done < "$iconv_test1" -# We read the file named TESTS2. All non-empty lines not starting with +# We read the file named ICONV_TEST2. All non-empty lines not starting with # `#' are interpreted as commands. while read utf8 from filename; do # Ignore empty and comment lines. @@ -182,7 +275,7 @@ while read utf8 from filename; do { echo "/FAILED"; failed=1; continue; } echo "OK" -done < TESTS2 +done < "$iconv_test2" # Check for crashes in decoders. printf '\016\377\377\377\377\377\377\377' > $temp1 diff --git a/localedata/charmaps/BIG5 b/localedata/charmaps/BIG5 index 50f5f16cd2..b552e43763 100644 --- a/localedata/charmaps/BIG5 +++ b/localedata/charmaps/BIG5 @@ -19,6 +19,9 @@ % /xa2/xcc, /xa2/xce, /xf9/xe9, /xf9/xea, /xf9/xeb, % /xf9/xf9, /xf9/xfa, /xf9/xfb, /xf9/xfc, /xf9/xfd % +% The %IRREVERSIBLE% markup is used to generate the data used by iconv. +% Please do not remove the %IRREVERSIBLE% markup. +% % alias BIG5-CP950 CHARMAP @@ -417,8 +420,10 @@ CHARMAP /xa2/xc9 HANGZHOU NUMERAL SEVEN /xa2/xca HANGZHOU NUMERAL EIGHT /xa2/xcb HANGZHOU NUMERAL NINE +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /xa2/xcc /xa2/xcd +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /xa2/xce /xa2/xcf FULLWIDTH LATIN CAPITAL LETTER A /xa2/xd0 FULLWIDTH LATIN CAPITAL LETTER B @@ -14050,6 +14055,7 @@ CHARMAP /xf9/xe6 BOX DRAWINGS DOWN SINGLE AND RIGHT DOUBLE /xf9/xe7 BOX DRAWINGS DOWN SINGLE AND HORIZONTAL DOUBLE /xf9/xe8 BOX DRAWINGS DOWN SINGLE AND LEFT DOUBLE +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /xf9/xe9 BOX DRAWINGS VERTICAL SINGLE AND RIGHT DOUBLE %IRREVERSIBLE% /xf9/xea BOX DRAWINGS VERTICAL SINGLE AND HORIZONTAL DOUBLE %IRREVERSIBLE% /xf9/xeb BOX DRAWINGS VERTICAL SINGLE AND LEFT DOUBLE @@ -14066,6 +14072,7 @@ CHARMAP /xf9/xf6 BOX DRAWINGS UP DOUBLE AND HORIZONTAL SINGLE /xf9/xf7 BOX DRAWINGS UP DOUBLE AND LEFT SINGLE /xf9/xf8 BOX DRAWINGS DOUBLE VERTICAL +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /xf9/xf9 BOX DRAWINGS DOUBLE HORIZONTAL %IRREVERSIBLE% /xf9/xfa BOX DRAWINGS LIGHT ARC DOWN AND RIGHT %IRREVERSIBLE% /xf9/xfb BOX DRAWINGS LIGHT ARC DOWN AND LEFT diff --git a/localedata/charmaps/BIG5-HKSCS b/localedata/charmaps/BIG5-HKSCS index 0735efc5c8..3489654fb2 100644 --- a/localedata/charmaps/BIG5-HKSCS +++ b/localedata/charmaps/BIG5-HKSCS @@ -10,6 +10,12 @@ % Last updated from the HKSCS-2008 standard % http://www.ogcio.gov.hk/en/business/tech_promotion/ccli/terms/doc/e_hkscs_2008.pdf % +% The latest available standard is HKSCS-2016 which has not yet been applied +% https://www.ogcio.gov.hk/en/our_work/business/tech_promotion/ccli/terms/doc/e_hkscs_2016.pdf +% +% The %IRREVERSIBLE% markup is used to generate the data used by iconv. +% Please do not remove the %IRREVERSIBLE% markup. +% CHARMAP /x00 NULL @@ -300,8 +306,31 @@ CHARMAP /x88/x5f LATIN CAPITAL LETTER O WITH ACUTE /x88/x60 LATIN CAPITAL LETTER O WITH CARON /x88/x61 LATIN CAPITAL LETTER O WITH GRAVE +% +% BIG5-HKSCS has 4 mappings that cannot be represented in a normal +% character map. For BIG5-HKSCS 2008 (Amd. 1 to Amd. 6) they are: +% /x88/x62 => +% /x88/x64 => +% /x88/xa3 => +% /x88/xa5 => +% +% These mappings are noted in BIG5-HKSCS 2016 standard as documented in the +% initial comments. +% +% These mappings cannot be represented in a traditional POSIX character map +% which has no way to indicate that a sequence of multi-byte characters +% generates multiple wide characters. Therefore we document them here but do +% not include them. Note that iconv is capable of and supports such +% conversions, but iconv when run with character maps as from-encoding or +% to-encoding is unable to support such conversions. +% +% Note that iconv is capable of and supports such conversions, but iconv +% when run with character maps as from-encoding or to-encoding is unable +% to support such conversions. + % /x88/x62 /x88/x63 LATIN CAPITAL LETTER E WITH CIRCUMFLEX AND ACUTE +% See note above. % /x88/x64 /x88/x65 LATIN CAPITAL LETTER E WITH CIRCUMFLEX AND GRAVE /x88/x66 LATIN CAPITAL LETTER E WITH CIRCUMFLEX @@ -331,8 +360,10 @@ CHARMAP /x88/x7e LATIN SMALL LETTER U WITH DIAERESIS AND CARON /x88/xa1 LATIN SMALL LETTER U WITH DIAERESIS AND GRAVE /x88/xa2 LATIN SMALL LETTER U WITH DIAERESIS +% See note above. % /x88/xa3 /x88/xa4 LATIN SMALL LETTER E WITH CIRCUMFLEX AND ACUTE +% See note above. % /x88/xa5 /x88/xa6 LATIN SMALL LETTER E WITH CIRCUMFLEX AND GRAVE /x88/xa7 LATIN SMALL LETTER E WITH CIRCUMFLEX @@ -4201,6 +4232,7 @@ CHARMAP /xa2/x7b BOX DRAWINGS LIGHT DOWN AND LEFT /xa2/x7c BOX DRAWINGS LIGHT UP AND RIGHT /xa2/x7d BOX DRAWINGS LIGHT UP AND LEFT +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /xa2/x7e BOX DRAWINGS LIGHT ARC DOWN AND RIGHT %IRREVERSIBLE% /xa2/xa1 BOX DRAWINGS LIGHT ARC DOWN AND LEFT %IRREVERSIBLE% /xa2/xa2 BOX DRAWINGS LIGHT ARC UP AND RIGHT diff --git a/localedata/charmaps/EUC-JP-MS b/localedata/charmaps/EUC-JP-MS index 6b1c9e4733..9fd1139ea4 100644 --- a/localedata/charmaps/EUC-JP-MS +++ b/localedata/charmaps/EUC-JP-MS @@ -839,16 +839,19 @@ CHARMAP /xad/xed SQUARE ERA NAME MEIZI /xad/xee SQUARE ERA NAME TAISYOU /xad/xef SQUARE ERA NAME SYOUWA +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /xad/xf0 APPROXIMATELY EQUAL TO OR THE IMAGE OF %IRREVERSIBLE% /xad/xf1 IDENTICAL TO %IRREVERSIBLE% /xad/xf2 INTEGRAL /xad/xf3 CONTOUR INTEGRAL /xad/xf4 N-ARY SUMMATION +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /xad/xf5 SQUARE ROOT %IRREVERSIBLE% /xad/xf6 UP TACK %IRREVERSIBLE% /xad/xf7 ANGLE /xad/xf8 RIGHT ANGLE /xad/xf9 RIGHT TRIANGLE +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /xad/xfa BECAUSE %IRREVERSIBLE% /xad/xfb INTERSECTION %IRREVERSIBLE% /xad/xfc UNION @@ -8163,6 +8166,7 @@ CHARMAP /x8f/xa2/xb4 MACRON /x8f/xa2/xb5 OGONEK /x8f/xa2/xb6 RING ABOVE +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /x8f/xa2/xb7 FULLWIDTH TILDE /x8f/xa2/xb8 GREEK TONOS /x8f/xa2/xb9 GREEK DIALYTIKA TONOS @@ -8175,6 +8179,7 @@ CHARMAP /x8f/xa2/xee REGISTERED SIGN /x8f/xa2/xef TRADE MARK SIGN /x8f/xa2/xf0 CURRENCY SIGN +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /x8f/xa2/xf1 NUMERO SIGN /x8f/xa6/xe1 GREEK CAPITAL LETTER ALPHA WITH TONOS /x8f/xa6/xe2 GREEK CAPITAL LETTER EPSILON WITH TONOS @@ -14232,6 +14237,7 @@ CHARMAP /x8f/xf3/xfa SMALL ROMAN NUMERAL EIGHT /x8f/xf3/xfb SMALL ROMAN NUMERAL NINE /x8f/xf3/xfc SMALL ROMAN NUMERAL TEN +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /x8f/xf3/xfd ROMAN NUMERAL ONE %IRREVERSIBLE% /x8f/xf3/xfe ROMAN NUMERAL TWO %IRREVERSIBLE% /x8f/xf4/xa1 ROMAN NUMERAL THREE @@ -14244,6 +14250,7 @@ CHARMAP %IRREVERSIBLE% /x8f/xf4/xa8 ROMAN NUMERAL TEN /x8f/xf4/xa9 FULLWIDTH APOSTROPHE /x8f/xf4/xaa FULLWIDTH QUOTATION MARK +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /x8f/xf4/xab PARENTHESIZED IDEOGRAPH STOCK %IRREVERSIBLE% /x8f/xf4/xac NUMERO SIGN %IRREVERSIBLE% /x8f/xf4/xad TELEPHONE SIGN diff --git a/localedata/charmaps/EUC-TW b/localedata/charmaps/EUC-TW index c9c9cdd82a..a3adf38347 100644 --- a/localedata/charmaps/EUC-TW +++ b/localedata/charmaps/EUC-TW @@ -6009,6 +6009,7 @@ CHARMAP % % CNS 11643-1992 Plane 1 again % +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /x8e/xa1/xa1/xa1 IDEOGRAPHIC SPACE %IRREVERSIBLE% /x8e/xa1/xa1/xa2 FULLWIDTH COMMA %IRREVERSIBLE% /x8e/xa1/xa1/xa3 IDEOGRAPHIC COMMA diff --git a/localedata/charmaps/IBM1132 b/localedata/charmaps/IBM1132 index 948d5a4416..6154070aef 100644 --- a/localedata/charmaps/IBM1132 +++ b/localedata/charmaps/IBM1132 @@ -115,6 +115,7 @@ CHARMAP /x6d LOW LINE /x6e GREATER-THAN SIGN /x6f QUESTION MARK +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /x70 LATIN SMALL LETTER K /x72 LAO LETTER LO LING /x73 LAO LETTER LO LOOT diff --git a/localedata/charmaps/IBM1133 b/localedata/charmaps/IBM1133 index a4848439db..cbe09214c3 100644 --- a/localedata/charmaps/IBM1133 +++ b/localedata/charmaps/IBM1133 @@ -219,6 +219,7 @@ CHARMAP /xdb LAO KO LA /xdd LAO HO NO /xde LAO HO MO +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /xdf LATIN SMALL LETTER K /xf0 LAO DIGIT ZERO /xf1 LAO DIGIT ONE diff --git a/localedata/charmaps/IBM1160 b/localedata/charmaps/IBM1160 index 646b921133..cce874f921 100644 --- a/localedata/charmaps/IBM1160 +++ b/localedata/charmaps/IBM1160 @@ -85,6 +85,7 @@ CHARMAP /x4e PLUS SIGN /x4f VERTICAL LINE /x50 AMPERSAND +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /x51 THAI CHARACTER MAI EK /x52 THAI CHARACTER CHO CHAN /x53 THAI CHARACTER CHO CHING @@ -206,6 +207,7 @@ CHARMAP /xc7 LATIN CAPITAL LETTER G /xc8 LATIN CAPITAL LETTER H /xc9 LATIN CAPITAL LETTER I +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /xca THAI CHARACTER MAI THO /xcb THAI CHARACTER SARA II /xcc THAI CHARACTER SARA UE @@ -229,6 +231,7 @@ CHARMAP /xde THAI CHARACTER SARA AI MAIMUAN /xdf THAI CHARACTER SARA AI MAIMALAI /xe0 REVERSE SOLIDUS +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /xe1 THAI CHARACTER MAI TRI /xe2 LATIN CAPITAL LETTER S /xe3 LATIN CAPITAL LETTER T @@ -257,6 +260,7 @@ CHARMAP /xfa THAI CHARACTER MAI CHATTAWA /xfb THAI CHARACTER THANTHAKHAT /xfc THAI CHARACTER NIKHAHIT +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /xfd THAI CHARACTER MAI CHATTAWA /xfe EURO SIGN /xff diff --git a/localedata/charmaps/IBM1161 b/localedata/charmaps/IBM1161 index 9340ba141b..52337c6117 100644 --- a/localedata/charmaps/IBM1161 +++ b/localedata/charmaps/IBM1161 @@ -132,6 +132,7 @@ CHARMAP /x7d RIGHT CURLY BRACKET /x7e TILDE /x7f +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /xa0 THAI CHARACTER MAI EK /xa1 THAI CHARACTER KO KAI /xa2 THAI CHARACTER KHO KHAI @@ -191,6 +192,7 @@ CHARMAP /xd8 THAI CHARACTER SARA U /xd9 THAI CHARACTER SARA UU /xda THAI CHARACTER PHINTHU +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /xdb THAI CHARACTER MAI THO %IRREVERSIBLE% /xdc THAI CHARACTER MAI TRI %IRREVERSIBLE% /xdd THAI CHARACTER MAI CHATTAWA diff --git a/localedata/charmaps/TSCII b/localedata/charmaps/TSCII index 9646f326cb..3d9ae1fb5e 100644 --- a/localedata/charmaps/TSCII +++ b/localedata/charmaps/TSCII @@ -2,8 +2,26 @@ % / 1 - 1 -% based on TSCII version 1.7 + 3 + +% Tamil Script Code for Information Interchange +% +% Based on TSCII version 1.7 +% +% The lower 128 code points are ASCII, but the upper code points are +% TSCII characters that often map to multiple Unicode code points. The +% one-to-many mapping means that much of the character map is commented +% out since we don't support many-to-one mappings in POSIX-compatible +% character maps. There are 179 such mappings where one encoded TSCII +% character is mapped to more than one Unicode code point. +% +% Note that iconv is capable of and supports such conversions, but iconv +% when run with character maps as from-encoding or to-encoding is unable +% to support such conversions. +% +% For conversion reference: +% https://www.unicode.org/notes/tn15/Tscii2Unicode2.pdf +% CHARMAP /x00 NULL @@ -134,27 +152,30 @@ CHARMAP /x7d RIGHT CURLY BRACKET /x7e TILDE /x7f DELETE - /x80 TAMIL DIGIT ZERO (currently unassigned) + /x80 TAMIL DIGIT ZERO (Since Unicode 4.1) /x81 TAMIL DIGIT ONE - /x82 TAMIL GLYPH SRI +% Note: Prior to Unicode 4.1 the SRI ligature was: +% , +% but since Unicode 4.1 the SRI ligature should start with . +% /x82 TAMIL GLYPH SRI /x83 TAMIL LETTER JA - /x83/xa4 TAMIL GLYPH JU - /x83/xa5 TAMIL GLYPH JUU +% /x83/xa4 TAMIL GLYPH JU +% /x83/xa5 TAMIL GLYPH JUU /x84 TAMIL LETTER SSA - /x84/xa4 TAMIL GLYPH SSU - /x84/xa5 TAMIL GLYPH SSUU +% /x84/xa4 TAMIL GLYPH SSU +% /x84/xa5 TAMIL GLYPH SSUU /x85 TAMIL LETTER SA /x86 TAMIL LETTER HA - /x87 TAMIL GLYPH KSHA - /x88 TAMIL GLYPH J - /x89 TAMIL GLYPH SS - /x8a TAMIL GLYPH S - /x8a/xa4 TAMIL GLYPH SU - /x8a/xa5 TAMIL GLYPH SUU - /x8b TAMIL GLYPH H - /x8b/xa4 TAMIL GLYPH HU - /x8b/xa5 TAMIL GLYPH HUU - /x8c TAMIL GLYPH KSH +% /x87 TAMIL GLYPH KSHA +% /x88 TAMIL GLYPH J +% /x89 TAMIL GLYPH SS +% /x8a TAMIL GLYPH S +% /x8a/xa4 TAMIL GLYPH SU +% /x8a/xa5 TAMIL GLYPH SUU +% /x8b TAMIL GLYPH H +% /x8b/xa4 TAMIL GLYPH HU +% /x8b/xa5 TAMIL GLYPH HUU +% /x8c TAMIL GLYPH KSH /x8d TAMIL DIGIT TWO /x8e TAMIL DIGIT THREE /x8f TAMIL DIGIT FOUR @@ -167,10 +188,10 @@ CHARMAP /x96 TAMIL DIGIT SEVEN /x97 TAMIL DIGIT EIGHT /x98 TAMIL DIGIT NINE - /x99 TAMIL GLYPH NGU - /x9a TAMIL GLYPH NYU - /x9b TAMIL GLYPH NGUU - /x9c TAMIL GLYPH NYUU +% /x99 TAMIL GLYPH NGU +% /x9a TAMIL GLYPH NYU +% /x9b TAMIL GLYPH NGUU +% /x9c TAMIL GLYPH NYUU /x9d TAMIL NUMBER TEN /x9e TAMIL NUMBER ONE HUNDRED /x9f TAMIL NUMBER ONE THOUSAND @@ -180,124 +201,136 @@ CHARMAP /xa4 TAMIL VOWEL SIGN U /xa5 TAMIL VOWEL SIGN UU /xa6 TAMIL VOWEL SIGN E - /xa6/xa1 TAMIL VOWEL SIGN O - /xa6/xb8 TAMIL GLYPH KE - /xa6/xb8/xa1 TAMIL GLYPH KAI - /xa6/xb9 TAMIL GLYPH NGE - /xa6/xb9/xa1 TAMIL GLYPH NGAI - /xa6/xba TAMIL GLYPH CE - /xa6/xba/xa1 TAMIL GLYPH CAI - /xa6/xbb TAMIL GLYPH NYE - /xa6/xbb/xa1 TAMIL GLYPH NYAI - /xa6/xbc TAMIL GLYPH TTE - /xa6/xbc/xa1 TAMIL GLYPH TTAI - /xa6/xbd TAMIL GLYPH NNE - /xa6/xbd/xa1 TAMIL GLYPH NNAI - /xa6/xbe TAMIL GLYPH TE - /xa6/xbe/xa1 TAMIL GLYPH TAI - /xa6/xbf TAMIL GLYPH NE - /xa6/xbf/xa1 TAMIL GLYPH NAI - /xa6/xc0 TAMIL GLYPH PE - /xa6/xc0/xa1 TAMIL GLYPH PAI - /xa6/xc1 TAMIL GLYPH ME - /xa6/xc1/xa1 TAMIL GLYPH MAI - /xa6/xc2 TAMIL GLYPH YE - /xa6/xc2/xa1 TAMIL GLYPH YAI - /xa6/xc3 TAMIL GLYPH RE - /xa6/xc3/xa1 TAMIL GLYPH RAI - /xa6/xc4 TAMIL GLYPH LE - /xa6/xc4/xa1 TAMIL GLYPH LAI - /xa6/xc5 TAMIL GLYPH VE - /xa6/xc5/xa1 TAMIL GLYPH VAI - /xa6/xc6 TAMIL GLYPH LLLE - /xa6/xc6/xa1 TAMIL GLYPH LLLAI - /xa6/xc7 TAMIL GLYPH LLE - /xa6/xc7/xa1 TAMIL GLYPH LLAI - /xa6/xc8 TAMIL GLYPH RRE - /xa6/xc8/xa1 TAMIL GLYPH RRAI - /xa6/xc9 TAMIL GLYPH NNNE - /xa6/xc9/xa1 TAMIL GLYPH NNNAI +% The encoded /xa6/xa1 is which is the decomposition of +% and is encoded exactly the same and collated the same. +% +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. +%IRREVERSIBLE% /xa6/xa1 TAMIL VOWEL SIGN O +% /xa6/xb8 TAMIL GLYPH KE +% /xa6/xb8/xa1 TAMIL GLYPH KAI +% /xa6/xb9 TAMIL GLYPH NGE +% /xa6/xb9/xa1 TAMIL GLYPH NGAI +% /xa6/xba TAMIL GLYPH CE +% /xa6/xba/xa1 TAMIL GLYPH CAI +% /xa6/xbb TAMIL GLYPH NYE +% /xa6/xbb/xa1 TAMIL GLYPH NYAI +% /xa6/xbc TAMIL GLYPH TTE +% /xa6/xbc/xa1 TAMIL GLYPH TTAI +% /xa6/xbd TAMIL GLYPH NNE +% /xa6/xbd/xa1 TAMIL GLYPH NNAI +% /xa6/xbe TAMIL GLYPH TE +% /xa6/xbe/xa1 TAMIL GLYPH TAI +% /xa6/xbf TAMIL GLYPH NE +% /xa6/xbf/xa1 TAMIL GLYPH NAI +% /xa6/xc0 TAMIL GLYPH PE +% /xa6/xc0/xa1 TAMIL GLYPH PAI +% /xa6/xc1 TAMIL GLYPH ME +% /xa6/xc1/xa1 TAMIL GLYPH MAI +% /xa6/xc2 TAMIL GLYPH YE +% /xa6/xc2/xa1 TAMIL GLYPH YAI +% /xa6/xc3 TAMIL GLYPH RE +% /xa6/xc3/xa1 TAMIL GLYPH RAI +% /xa6/xc4 TAMIL GLYPH LE +% /xa6/xc4/xa1 TAMIL GLYPH LAI +% /xa6/xc5 TAMIL GLYPH VE +% /xa6/xc5/xa1 TAMIL GLYPH VAI +% /xa6/xc6 TAMIL GLYPH LLLE +% /xa6/xc6/xa1 TAMIL GLYPH LLLAI +% /xa6/xc7 TAMIL GLYPH LLE +% /xa6/xc7/xa1 TAMIL GLYPH LLAI +% /xa6/xc8 TAMIL GLYPH RRE +% /xa6/xc8/xa1 TAMIL GLYPH RRAI +% /xa6/xc9 TAMIL GLYPH NNNE +% /xa6/xc9/xa1 TAMIL GLYPH NNNAI /xa7 TAMIL VOWEL SIGN EE - /xa7/xa1 TAMIL VOWEL SIGN OO +% The encoded /xa7/xa1 is which is the decomposition of +% and is encoded exactly the same and collated the same. +% +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. +%IRREVERSIBLE% /xa7/xa1 TAMIL VOWEL SIGN OO /xa7/xaa TAMIL VOWEL SIGN AU - /xa7/xb8 TAMIL GLYPH KEE - /xa7/xb8/xa1 TAMIL GLYPH KOO - /xa7/xb8/xaa TAMIL GLYPH KAU - /xa7/xb9 TAMIL GLYPH NGEE - /xa7/xb9/xa1 TAMIL GLYPH NGOO - /xa7/xb9/xaa TAMIL GLYPH NGAU - /xa7/xba TAMIL GLYPH CEE - /xa7/xba/xa1 TAMIL GLYPH COO - /xa7/xba/xaa TAMIL GLYPH CAU - /xa7/xbb TAMIL GLYPH NYEE - /xa7/xbb/xa1 TAMIL GLYPH NYOO - /xa7/xbb/xaa TAMIL GLYPH NYAU - /xa7/xbc TAMIL GLYPH TTEE - /xa7/xbc/xa1 TAMIL GLYPH TTOO - /xa7/xbc/xaa TAMIL GLYPH TTAU - /xa7/xbd TAMIL GLYPH NNEE - /xa7/xbd/xa1 TAMIL GLYPH NNOO - /xa7/xbd/xaa TAMIL GLYPH NNAU - /xa7/xbe TAMIL GLYPH TEE - /xa7/xbe/xa1 TAMIL GLYPH TOO - /xa7/xbe/xaa TAMIL GLYPH TAU - /xa7/xbf TAMIL GLYPH NEE - /xa7/xbf/xa1 TAMIL GLYPH NOO - /xa7/xbf/xaa TAMIL GLYPH NAU - /xa7/xc0 TAMIL GLYPH PEE - /xa7/xc0/xa1 TAMIL GLYPH POO - /xa7/xc0/xaa TAMIL GLYPH PAU - /xa7/xc1 TAMIL GLYPH MEE - /xa7/xc1/xa1 TAMIL GLYPH MOO - /xa7/xc1/xaa TAMIL GLYPH MAU - /xa7/xc2 TAMIL GLYPH YEE - /xa7/xc2/xa1 TAMIL GLYPH YOO - /xa7/xc2/xaa TAMIL GLYPH YAU - /xa7/xc3 TAMIL GLYPH REE - /xa7/xc3/xa1 TAMIL GLYPH ROO - /xa7/xc3/xaa TAMIL GLYPH RAU - /xa7/xc4 TAMIL GLYPH LEE - /xa7/xc4/xa1 TAMIL GLYPH LOO - /xa7/xc4/xaa TAMIL GLYPH LAU - /xa7/xc5 TAMIL GLYPH VEE - /xa7/xc5/xa1 TAMIL GLYPH VOO - /xa7/xc5/xaa TAMIL GLYPH VAU - /xa7/xc6 TAMIL GLYPH LLLEE - /xa7/xc6/xa1 TAMIL GLYPH LLLOO - /xa7/xc6/xaa TAMIL GLYPH LLLAU - /xa7/xc7 TAMIL GLYPH LLEE - /xa7/xc7/xa1 TAMIL GLYPH LLOO - /xa7/xc7/xaa TAMIL GLYPH LLAU - /xa7/xc8 TAMIL GLYPH RREE - /xa7/xc8/xa1 TAMIL GLYPH RROO - /xa7/xc8/xaa TAMIL GLYPH RRAU - /xa7/xc9 TAMIL GLYPH NNNEE - /xa7/xc9/xa1 TAMIL GLYPH NNNOO - /xa7/xc9/xaa TAMIL GLYPH NNNAU +% /xa7/xb8 TAMIL GLYPH KEE +% /xa7/xb8/xa1 TAMIL GLYPH KOO +% /xa7/xb8/xaa TAMIL GLYPH KAU +% /xa7/xb9 TAMIL GLYPH NGEE +% /xa7/xb9/xa1 TAMIL GLYPH NGOO +% /xa7/xb9/xaa TAMIL GLYPH NGAU +% /xa7/xba TAMIL GLYPH CEE +% /xa7/xba/xa1 TAMIL GLYPH COO +% /xa7/xba/xaa TAMIL GLYPH CAU +% /xa7/xbb TAMIL GLYPH NYEE +% /xa7/xbb/xa1 TAMIL GLYPH NYOO +% /xa7/xbb/xaa TAMIL GLYPH NYAU +% /xa7/xbc TAMIL GLYPH TTEE +% /xa7/xbc/xa1 TAMIL GLYPH TTOO +% /xa7/xbc/xaa TAMIL GLYPH TTAU +% /xa7/xbd TAMIL GLYPH NNEE +% /xa7/xbd/xa1 TAMIL GLYPH NNOO +% /xa7/xbd/xaa TAMIL GLYPH NNAU +% /xa7/xbe TAMIL GLYPH TEE +% /xa7/xbe/xa1 TAMIL GLYPH TOO +% /xa7/xbe/xaa TAMIL GLYPH TAU +% /xa7/xbf TAMIL GLYPH NEE +% /xa7/xbf/xa1 TAMIL GLYPH NOO +% /xa7/xbf/xaa TAMIL GLYPH NAU +% /xa7/xc0 TAMIL GLYPH PEE +% /xa7/xc0/xa1 TAMIL GLYPH POO +% /xa7/xc0/xaa TAMIL GLYPH PAU +% /xa7/xc1 TAMIL GLYPH MEE +% /xa7/xc1/xa1 TAMIL GLYPH MOO +% /xa7/xc1/xaa TAMIL GLYPH MAU +% /xa7/xc2 TAMIL GLYPH YEE +% /xa7/xc2/xa1 TAMIL GLYPH YOO +% /xa7/xc2/xaa TAMIL GLYPH YAU +% /xa7/xc3 TAMIL GLYPH REE +% /xa7/xc3/xa1 TAMIL GLYPH ROO +% /xa7/xc3/xaa TAMIL GLYPH RAU +% /xa7/xc4 TAMIL GLYPH LEE +% /xa7/xc4/xa1 TAMIL GLYPH LOO +% /xa7/xc4/xaa TAMIL GLYPH LAU +% /xa7/xc5 TAMIL GLYPH VEE +% /xa7/xc5/xa1 TAMIL GLYPH VOO +% /xa7/xc5/xaa TAMIL GLYPH VAU +% /xa7/xc6 TAMIL GLYPH LLLEE +% /xa7/xc6/xa1 TAMIL GLYPH LLLOO +% /xa7/xc6/xaa TAMIL GLYPH LLLAU +% /xa7/xc7 TAMIL GLYPH LLEE +% /xa7/xc7/xa1 TAMIL GLYPH LLOO +% /xa7/xc7/xaa TAMIL GLYPH LLAU +% /xa7/xc8 TAMIL GLYPH RREE +% /xa7/xc8/xa1 TAMIL GLYPH RROO +% /xa7/xc8/xaa TAMIL GLYPH RRAU +% /xa7/xc9 TAMIL GLYPH NNNEE +% /xa7/xc9/xa1 TAMIL GLYPH NNNOO +% /xa7/xc9/xaa TAMIL GLYPH NNNAU /xa8 TAMIL VOWEL SIGN AI - /xa8/xb8 TAMIL GLYPH KA - /xa8/xb9 TAMIL GLYPH NGA - /xa8/xba TAMIL GLYPH CA - /xa8/xbb TAMIL GLYPH NYA - /xa8/xbc TAMIL GLYPH TTA - /xa8/xbd TAMIL GLYPH NNA - /xa8/xbe TAMIL GLYPH TA - /xa8/xbf TAMIL GLYPH NA - /xa8/xc0 TAMIL GLYPH PA - /xa8/xc1 TAMIL GLYPH MA - /xa8/xc2 TAMIL GLYPH YA - /xa8/xc3 TAMIL GLYPH RA - /xa8/xc4 TAMIL GLYPH LA - /xa8/xc5 TAMIL GLYPH VA - /xa8/xc6 TAMIL GLYPH LLLA - /xa8/xc7 TAMIL GLYPH LLA - /xa8/xc8 TAMIL GLYPH RRA - /xa8/xc9 TAMIL GLYPH NNNA +% /xa8/xb8 TAMIL GLYPH KA +% /xa8/xb9 TAMIL GLYPH NGA +% /xa8/xba TAMIL GLYPH CA +% /xa8/xbb TAMIL GLYPH NYA +% /xa8/xbc TAMIL GLYPH TTA +% /xa8/xbd TAMIL GLYPH NNA +% /xa8/xbe TAMIL GLYPH TA +% /xa8/xbf TAMIL GLYPH NA +% /xa8/xc0 TAMIL GLYPH PA +% /xa8/xc1 TAMIL GLYPH MA +% /xa8/xc2 TAMIL GLYPH YA +% /xa8/xc3 TAMIL GLYPH RA +% /xa8/xc4 TAMIL GLYPH LA +% /xa8/xc5 TAMIL GLYPH VA +% /xa8/xc6 TAMIL GLYPH LLLA +% /xa8/xc7 TAMIL GLYPH LLA +% /xa8/xc8 TAMIL GLYPH RRA +% /xa8/xc9 TAMIL GLYPH NNNA /xa9 COPYRIGHT SIGN /xaa TAMIL AU LENGTH MARK /xab TAMIL LETTER A /xac TAMIL LETTER AA -%IRREVERSIBLE% /xad TAMIL LETTER I +% In TSCII 1.7 the hex value for TAMIL LETTER I was moved from /xad +% to /xfe, thus we leave the next line commented out. +% +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. +%IRREVERSIBLE% /xad TAMIL LETTER I /xae TAMIL LETTER II /xaf TAMIL LETTER U /xb0 TAMIL LETTER UU @@ -326,63 +359,57 @@ CHARMAP /xc7 TAMIL LETTER LLA /xc8 TAMIL LETTER RRA /xc9 TAMIL LETTER NNNA - /xca TAMIL GLYPH TI - /xcb TAMIL GLYPH TII - /xcc TAMIL GLYPH KU - /xcd TAMIL GLYPH CU - /xce TAMIL GLYPH TTU - /xcf TAMIL GLYPH NNU - /xd0 TAMIL GLYPH TU - /xd1 TAMIL GLYPH NU - /xd2 TAMIL GLYPH PU - /xd3 TAMIL GLYPH MU - /xd4 TAMIL GLYPH YU - /xd5 TAMIL GLYPH RU - /xd6 TAMIL GLYPH LU - /xd7 TAMIL GLYPH VU - /xd8 TAMIL GLYPH LLLU - /xd9 TAMIL GLYPH LLU - /xda TAMIL GLYPH RRU - /xdb TAMIL GLYPH NNNU - /xdc TAMIL GLYPH KUU - /xdd TAMIL GLYPH CUU - /xde TAMIL GLYPH TTUU - /xdf TAMIL GLYPH NNUU - /xe0 TAMIL GLYPH TUU - /xe1 TAMIL GLYPH NUU - /xe2 TAMIL GLYPH PUU - /xe3 TAMIL GLYPH MUU - /xe4 TAMIL GLYPH YUU - /xe5 TAMIL GLYPH RUU - /xe6 TAMIL GLYPH LUU - /xe7 TAMIL GLYPH VUU - /xe8 TAMIL GLYPH LLLUU - /xe9 TAMIL GLYPH LLUU - /xea TAMIL GLYPH RRUU - /xeb TAMIL GLYPH NNNUU - /xec TAMIL GLYPH K - /xed TAMIL GLYPH NG - /xee TAMIL GLYPH C - /xef TAMIL GLYPH NY - /xf0 TAMIL GLYPH TT - /xf1 TAMIL GLYPH NN - /xf2 TAMIL GLYPH T - /xf3 TAMIL GLYPH N - /xf4 TAMIL GLYPH P - /xf5 TAMIL GLYPH M - /xf6 TAMIL GLYPH Y - /xf7 TAMIL GLYPH R - /xf8 TAMIL GLYPH L - /xf9 TAMIL GLYPH V - /xfa TAMIL GLYPH LLL - /xfb TAMIL GLYPH LL - /xfc TAMIL GLYPH RR - /xfd TAMIL GLYPH NNN +% /xca TAMIL GLYPH TI +% /xcb TAMIL GLYPH TII +% /xcc TAMIL GLYPH KU +% /xcd TAMIL GLYPH CU +% /xce TAMIL GLYPH TTU +% /xcf TAMIL GLYPH NNU +% /xd0 TAMIL GLYPH TU +% /xd1 TAMIL GLYPH NU +% /xd2 TAMIL GLYPH PU +% /xd3 TAMIL GLYPH MU +% /xd4 TAMIL GLYPH YU +% /xd5 TAMIL GLYPH RU +% /xd6 TAMIL GLYPH LU +% /xd7 TAMIL GLYPH VU +% /xd8 TAMIL GLYPH LLLU +% /xd9 TAMIL GLYPH LLU +% /xda TAMIL GLYPH RRU +% /xdb TAMIL GLYPH NNNU +% /xdc TAMIL GLYPH KUU +% /xdd TAMIL GLYPH CUU +% /xde TAMIL GLYPH TTUU +% /xdf TAMIL GLYPH NNUU +% /xe0 TAMIL GLYPH TUU +% /xe1 TAMIL GLYPH NUU +% /xe2 TAMIL GLYPH PUU +% /xe3 TAMIL GLYPH MUU +% /xe4 TAMIL GLYPH YUU +% /xe5 TAMIL GLYPH RUU +% /xe6 TAMIL GLYPH LUU +% /xe7 TAMIL GLYPH VUU +% /xe8 TAMIL GLYPH LLLUU +% /xe9 TAMIL GLYPH LLUU +% /xea TAMIL GLYPH RRUU +% /xeb TAMIL GLYPH NNNUU +% /xec TAMIL GLYPH K +% /xed TAMIL GLYPH NG +% /xee TAMIL GLYPH C +% /xef TAMIL GLYPH NY +% /xf0 TAMIL GLYPH TT +% /xf1 TAMIL GLYPH NN +% /xf2 TAMIL GLYPH T +% /xf3 TAMIL GLYPH N +% /xf4 TAMIL GLYPH P +% /xf5 TAMIL GLYPH M +% /xf6 TAMIL GLYPH Y +% /xf7 TAMIL GLYPH R +% /xf8 TAMIL GLYPH L +% /xf9 TAMIL GLYPH V +% /xfa TAMIL GLYPH LLL +% /xfb TAMIL GLYPH LL +% /xfc TAMIL GLYPH RR +% /xfd TAMIL GLYPH NNN /xfe TAMIL LETTER I END CHARMAP - -WIDTH - 0 - 0 - 0 -END WIDTH diff --git a/localedata/charmaps/WINDOWS-31J b/localedata/charmaps/WINDOWS-31J index d5ab12abef..63e987f4ac 100644 --- a/localedata/charmaps/WINDOWS-31J +++ b/localedata/charmaps/WINDOWS-31J @@ -734,16 +734,19 @@ CHARMAP /x87/x8d SQUARE ERA NAME MEIZI /x87/x8e SQUARE ERA NAME TAISYOU /x87/x8f SQUARE ERA NAME SYOUWA +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /x87/x90 APPROXIMATELY EQUAL TO OR THE IMAGE OF %IRREVERSIBLE% /x87/x91 IDENTICAL TO %IRREVERSIBLE% /x87/x92 INTEGRAL /x87/x93 CONTOUR INTEGRAL /x87/x94 N-ARY SUMMATION +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /x87/x95 SQUARE ROOT %IRREVERSIBLE% /x87/x96 UP TACK %IRREVERSIBLE% /x87/x97 ANGLE /x87/x98 RIGHT ANGLE /x87/x99 RIGHT TRIANGLE +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /x87/x9a BECAUSE %IRREVERSIBLE% /x87/x9b INTERSECTION %IRREVERSIBLE% /x87/x9c UNION @@ -7167,7 +7170,7 @@ CHARMAP /xea/xa2 /xea/xa3 /xea/xa4 - +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /xed/x40 %IRREVERSIBLE% /xed/x41 %IRREVERSIBLE% /xed/x42 @@ -9434,6 +9437,7 @@ CHARMAP /xfa/x47 SMALL ROMAN NUMERAL EIGHT /xfa/x48 SMALL ROMAN NUMERAL NINE /xfa/x49 SMALL ROMAN NUMERAL TEN +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /xfa/x4a ROMAN NUMERAL ONE %IRREVERSIBLE% /xfa/x4b ROMAN NUMERAL TWO %IRREVERSIBLE% /xfa/x4c ROMAN NUMERAL THREE @@ -9448,6 +9452,7 @@ CHARMAP /xfa/x55 FULLWIDTH BROKEN BAR /xfa/x56 FULLWIDTH APOSTROPHE /xfa/x57 FULLWIDTH QUOTATION MARK +% The "IRREVERSIBLE" markup is used by iconv testing. Please do not remove. %IRREVERSIBLE% /xfa/x58 PARENTHESIZED IDEOGRAPH STOCK %IRREVERSIBLE% /xfa/x59 NUMERO SIGN %IRREVERSIBLE% /xfa/x5a TELEPHONE SIGN