Locales: Cyrillic -> ASCII transliteration table [BZ #2872] v2
Commit Message
Dear locale maintainers,
fix the glibc bug 2872 "Transliteration Cyrillic -> ASCII fails"
https://sourceware.org/bugzilla/show_bug.cgi?id=2872 [1]
add Cyrillic transliteration table translit_cyrillic file
https://sourceware.org/bugzilla/attachment.cgi?id=8591 [7]
to localedata/locales/ and include it in all your locales going forward.
Patch included inline below.
From this patch I have excluded locales that already mention cyrillic or
have a transliteration table for it:
az_AZ
iso14651_t1_common
ky_KG
mn_MN
sr_RS
tg_TJ
tk_TM
tt_RU
uk_UA
uz_UZ
uz_UZ@cyrillic
Their maintainers are requested to make an explicit decision on how and
whether at all to include this patch.
Current bug effect:
The glibc wiki explicitly lists this use case as the test example
https://sourceware.org/glibc/wiki/Locales#Testing_Locales :
LC_ALL=$LOCALE.UTF-8 iconv -f UTF-8 -t ASCII//TRANSLIT <
translit-test-input.txt
currently it fails on Cyrillic texts in most locales including ru_RU [1]
[8] [9]:
LC_ALL=ru_RU.UTF-8 iconv -f UTF-8 -t ASCII//TRANSLIT <
translit-test-input.txt |grep CYRILLIC
CYRILLIC ????? ??? ???? ?????? ??????????? ?????, ?? ????? ?? ???.
- It produces a string of question marks and spaces.
This is what it should produce and it does so after the patch applied:
CYRILLIC S``esh` eshhyo e`tix myagkix franczuzskix bulok, da vy'pej zhe
chayu.
Root problem and the fix:
The root problem is the missing transliteration table that I am
supplying here. Furthermore it has to be referenced/included into the
active locale at the compilation time to be used by iconv.
COMMIT MESSAGE:
This translit_cyrillic table enables conversion (e.g. with iconv) from a
UTF-8 encoded text based on Cyrillic alphabet to a ASCII//TRANSLIT text.
Examples: iconv -f UTF-8 -t ASCII//TRANSLIT will produce ASCII
compatible transcription and iconv -f UTF-8 -t ISO-8859-15//TRANSLIT |
iconv -f ISO-8859-15 -t UTF-8 will produce Latin transliteration as per
ISO 9.1995.
While a UTF-encoded Cyrillic text requires Cyrillic fonts the result of
a transliteration/transcription has only Latin/ASCII codes but still can
be read by a native speaker. Among other things it is useful for
processing the Cyrillic texts and filenames by programs or on systems
that are not specifically prepared to work with Cyrillic, don't have
corresponding fonts installed or can't handle UTF-8.
The transliteration table itself is attached as a file translit_cyrillic
[7]. Its content (mapping) is based on ISO 9.1995 standard [10] and its
derivative GOST 7.79-2000 official source (Federal Agency on Technical
Regulating and Metrology Of Russian Federation [2]). Technically an
independent but mostly identical source [3] was used and prepared in a
spreadsheet [6].
The documentation suggests that the transliteration tables inclusion is
done by adding *include "translit_cyrillic";""* string into LC_CTYPE
translit_start section
http://man7.org/linux/man-pages/man5/locale.5.html [5]
Practically I have searched for all locales that have a
translit_start/end stance and generated a patch for them.
The Cyrillic transliteration of e.g. Russian text may have already
worked to some extent for mn_MN, sr_RS, tk_TM, uz_UZ, uk_UA locales that
have their transliteration tables included inline.
I am excluding these locales from this proposed patch. I have written
directly to locale maintainer emails listed in the files. Volodymyr
Lisivka <vlisivka@gmail.com>, Max Kutny <mkutny@gmail.com> (uk_UA),
Данило Шеган <danilo@gnome.org> (sr_YU, sr_CS) have confirmed the
exclusion.
Links:
[1] This bug entry https://sourceware.org/bugzilla/show_bug.cgi?id=2872
[2] GOST 7.79-2000 official source
http://protect.gost.ru/document.aspx?control=7&id=130715 (is only
available in low quality gif format)
[3] http://transliteration.ru/gost-7-79-2000/ and
http://www.yfermer.ru/specifications/285821.html
[4] Wikipedia article on Cyrillic transliteration with Latin alphabet
https://ru.wikipedia.org/wiki/%D0%A2%D1%80%D0%B0%D0%BD%D1%81%D0%BB%D0%B8%D1%82%D0%B5%D1%80%D0%B0%D1%86%D0%B8%D1%8F_%D1%80%D1%83%D1%81%D1%81%D0%BA%D0%BE%D0%B3%D0%BE_%D0%B0%D0%BB%D1%84%D0%B0%D0%B2%D0%B8%D1%82%D0%B0_%D0%BB%D0%B0%D1%82%D0%B8%D0%BD%D0%B8%D1%86%D0%B5%D0%B9
[5] http://man7.org/linux/man-pages/man5/locale.5.html
[6] Spreadsheet for generating translit_cyrillic
https://sourceware.org/bugzilla/attachment.cgi?id=11301
[7] translit_cyrillic
https://sourceware.org/bugzilla/attachment.cgi?id=11302
[8] https://sourceware.org/glibc/wiki/Locales#Testing_Locales
[9] translit-test-input.txt
https://sourceware.org/bugzilla/attachment.cgi?id=11304
[10] https://en.wikipedia.org/wiki/ISO_9#ISO_9:1995,_or_GOST_7.79_System_A
Best regards,
Egor Kobylkin
---
2018-10-11 Egor Kobylkin <egor@kobylkin.com>
[BZ #2872]
* localedata/locales/translit_cyrillic: add ISO 9.1995, GOST 7.79
System A transliteration System B transcription table from Cyrillic to
Latin/ASCII.
* localedata/locales/C: add include "translit_cyrillic";"" to LC_CTYPE
translit section.
* localedata/locales/aa_DJ: Likewise.
* localedata/locales/af_ZA: Likewise.
* localedata/locales/ak_GH: Likewise.
* localedata/locales/am_ET: Likewise.
* localedata/locales/ar_EG: Likewise.
* localedata/locales/be_BY: Likewise.
* localedata/locales/bem_ZM: Likewise.
* localedata/locales/ber_DZ: Likewise.
* localedata/locales/ber_MA: Likewise.
* localedata/locales/bg_BG: Likewise.
* localedata/locales/bi_VU: Likewise.
* localedata/locales/bn_BD: Likewise.
* localedata/locales/bo_CN: Likewise.
* localedata/locales/ca_ES: Likewise.
* localedata/locales/ce_RU: Likewise.
* localedata/locales/cmn_TW: Likewise.
* localedata/locales/cs_CZ: Likewise.
* localedata/locales/cv_RU: Likewise.
* localedata/locales/cy_GB: Likewise.
* localedata/locales/da_DK: Likewise.
* localedata/locales/de_DE: Likewise.
* localedata/locales/dv_MV: Likewise.
* localedata/locales/dz_BT: Likewise.
* localedata/locales/el_GR: Likewise.
* localedata/locales/en_GB: Likewise.
* localedata/locales/en_NG: Likewise.
* localedata/locales/en_ZM: Likewise.
* localedata/locales/es_CU: Likewise.
* localedata/locales/es_ES: Likewise.
* localedata/locales/et_EE: Likewise.
* localedata/locales/fa_IR: Likewise.
* localedata/locales/ff_SN: Likewise.
* localedata/locales/fi_FI: Likewise.
* localedata/locales/fr_FR: Likewise.
* localedata/locales/ga_IE: Likewise.
* localedata/locales/gd_GB: Likewise.
* localedata/locales/gu_IN: Likewise.
* localedata/locales/gv_GB: Likewise.
* localedata/locales/he_IL: Likewise.
* localedata/locales/hi_IN: Likewise.
* localedata/locales/hif_FJ: Likewise.
* localedata/locales/hr_HR: Likewise.
* localedata/locales/ht_HT: Likewise.
* localedata/locales/hu_HU: Likewise.
* localedata/locales/hy_AM: Likewise.
* localedata/locales/id_ID: Likewise.
* localedata/locales/is_IS: Likewise.
* localedata/locales/it_IT: Likewise.
* localedata/locales/ja_JP: Likewise.
* localedata/locales/kab_DZ: Likewise.
* localedata/locales/kk_KZ: Likewise.
* localedata/locales/km_KH: Likewise.
* localedata/locales/kn_IN: Likewise.
* localedata/locales/ko_KR: Likewise.
* localedata/locales/ks_IN: Likewise.
* localedata/locales/kw_GB: Likewise.
* localedata/locales/lb_LU: Likewise.
* localedata/locales/lg_UG: Likewise.
* localedata/locales/lij_IT: Likewise.
* localedata/locales/ln_CD: Likewise.
* localedata/locales/lo_LA: Likewise.
* localedata/locales/lt_LT: Likewise.
* localedata/locales/lv_LV: Likewise.
* localedata/locales/mg_MG: Likewise.
* localedata/locales/mhr_RU: Likewise.
* localedata/locales/mk_MK: Likewise.
* localedata/locales/ml_IN: Likewise.
* localedata/locales/ms_MY: Likewise.
* localedata/locales/mt_MT: Likewise.
* localedata/locales/nan_TW@latin: Likewise.
* localedata/locales/nb_NO: Likewise.
* localedata/locales/ne_NP: Likewise.
* localedata/locales/nhn_MX: Likewise.
* localedata/locales/niu_NU: Likewise.
* localedata/locales/niu_NZ: Likewise.
* localedata/locales/nl_NL: Likewise.
* localedata/locales/nr_ZA: Likewise.
* localedata/locales/oc_FR: Likewise.
* localedata/locales/om_KE: Likewise.
* localedata/locales/or_IN: Likewise.
* localedata/locales/os_RU: Likewise.
* localedata/locales/pa_IN: Likewise.
* localedata/locales/pa_PK: Likewise.
* localedata/locales/pl_PL: Likewise.
* localedata/locales/pt_PT: Likewise.
* localedata/locales/quz_PE: Likewise.
* localedata/locales/ro_RO: Likewise.
* localedata/locales/ru_RU: Likewise.
* localedata/locales/rw_RW: Likewise.
* localedata/locales/sa_IN: Likewise.
* localedata/locales/sd_IN: Likewise.
* localedata/locales/sd_IN@devanagari: Likewise.
* localedata/locales/sd_PK: Likewise.
* localedata/locales/se_NO: Likewise.
* localedata/locales/sgs_LT: Likewise.
* localedata/locales/shn_MM: Likewise.
* localedata/locales/si_LK: Likewise.
* localedata/locales/sk_SK: Likewise.
* localedata/locales/sl_SI: Likewise.
* localedata/locales/sm_WS: Likewise.
* localedata/locales/so_SO: Likewise.
* localedata/locales/sq_AL: Likewise.
* localedata/locales/ss_ZA: Likewise.
* localedata/locales/st_ZA: Likewise.
* localedata/locales/sv_SE: Likewise.
* localedata/locales/sw_KE: Likewise.
* localedata/locales/ta_IN: Likewise.
* localedata/locales/te_IN: Likewise.
* localedata/locales/th_TH: Likewise.
* localedata/locales/ti_ET: Likewise.
* localedata/locales/tn_ZA: Likewise.
* localedata/locales/to_TO: Likewise.
* localedata/locales/tpi_PG: Likewise.
* localedata/locales/tr_TR: Likewise.
* localedata/locales/ts_ZA: Likewise.
* localedata/locales/unm_US: Likewise.
* localedata/locales/ur_IN: Likewise.
* localedata/locales/ur_PK: Likewise.
* localedata/locales/ve_ZA: Likewise.
* localedata/locales/vi_VN: Likewise.
* localedata/locales/wa_BE: Likewise.
* localedata/locales/wo_SN: Likewise.
* localedata/locales/xh_ZA: Likewise.
* localedata/locales/yi_US: Likewise.
* localedata/locales/yuw_PG: Likewise.
* localedata/locales/zh_CN: Likewise.
* localedata/locales/zu_ZA: Likewise.
Comments
Hi,
Looks like there's one rule after all which might be debatable, I'll
just highlight it and let others to comment and decide what to do with it.
On 2018-10-11 01:29, Egor Kobylkin wrote:
>
> +% RIGHT SINGLE QUOTATION MARK
> +<U2019> <U2035>;<U0027>
translit_neutral (which is included by i18n) has:
% RIGHT SINGLE QUOTATION MARK
<U2019> <U0027> % not <U00B4> because it's often used as an apostrophe
In practice the end result might well be the same (since if U+2019 is
not available then probably U+2035 is neither and both rules produce
U+0027). However, given that translit_cyrillic would be included in
every locale, I'm not sure is this kind of minor discrepancy ok or not.
Thanks,
Thank you, Egor. I am looking at your patch and although I have
not yet finished, here are some remarks:
First of all, I think that such a large patch should also include
the tests. Please see how automatic tests are performed in locale
data and write your own.
11.10.2018 00:29 Egor Kobylkin <egor@kobylkin.com> wrote:
> [...]
> From this patch I have excluded locales that already mention cyrillic or
> have a transliteration table for it:
> az_AZ
> iso14651_t1_common
> ky_KG
> mn_MN
> sr_RS
> tg_TJ
> tk_TM
> tt_RU
> uk_UA
> uz_UZ
> uz_UZ@cyrillic
> [...]
I think that eventually we would like to include your translit_cyrillic
also in these locales because I assume that your rules should work good
for them as well, also should include more characters than the individual
language contributors took into account. Similarly to Mike's work on
collation: a common rules were created and all locales include them adding
their own language specific modifications.
> [...]
> COMMIT MESSAGE:
> [...]
> I am excluding these locales from this proposed patch. I have written
> directly to locale maintainer emails listed in the files. Volodymyr
> Lisivka <vlisivka@gmail.com>, Max Kutny <mkutny@gmail.com> (uk_UA),
> Данило Шеган <danilo@gnome.org> (sr_YU, sr_CS) have confirmed the
I am not sure if we want Cyrillic text in the commit message. Shouldn't
it be, uhm, tranlisterated? :-)
"sr_CS" - I guess you meant "sr_RS".
"sr_YU" has been dropped, do we want to mention it?
> [...]
> [BZ #2872]
> * localedata/locales/translit_cyrillic: add ISO 9.1995, GOST 7.79
Please start "Add" with an uppercase. BTW, shouldn't it be "New file"
instead?
> System A transliteration System B transcription table from Cyrillic to
> Latin/ASCII.
> * localedata/locales/C: add include "translit_cyrillic";"" to LC_CTYPE
> translit section.
Same, "Add" here.
> * localedata/locales/aa_DJ: Likewise.
Good (here and everywhere below).
> [...]
> diff -uNr a/localedata/locales/translit_cyrillic
> b/localedata/locales/translit_cyrillic
> --- a/localedata/locales/translit_cyrillic 1970-01-01 00:00:00.000000000
> +0000
> +++ b/localedata/locales/translit_cyrillic 2018-10-09 19:02:54.000000000
> +0000
> @@ -0,0 +1,383 @@
> +escape_char /
> +comment_char %
> +
> +% This file is part of the GNU C Library and contains locale data.
> +% The Free Software Foundation does not claim any copyright interest
> +% in the locale data contained in this file. The foregoing does not
> +% affect the license of the GNU C Library as a whole. It does not
> +% exempt you from the conditions of the license if your use would
> +% otherwise be governed by that license.
> +
> +% Transliterations of cyrillic letters to latin and/or ascii symbols.
"cyrillic" -> "Cyrillic"; "latin" -> "Latin"; "ascii" -> "ASCII".
> +% Inspired by ISO 9.1995 / GOST 7.79-2000.
> +% Covers Unicode Range https://www.unicode.org/charts/PDF/U0400.pdf
> +% i.e [U4001-U4F9, U2019] but only the letters covered by ISO 9.1995
Typos:
"i.e" -> "i.e.," (somebody please fix me if I'm wrong here)
"U4001" - I guess you meant "U0401"
"U4F9" -> "U04F9". I think that "U4F9" is not definitely bad but
let's be consistent.
Also I can see some gaps in the range. Are you going to fill them
or maybe for now just mention that they exist?
> +% It implements the GOST_7.79 System A (Latin Script) as a first
> +% option and System B Cyrillic (ASCII) as a second option. Check
> +% https://en.wikipedia.org/wiki/ISO_9 for reference.
> +% The System B is extended from GOST_7.79-Russian using open sources
> +% of the transliteration mappings and the "h/`" diacritics logic.
What is "h/`" diacritics logic?
> +
> +% Usage examples:
> +% iconv -f UTF-8 -t ISO-8859-15//TRANSLIT \
> +% | iconv -f ISO-8859-15 -t UTF-8 # System A
> +% iconv -f UTF-8 -t ASCII//TRANSLIT # System B.
> +
> +% Contributions welcome for the rest of Cyrillic script in Unicode
Sure, I'm not going to stop you from pushing these changes just because
there are missing characters. I will consider adding them later.
> +% https://en.wikipedia.org/wiki/Cyrillic_script_in_Unicode.
> +% Bugfix for https://sourceware.org/bugzilla/show_bug.cgi?id=2872.
> +% Generated from UnicodeData.txt with
> +% https://sourceware.org/bugzilla/attachment.cgi?id=11301.
1. Is the file really generated with a script and not modified later?
If yes then maybe you should contribute the script instead? In that case,
you should also not post this file to libc-locale, maintainers and
developers should be able to regenerate it.
2. The link leads to a LibreOffice spreadsheet.
> +LC_CTYPE
> +
> +translit_start
> +
<U0400> is missing here. Are you going to leave it for now?
> +% CYRILLIC CAPITAL LETTER IO
> +<U0401> <U00CB>;"<U0059><U004F>"
> [...]
> +% CYRILLIC CAPITAL LETTER KJE
> +<U040C> <U1E30>;"<U004B><U0060>"
<U040D> is missing here. Can we add it already?
> +% CYRILLIC CAPITAL LETTER SHORT U
> +<U040E> <U016C>;"<U0055><U0060>"
> [...]
> +% CYRILLIC CAPITAL LETTER U
> +<U0423> <U0055>
> +% CYRILLIC UNDEFINED
> +<U0423><U0301> <U00DA>;"<U0055><U0060>"
This still makes me wonder.
Does it work at all?
What if we remove this rule, won't it be transliterated as
<U0423> => "U", <U0301> - left unchanged, so "U" + <U0301>"
will eventually produce "Ú"?
Why is it called "UNDEFINED"?
Do we need similar rules for other characters?
> [...]
> +% CYRILLIC SMALL LETTER U
> +<U0443> <U0075>
> +% CYRILLIC UNDEFINED
> +<U0443><U0301> <U00FA>;"<U0075><U0060>"
Same here.
> [...]
> +% CYRILLIC SMALL LETTER YA
> +<U044F> <U00E2>;"<U0079><U0061>"
Again <U0450> missing (because it is lowercase variant of <U0400>).
> +% CYRILLIC SMALL LETTER IO
> +<U0451> <U00EB>;"<U0079><U006F>"
> [...]
> +% CYRILLIC SMALL LETTER KJE
> +<U045C> <U1E31>;"<U006B><U0060>"
<U045D> missing (same reason as <U040D>).
> +% CYRILLIC SMALL LETTER SHORT U
> +<U045E> <U016D>;"<U0075><U0060>"
> +% CYRILLIC SMALL LETTER DZHE
> +<U045F> "<U0064><U0302>";"<U0064><U0068>"
More letters missing here. Is this because they are historic so we
don't want to include them now? Well, but "YUS" is also historic.
(Please, do not remove YUS for consistency).
> +% CYRILLIC CAPITAL LETTER BIG YUS
> +<U046A> <U01CD>;"<U004F><U0060>"
> +% CYRILLIC SMALL LETTER BIG YUS
> +<U046B> <U01CE>;"<U006F><U0060>"
> [...]
I will continue but, again, I don't give any ETA so other reviewers
are welcome here.
Regards,
Rafal
Hi,
On 2018-10-11 14:04, Rafal Luzynski wrote:
>
> First of all, I think that such a large patch should also include
> the tests. Please see how automatic tests are performed in locale
> data and write your own.
>
> 11.10.2018 00:29 Egor Kobylkin <egor@kobylkin.com> wrote:
>
> Also I can see some gaps in the range. Are you going to fill them
> or maybe for now just mention that they exist?
>
> <U040D> is missing here. Can we add it already?
>
> Sure, I'm not going to stop you from pushing these changes just because
> there are missing characters. I will consider adding them later.
>
> <U0400> is missing here. Are you going to leave it for now?
See check https://sourceware.org/ml/libc-alpha/2018-10/msg00160.html.
>> +% CYRILLIC CAPITAL LETTER U
>> +<U0423> <U0055>
>> +% CYRILLIC UNDEFINED
>> +<U0423><U0301> <U00DA>;"<U0055><U0060>"
>
> This still makes me wonder.
>
> Does it work at all?
No, see the above link.
More importantly, I realized that ICU uconv(1) I mentioned earlier
should make a great reference for this data; output of the currently
included transliteration rules should match uconv(1) output. If that is
not the case, the patch or uconv(1) might have an issue. If the outputs
match, then we should be able to safely assume the patch is ok.
It could also be considered to use uconv(1) output as reference how the
handle to currently missing characters.
(uconv(1) is part of the icu package on Fedora/CentOS/RHEL/openSUSE.)
Thanks,
чт, 11 жовт. 2018 о 14:05 Rafal Luzynski <digitalfreak@lingonborough.com> пише:
>
> Thank you, Egor. I am looking at your patch and although I have
> not yet finished, here are some remarks:
>
> First of all, I think that such a large patch should also include
> the tests. Please see how automatic tests are performed in locale
> data and write your own.
>
> 11.10.2018 00:29 Egor Kobylkin <egor@kobylkin.com> wrote:
> > [...]
> > From this patch I have excluded locales that already mention cyrillic or
> > have a transliteration table for it:
> > az_AZ
> > iso14651_t1_common
> > ky_KG
> > mn_MN
> > sr_RS
> > tg_TJ
> > tk_TM
> > tt_RU
> > uk_UA
> > uz_UZ
> > uz_UZ@cyrillic
> > [...]
>
> I think that eventually we would like to include your translit_cyrillic
> also in these locales because I assume that your rules should work good
> for them as well, also should include more characters than the individual
> language contributors took into account.
It's very good idea. Transliteration in Ukrainian locale predates this
work for about decade. It well tested. I also have automatic test
cases, which I can adapt to current standard. Let's drop Russian
transliteration rules and replace them with Ukrainian transliteration
rules. I assume that Ukrainian rules should work good for them as
well.
Ukrainian language is the oldest and most developed language in Slavic
family - last king of all Slavs named Madzhak/Muzhik (Brave), leader
of Volyniana union, was lived in Western Ukraine in Volyn` region.
After Madzhak capturing of Madzhak, kingdom was split into multiple
western parts and eastern part, where 9 Slavic tribes were united by
Rus` tribe, which abandoned their city, now known as Old Russa,
because of epidemic. IMHO, it's will be fair to use rules of the
oldest Slavic union.
> Similarly to Mike's work on
> collation: a common rules were created and all locales include them adding
> their own language specific modifications.
It's good idea too. In our own locale we prefer that words in our
language will be at top of a sorted list. Currently, in Ukrainian
locale it works as intended, but Russian locale has inverted order.
IMHO, Russian locale should use Ukrainian rules.
$ echo 'один два three four'| tr ' ' '\n' | LANG=uk_UA.utf8 sort
два
один
four
three
$ echo 'один два three four'| tr ' ' '\n' | LANG=ru_RU.utf8 sort
four
three
два
один
>
> > [...]
> > COMMIT MESSAGE:
> > [...]
> > I am excluding these locales from this proposed patch. I have written
> > directly to locale maintainer emails listed in the files. Volodymyr
> > Lisivka <vlisivka@gmail.com>, Max Kutny <mkutny@gmail.com> (uk_UA),
> > Данило Шеган <danilo@gnome.org> (sr_YU, sr_CS) have confirmed the
>
> I am not sure if we want Cyrillic text in the commit message. Shouldn't
> it be, uhm, tranlisterated? :-)
>
> "sr_CS" - I guess you meant "sr_RS".
>
> "sr_YU" has been dropped, do we want to mention it?
>
> > [...]
> > [BZ #2872]
> > * localedata/locales/translit_cyrillic: add ISO 9.1995, GOST 7.79
>
> Please start "Add" with an uppercase. BTW, shouldn't it be "New file"
> instead?
>
> > System A transliteration System B transcription table from Cyrillic to
> > Latin/ASCII.
> > * localedata/locales/C: add include "translit_cyrillic";"" to LC_CTYPE
> > translit section.
>
> Same, "Add" here.
>
> > * localedata/locales/aa_DJ: Likewise.
>
> Good (here and everywhere below).
>
> > [...]
> > diff -uNr a/localedata/locales/translit_cyrillic
> > b/localedata/locales/translit_cyrillic
> > --- a/localedata/locales/translit_cyrillic 1970-01-01 00:00:00.000000000
> > +0000
> > +++ b/localedata/locales/translit_cyrillic 2018-10-09 19:02:54.000000000
> > +0000
> > @@ -0,0 +1,383 @@
> > +escape_char /
> > +comment_char %
> > +
> > +% This file is part of the GNU C Library and contains locale data.
> > +% The Free Software Foundation does not claim any copyright interest
> > +% in the locale data contained in this file. The foregoing does not
> > +% affect the license of the GNU C Library as a whole. It does not
> > +% exempt you from the conditions of the license if your use would
> > +% otherwise be governed by that license.
> > +
> > +% Transliterations of cyrillic letters to latin and/or ascii symbols.
>
> "cyrillic" -> "Cyrillic"; "latin" -> "Latin"; "ascii" -> "ASCII".
>
> > +% Inspired by ISO 9.1995 / GOST 7.79-2000.
> > +% Covers Unicode Range https://www.unicode.org/charts/PDF/U0400.pdf
> > +% i.e [U4001-U4F9, U2019] but only the letters covered by ISO 9.1995
>
> Typos:
>
> "i.e" -> "i.e.," (somebody please fix me if I'm wrong here)
> "U4001" - I guess you meant "U0401"
> "U4F9" -> "U04F9". I think that "U4F9" is not definitely bad but
> let's be consistent.
>
> Also I can see some gaps in the range. Are you going to fill them
> or maybe for now just mention that they exist?
>
> > +% It implements the GOST_7.79 System A (Latin Script) as a first
> > +% option and System B Cyrillic (ASCII) as a second option. Check
> > +% https://en.wikipedia.org/wiki/ISO_9 for reference.
> > +% The System B is extended from GOST_7.79-Russian using open sources
> > +% of the transliteration mappings and the "h/`" diacritics logic.
>
> What is "h/`" diacritics logic?
>
> > +
> > +% Usage examples:
> > +% iconv -f UTF-8 -t ISO-8859-15//TRANSLIT \
> > +% | iconv -f ISO-8859-15 -t UTF-8 # System A
> > +% iconv -f UTF-8 -t ASCII//TRANSLIT # System B.
> > +
> > +% Contributions welcome for the rest of Cyrillic script in Unicode
>
> Sure, I'm not going to stop you from pushing these changes just because
> there are missing characters. I will consider adding them later.
>
> > +% https://en.wikipedia.org/wiki/Cyrillic_script_in_Unicode.
> > +% Bugfix for https://sourceware.org/bugzilla/show_bug.cgi?id=2872.
> > +% Generated from UnicodeData.txt with
> > +% https://sourceware.org/bugzilla/attachment.cgi?id=11301.
>
> 1. Is the file really generated with a script and not modified later?
> If yes then maybe you should contribute the script instead? In that case,
> you should also not post this file to libc-locale, maintainers and
> developers should be able to regenerate it.
> 2. The link leads to a LibreOffice spreadsheet.
>
> > +LC_CTYPE
> > +
> > +translit_start
> > +
>
> <U0400> is missing here. Are you going to leave it for now?
>
> > +% CYRILLIC CAPITAL LETTER IO
> > +<U0401> <U00CB>;"<U0059><U004F>"
> > [...]
> > +% CYRILLIC CAPITAL LETTER KJE
> > +<U040C> <U1E30>;"<U004B><U0060>"
>
> <U040D> is missing here. Can we add it already?
>
> > +% CYRILLIC CAPITAL LETTER SHORT U
> > +<U040E> <U016C>;"<U0055><U0060>"
> > [...]
> > +% CYRILLIC CAPITAL LETTER U
> > +<U0423> <U0055>
> > +% CYRILLIC UNDEFINED
> > +<U0423><U0301> <U00DA>;"<U0055><U0060>"
>
> This still makes me wonder.
>
> Does it work at all?
> What if we remove this rule, won't it be transliterated as
> <U0423> => "U", <U0301> - left unchanged, so "U" + <U0301>"
> will eventually produce "Ú"?
> Why is it called "UNDEFINED"?
> Do we need similar rules for other characters?
>
> > [...]
> > +% CYRILLIC SMALL LETTER U
> > +<U0443> <U0075>
> > +% CYRILLIC UNDEFINED
> > +<U0443><U0301> <U00FA>;"<U0075><U0060>"
>
> Same here.
>
> > [...]
> > +% CYRILLIC SMALL LETTER YA
> > +<U044F> <U00E2>;"<U0079><U0061>"
>
> Again <U0450> missing (because it is lowercase variant of <U0400>).
>
> > +% CYRILLIC SMALL LETTER IO
> > +<U0451> <U00EB>;"<U0079><U006F>"
> > [...]
> > +% CYRILLIC SMALL LETTER KJE
> > +<U045C> <U1E31>;"<U006B><U0060>"
>
> <U045D> missing (same reason as <U040D>).
>
> > +% CYRILLIC SMALL LETTER SHORT U
> > +<U045E> <U016D>;"<U0075><U0060>"
> > +% CYRILLIC SMALL LETTER DZHE
> > +<U045F> "<U0064><U0302>";"<U0064><U0068>"
>
> More letters missing here. Is this because they are historic so we
> don't want to include them now? Well, but "YUS" is also historic.
> (Please, do not remove YUS for consistency).
>
> > +% CYRILLIC CAPITAL LETTER BIG YUS
> > +<U046A> <U01CD>;"<U004F><U0060>"
> > +% CYRILLIC SMALL LETTER BIG YUS
> > +<U046B> <U01CE>;"<U006F><U0060>"
> > [...]
>
> I will continue but, again, I don't give any ETA so other reviewers
> are welcome here.
>
> Regards,
>
> Rafal
Hi Rafal
On 11.10.2018 13:04, Rafal Luzynski wrote:
> Thank you, Egor. I am looking at your patch and although I have
> not yet finished, here are some remarks:
>
> First of all, I think that such a large patch should also include
> the tests. Please see how automatic tests are performed in locale
> data and write your own.
Could you please point me to the existing automatic tests?
Locally I am using the test suggested in glibc locales wiki.
From my commit message:
"The glibc wiki explicitly lists this use case as the test example
https://sourceware.org/glibc/wiki/Locales#Testing_Locales :
LC_ALL=$LOCALE.UTF-8 iconv -f UTF-8 -t ASCII//TRANSLIT <
translit-test-input.txt
"
I am visually checking whether any iconv run fails for all those locales
but you must refer to some automated unit test with a boolean outcome,
right?
>
> 11.10.2018 00:29 Egor Kobylkin <egor@kobylkin.com> wrote:
>> [...]
>> From this patch I have excluded locales that already mention cyrillic or
>> have a transliteration table for it:
>> az_AZ
>> iso14651_t1_common
>> ky_KG
>> mn_MN
>> sr_RS
>> tg_TJ
>> tk_TM
>> tt_RU
>> uk_UA
>> uz_UZ
>> uz_UZ@cyrillic
>> [...]
>
> I think that eventually we would like to include your translit_cyrillic
> also in these locales because I assume that your rules should work good
> for them as well, also should include more characters than the individual
> language contributors took into account. Similarly to Mike's work on
> collation: a common rules were created and all locales include them adding
> their own language specific modifications.
This is fine with me. Should anybody supply translit_xxxxxxxxxxxx for
any of the mentioned locales we can include them as well. Wouldn't it be
easier to coordinate those as separate patches though?
>
>> [...]
>> COMMIT MESSAGE:
>> [...]
>> I am excluding these locales from this proposed patch. I have written
>> directly to locale maintainer emails listed in the files. Volodymyr
>> Lisivka <vlisivka@gmail.com>, Max Kutny <mkutny@gmail.com> (uk_UA),
>> Данило Шеган <danilo@gnome.org> (sr_YU, sr_CS) have confirmed the
>
> I am not sure if we want Cyrillic text in the commit message. Shouldn't
> it be, uhm, tranlisterated? :-)
I do not see any Cyrillic text in the commit message.
the ?????? you see are the actual "?" symbols coming out of iconv now.
>
> "sr_CS" - I guess you meant "sr_RS".
>
> "sr_YU" has been dropped, do we want to mention it?
The list of locales and the patch itself is generated from the actual
locales - I do not hand pick them, only exclude the ones in the
exclusion list above.
>
>> [...]
>> [BZ #2872]
>> * localedata/locales/translit_cyrillic: add ISO 9.1995, GOST 7.79
>
> Please start "Add" with an uppercase. BTW, shouldn't it be "New file"
> instead?
>
>> System A transliteration System B transcription table from Cyrillic to
>> Latin/ASCII.
>> * localedata/locales/C: add include "translit_cyrillic";"" to LC_CTYPE
>> translit section.
>
> Same, "Add" here.
>
>> * localedata/locales/aa_DJ: Likewise.
>
> Good (here and everywhere below).
>
>> [...]
>> diff -uNr a/localedata/locales/translit_cyrillic
>> b/localedata/locales/translit_cyrillic
>> --- a/localedata/locales/translit_cyrillic 1970-01-01 00:00:00.000000000
>> +0000
>> +++ b/localedata/locales/translit_cyrillic 2018-10-09 19:02:54.000000000
>> +0000
>> @@ -0,0 +1,383 @@
>> +escape_char /
>> +comment_char %
>> +
>> +% This file is part of the GNU C Library and contains locale data.
>> +% The Free Software Foundation does not claim any copyright interest
>> +% in the locale data contained in this file. The foregoing does not
>> +% affect the license of the GNU C Library as a whole. It does not
>> +% exempt you from the conditions of the license if your use would
>> +% otherwise be governed by that license.
>> +
>> +% Transliterations of cyrillic letters to latin and/or ascii symbols.
>
> "cyrillic" -> "Cyrillic"; "latin" -> "Latin"; "ascii" -> "ASCII".
>
>> +% Inspired by ISO 9.1995 / GOST 7.79-2000.
>> +% Covers Unicode Range https://www.unicode.org/charts/PDF/U0400.pdf
>> +% i.e [U4001-U4F9, U2019] but only the letters covered by ISO 9.1995
>
> Typos:
>
> "i.e" -> "i.e.," (somebody please fix me if I'm wrong here)
> "U4001" - I guess you meant "U0401"
> "U4F9" -> "U04F9". I think that "U4F9" is not definitely bad but
> let's be consistent.
These are all good catches. I will fix them and resubmit.
>
> Also I can see some gaps in the range. Are you going to fill them
> or maybe for now just mention that they exist?
>
No, were not going to fill them please see this:
On 10.10.2018 14:34, Marko Myllynen wrote:
> On 2018-10-10 15:19, Egor Kobylkin wrote:
>> On 10.10.2018 13:22, Marko Myllynen wrote:
>>>> correct link https://sourceware.org/bugzilla/attachment.cgi?id=11303
>>> Although I haven't checked every rule this in general looks very good
>>> (but see below).
>>> Not sure do we want to add the few missing characters
>>> mentioned at https://en.wikipedia.org/wiki/Cyrillic_script_in_Unicode,
>>> e.g., one instantly notices that U+0400 is missing. (I wouldn't add at
>>> least initially the more exotic characters, like the historic ones,
>>> though.) Perhaps filing a bug or two for these cases for separate
>>> consideration would be ok.
>> The question here is what should serve as their transliteration and
>> transcription?
> Not sure, so filing a separate bug about this once your patch is merged
> might be the most suitable action for now, I don't think we want to
> postpone merging your work further due to these non-ISO 9 cases.
>
>> +% It implements the GOST_7.79 System A (Latin Script) as a first
>> +% option and System B Cyrillic (ASCII) as a second option. Check
>> +% https://en.wikipedia.org/wiki/ISO_9 for reference.
>> +% The System B is extended from GOST_7.79-Russian using open sources
>> +% of the transliteration mappings and the "h/`" diacritics logic.
>
> What is "h/`" diacritics logic?
Basically some Linguist mentioned that they have chosen "h" and '`" to
represent the diacritics for the transcription (i.e. GOST 7.79 System
B). This way there is some resemblance to the watertight transliteration
as per ISO 9 (Sysetem A) but it is still all in ASCII. We have decided
to extend GOST 7.79 to the all ISO 9 characters and so I have extended
it following that Linguist logic.
>> +% https://en.wikipedia.org/wiki/Cyrillic_script_in_Unicode.
>> +% Bugfix for https://sourceware.org/bugzilla/show_bug.cgi?id=2872.
>> +% Generated from UnicodeData.txt with
>> +% https://sourceware.org/bugzilla/attachment.cgi?id=11301.
>
> 1. Is the file really generated with a script and not modified later?
> If yes then maybe you should contribute the script instead? In that case,
> you should also not post this file to libc-locale, maintainers and
> developers should be able to regenerate it.
> 2. The link leads to a LibreOffice spreadsheet.
No, I do not have a script. The "generated" means it is a result of
formulas in that spreadsheet. People are welcome to write a script that
should be straightforward implementation of those rules in formulas.
>
>> +LC_CTYPE
>> +
>> +translit_start
>> +
>
> <U0400> is missing here. Are you going to leave it for now?
Yes, it is to be left out, not in ISO 9. See the exchange with Marko above.
>
>> +% CYRILLIC CAPITAL LETTER IO
>> +<U0401> <U00CB>;"<U0059><U004F>"
>> [...]
>> +% CYRILLIC CAPITAL LETTER KJE
>> +<U040C> <U1E30>;"<U004B><U0060>"
>
> <U040D> is missing here. Can we add it already?
Yes, it is to be left out, not in ISO 9. See the exchange with Marko above.
>
>> +% CYRILLIC CAPITAL LETTER SHORT U
>> +<U040E> <U016C>;"<U0055><U0060>"
>> [...]
>> +% CYRILLIC CAPITAL LETTER U
>> +<U0423> <U0055>
>> +% CYRILLIC UNDEFINED
>> +<U0423><U0301> <U00DA>;"<U0055><U0060>"
>
> This still makes me wonder.
>
> Does it work at all?
> What if we remove this rule, won't it be transliterated as
> <U0423> => "U", <U0301> - left unchanged, so "U" + <U0301>"
> will eventually produce "Ú"?
> Why is it called "UNDEFINED"?
On 10.10.2018 14:34, Marko Myllynen wrote:
> On 2018-10-10 15:19, Egor Kobylkin wrote:
>> On 10.10.2018 13:22, Marko Myllynen wrote:
...
>>> I'm not sure this will work, no existing rule in translit_* files
>>> contain two characters, I'd assume that the rule for U+0423 is applied
>>> first and then the below rule is never used.
>>>
>>> % CYRILLIC UNDEFINED
>>> <U0423><U0301> <U00DA>;"<U0055><U0060>"
>>>
>>> Perhaps this should be commented out or removed altogether if it's not
>>> working as intended.
>>
>> So yes, they are not processed. I would drop them to not to have special
>> cases. But I am also fine with keeping them because all work is done
>> already.
> I'd probably drop them but I don't feel strongly about this either way.
>
> Thanks for your efforts, I don't have any further comments, I'll leave
> this now for Rafal and Mike to provide additional feedback and hopefully
> merge soon.
Could you also please check the discussion with Marko on UNDEFINED and
other related topics? You were on To: or CC: for those emails.
The same for the other characters below.
> Do we need similar rules for other characters?
>
>> [...]
>> +% CYRILLIC SMALL LETTER U
>> +<U0443> <U0075>
>> +% CYRILLIC UNDEFINED
>> +<U0443><U0301> <U00FA>;"<U0075><U0060>"
>
> Same here.
>
>> [...]
>> +% CYRILLIC SMALL LETTER YA
>> +<U044F> <U00E2>;"<U0079><U0061>"
>
> Again <U0450> missing (because it is lowercase variant of <U0400>).
>
>> +% CYRILLIC SMALL LETTER IO
>> +<U0451> <U00EB>;"<U0079><U006F>"
>> [...]
>> +% CYRILLIC SMALL LETTER KJE
>> +<U045C> <U1E31>;"<U006B><U0060>"
>
> <U045D> missing (same reason as <U040D>).
>
>> +% CYRILLIC SMALL LETTER SHORT U
>> +<U045E> <U016D>;"<U0075><U0060>"
>> +% CYRILLIC SMALL LETTER DZHE
>> +<U045F> "<U0064><U0302>";"<U0064><U0068>"
>
> More letters missing here. Is this because they are historic so we
> don't want to include them now? Well, but "YUS" is also historic.
> (Please, do not remove YUS for consistency).
>
>> +% CYRILLIC CAPITAL LETTER BIG YUS
>> +<U046A> <U01CD>;"<U004F><U0060>"
>> +% CYRILLIC SMALL LETTER BIG YUS
>> +<U046B> <U01CE>;"<U006F><U0060>"
>> [...]
>
> I will continue but, again, I don't give any ETA so other reviewers
> are welcome here.
>
> Regards,
>
> Rafal
>
Bests,
Egor
Hi,
On 2018-10-10 15:19, Egor Kobylkin wrote:
> On 10.10.2018 13:22, Marko Myllynen wrote:
>>> correct link https://sourceware.org/bugzilla/attachment.cgi?id=11303
>>
>> Although I haven't checked every rule this in general looks very good
>> (but see below).
>
>> Not sure do we want to add the few missing characters
>> mentioned at https://en.wikipedia.org/wiki/Cyrillic_script_in_Unicode,
>> e.g., one instantly notices that U+0400 is missing. (I wouldn't add at
>> least initially the more exotic characters, like the historic ones,
>> though.) Perhaps filing a bug or two for these cases for separate
>> consideration would be ok.
>
> The question here is what should serve as their transliteration and
> transcription?
Not sure, so filing a separate bug about this once your patch is merged
might be the most suitable action for now, I don't think we want to
postpone merging your work further due to these non-ISO 9 cases.
>> I'm not sure this will work, no existing rule in translit_* files
>> contain two characters, I'd assume that the rule for U+0423 is applied
>> first and then the below rule is never used.
>>
>> % CYRILLIC UNDEFINED
>> <U0423><U0301> <U00DA>;"<U0055><U0060>"
>>
>> Perhaps this should be commented out or removed altogether if it's not
>> working as intended.
>
> So yes, they are not processed. I would drop them to not to have special
> cases. But I am also fine with keeping them because all work is done
> already.
I'd probably drop them but I don't feel strongly about this either way.
Thanks for your efforts, I don't have any further comments, I'll leave
this now for Rafal and Mike to provide additional feedback and hopefully
merge soon.
Thanks,
On 11.10.2018 13:04, Rafal Luzynski wrote:
> Thank you, Egor. I am looking at your patch and although I have
> not yet finished, here are some remarks:
...
>> [...]
>> [BZ #2872]
>> * localedata/locales/translit_cyrillic: add ISO 9.1995, GOST 7.79
>
> Please start "Add" with an uppercase. BTW, shouldn't it be "New file"
> instead?
"New file or Add" - I don't know. You tell me.
>
>> System A transliteration System B transcription table from Cyrillic to
>> Latin/ASCII.
>> * localedata/locales/C: add include "translit_cyrillic";"" to LC_CTYPE
>> translit section.
>
> Same, "Add" here.
>
Same, please advise.
Bests,
Egor
On 11.10.2018 16:59, Egor Kobylkin wrote:
> On 11.10.2018 13:04, Rafal Luzynski wrote:
>>> COMMIT MESSAGE:
>>> [...]
>>> I am excluding these locales from this proposed patch. I have written
>>> directly to locale maintainer emails listed in the files. Volodymyr
>>> Lisivka <vlisivka@gmail.com>, Max Kutny <mkutny@gmail.com> (uk_UA),
>>> Данило Шеган <danilo@gnome.org> (sr_YU, sr_CS) have confirmed the
>>
>> I am not sure if we want Cyrillic text in the commit message. Shouldn't
>> it be, uhm, tranlisterated? :-)
>
> I do not see any Cyrillic text in the commit message.
> the ?????? you see are the actual "?" symbols coming out of iconv now.
>
>>
>> "sr_CS" - I guess you meant "sr_RS".
>>
>> "sr_YU" has been dropped, do we want to mention it?
>
> The list of locales and the patch itself is generated from the actual
> locales - I do not hand pick them, only exclude the ones in the
> exclusion list above.
Ah, yes, that message above should read sr_RS. Will fix.
There is no sr_YU anymore indeed, so I will drop it. No changes to the
patch, just the commit message.
Bests,
Egor
@@ -2293,6 +2293,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -68,6 +68,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -70,6 +70,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -54,6 +54,7 @@
copy "i18n"
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -1394,6 +1394,7 @@
<U137A> <U0060><U0039><U0030>
<U137B> <U0060><U0031><U0030><U0030>
<U137C> <U0060><U0031><U0030><U0030><U0030><U0030>
+include "translit_cyrillic";""
translit_end
%
END LC_CTYPE
@@ -44,6 +44,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -68,6 +68,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -41,6 +41,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -165,6 +165,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -85,6 +85,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -49,6 +49,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -39,6 +39,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -61,6 +61,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -43,6 +43,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -71,6 +71,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -38,6 +38,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -49,6 +49,7 @@
copy "i18n"
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
class "hanzi"; /
@@ -204,6 +204,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -108,6 +108,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -65,6 +65,7 @@
copy "i18n"
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -166,6 +166,7 @@
% LATIN SMALL LETTER O WITH STROKE -> "oe"
<U00F8> "<U006F><U0338>";"<U006F><U0065>"
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -78,6 +78,7 @@
% DOUBLE HIGH-REVERSED-9 QUOTATION MARK
<U201F> <U00AB>;<U0022>
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -51,6 +51,7 @@
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -59,6 +59,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -58,6 +58,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -54,6 +54,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -49,6 +49,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -41,6 +41,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -59,6 +59,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -72,6 +72,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -112,6 +112,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -78,6 +78,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -41,6 +41,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -136,6 +136,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -58,6 +58,7 @@
% In France, accents are simply omitted if they cannot be represented.
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -53,6 +53,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -45,6 +45,7 @@
copy "i18n"
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -62,6 +62,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -56,6 +56,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -58,6 +58,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -61,6 +61,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -37,6 +37,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -61,6 +61,7 @@
% transliterate <U0111> {Ä‘} into d + j
<U0111> "<U0064><U006A>"
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -57,6 +57,7 @@
copy "i18n"
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -476,6 +476,7 @@
<U00FC> "<U0075><U0308>";"<U0075><U00A8>";"<U0075><U003A>"
<U0171> "<U0075><U030B>";"<U0075><U02DD>";"<U0075><U0022>"
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -75,6 +75,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -54,6 +54,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -149,6 +149,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -58,6 +58,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -1681,6 +1681,7 @@
include "translit_combining";""
include "translit_cjk_variants";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -41,6 +41,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -157,6 +157,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -42,6 +42,7 @@
copy "i18n"
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -63,6 +63,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -6099,6 +6099,7 @@
include "translit_combining";""
include "translit_hangul";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -46,6 +46,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -57,6 +57,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -77,6 +77,7 @@
% LATIN SMALL LETTER E WITH CIRCUMFLEX
<U00EA> "e^"
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -56,6 +56,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -47,6 +47,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -39,6 +39,7 @@
copy "i18n"
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -50,6 +50,7 @@
copy "i18n"
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -163,6 +163,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -110,6 +110,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -54,6 +54,7 @@
% Accents are simply omitted if they cannot be represented.
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -58,6 +58,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -48,6 +48,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -60,6 +60,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
%
@@ -45,6 +45,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -47,6 +47,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -52,6 +52,7 @@
% accents are simply omitted if they cannot be represented.
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -154,6 +154,7 @@
% LATIN SMALL LETTER O WITH STROKE -> "oe"
<U00F8> "<U006F><U0338>";"<U006F><U0065>"
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -43,6 +43,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -59,6 +59,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -58,6 +58,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -58,6 +58,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -56,6 +56,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -64,6 +64,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -54,6 +54,7 @@
copy "i18n"
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -138,6 +138,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -62,6 +62,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -69,6 +69,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -60,6 +60,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -57,6 +57,7 @@
% Farsi yeh -> yeh
<U06CC> "<U064A>"
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -116,6 +116,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -58,6 +58,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -55,6 +55,7 @@
copy "i18n"
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -143,6 +143,7 @@
<U0162> "<U021A>";"<U0054>"
<U0163> "<U021B>";"<U0074>"
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -73,6 +73,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -45,6 +45,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -44,6 +44,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -46,6 +46,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -44,6 +44,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -39,6 +39,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -204,6 +204,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -58,6 +58,7 @@
copy "i18n"
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -58,6 +58,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -44,6 +44,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -67,6 +67,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -90,6 +90,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -37,6 +37,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -68,6 +68,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -45,6 +45,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -66,6 +66,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -62,6 +62,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -138,6 +138,7 @@
% LATIN SMALL LETTER O WITH STROKE -> "oe"
<U00F8> "<U006F><U0338>";"<U006F><U0065>"
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -43,6 +43,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -63,6 +63,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -63,6 +63,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -57,6 +57,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -864,6 +864,7 @@
<U137C> <U0060><U0031><U0030><U0030><U0030><U0030>
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
%
END LC_CTYPE
@@ -67,6 +67,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -36,6 +36,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -44,6 +44,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -2423,6 +2423,7 @@
% TURKISH LIRA SIGN
<U20BA> "<U0054><U004C>"
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -0,0 +1,383 @@
+escape_char /
+comment_char %
+
+% This file is part of the GNU C Library and contains locale data.
+% The Free Software Foundation does not claim any copyright interest
+% in the locale data contained in this file. The foregoing does not
+% affect the license of the GNU C Library as a whole. It does not
+% exempt you from the conditions of the license if your use would
+% otherwise be governed by that license.
+
+% Transliterations of cyrillic letters to latin and/or ascii symbols.
+% Inspired by ISO 9.1995 / GOST 7.79-2000.
+% Covers Unicode Range https://www.unicode.org/charts/PDF/U0400.pdf
+% i.e [U4001-U4F9, U2019] but only the letters covered by ISO 9.1995
+% It implements the GOST_7.79 System A (Latin Script) as a first
+% option and System B Cyrillic (ASCII) as a second option. Check
+% https://en.wikipedia.org/wiki/ISO_9 for reference.
+% The System B is extended from GOST_7.79-Russian using open sources
+% of the transliteration mappings and the "h/`" diacritics logic.
+
+% Usage examples:
+% iconv -f UTF-8 -t ISO-8859-15//TRANSLIT \
+% | iconv -f ISO-8859-15 -t UTF-8 # System A
+% iconv -f UTF-8 -t ASCII//TRANSLIT # System B.
+
+% Contributions welcome for the rest of Cyrillic script in Unicode
+% https://en.wikipedia.org/wiki/Cyrillic_script_in_Unicode.
+% Bugfix for https://sourceware.org/bugzilla/show_bug.cgi?id=2872.
+% Generated from UnicodeData.txt with
+% https://sourceware.org/bugzilla/attachment.cgi?id=11301.
+
+LC_CTYPE
+
+translit_start
+
+% CYRILLIC CAPITAL LETTER IO
+<U0401> <U00CB>;"<U0059><U004F>"
+% CYRILLIC CAPITAL LETTER DJE
+<U0402> <U0110>;"<U0044><U004A>"
+% CYRILLIC CAPITAL LETTER GJE
+<U0403> <U01F4>;"<U0047><U0060>"
+% CYRILLIC CAPITAL LETTER UKRAINIAN IE
+<U0404> <U00CA>;"<U0059><U0065>"
+% CYRILLIC CAPITAL LETTER DZE
+<U0405> <U1E90>;"<U005A><U0060>"
+% CYRILLIC CAPITAL LETTER BYELORUSSIAN-UKRAINIAN I
+<U0406> <U00CC>;<U0049>
+% CYRILLIC CAPITAL LETTER YI
+<U0407> <U00CF>;"<U0059><U0069>"
+% CYRILLIC CAPITAL LETTER JE
+<U0408> "<U004A><U030C>";<U004A>
+% CYRILLIC CAPITAL LETTER LJE
+<U0409> "<U004C><U0302>";"<U004C><U0060>"
+% CYRILLIC CAPITAL LETTER NJE
+<U040A> "<U004E><U0302>";"<U004E><U0060>"
+% CYRILLIC CAPITAL LETTER TSHE
+<U040B> <U0106>;"<U0054><U0053><U0048>"
+% CYRILLIC CAPITAL LETTER KJE
+<U040C> <U1E30>;"<U004B><U0060>"
+% CYRILLIC CAPITAL LETTER SHORT U
+<U040E> <U016C>;"<U0055><U0060>"
+% CYRILLIC CAPITAL LETTER DZHE
+<U040F> "<U0044><U0302>";"<U0044><U0068>"
+% CYRILLIC CAPITAL LETTER A
+<U0410> <U0041>
+% CYRILLIC CAPITAL LETTER BE
+<U0411> <U0042>
+% CYRILLIC CAPITAL LETTER VE
+<U0412> <U0056>
+% CYRILLIC CAPITAL LETTER GHE
+<U0413> <U0047>
+% CYRILLIC CAPITAL LETTER DE
+<U0414> <U0044>
+% CYRILLIC CAPITAL LETTER IE
+<U0415> <U0045>
+% CYRILLIC CAPITAL LETTER ZHE
+<U0416> <U017D>;"<U005A><U0048>"
+% CYRILLIC CAPITAL LETTER ZE
+<U0417> <U005A>
+% CYRILLIC CAPITAL LETTER I
+<U0418> <U0049>
+% CYRILLIC CAPITAL LETTER SHORT I
+<U0419> <U004A>
+% CYRILLIC CAPITAL LETTER KA
+<U041A> <U004B>
+% CYRILLIC CAPITAL LETTER EL
+<U041B> <U004C>
+% CYRILLIC CAPITAL LETTER EM
+<U041C> <U004D>
+% CYRILLIC CAPITAL LETTER EN
+<U041D> <U004E>
+% CYRILLIC CAPITAL LETTER O
+<U041E> <U004F>
+% CYRILLIC CAPITAL LETTER PE
+<U041F> <U0050>
+% CYRILLIC CAPITAL LETTER ER
+<U0420> <U0052>
+% CYRILLIC CAPITAL LETTER ES
+<U0421> <U0053>
+% CYRILLIC CAPITAL LETTER TE
+<U0422> <U0054>
+% CYRILLIC CAPITAL LETTER U
+<U0423> <U0055>
+% CYRILLIC UNDEFINED
+<U0423><U0301> <U00DA>;"<U0055><U0060>"
+% CYRILLIC CAPITAL LETTER EF
+<U0424> <U0046>
+% CYRILLIC CAPITAL LETTER HA
+<U0425> <U0048>;<U0058>
+% CYRILLIC CAPITAL LETTER TSE
+<U0426> <U0043>;"<U0043><U005A>"
+% CYRILLIC CAPITAL LETTER CHE
+<U0427> <U010C>;"<U0043><U0048>"
+% CYRILLIC CAPITAL LETTER SHA
+<U0428> <U0160>;"<U0053><U0048>"
+% CYRILLIC CAPITAL LETTER SHCHA
+<U0429> <U015C>;"<U0053><U0048><U0048>"
+% CYRILLIC CAPITAL LETTER HARD SIGN
+<U042A> <U02BA>;"<U0041><U0060>"
+% CYRILLIC CAPITAL LETTER YERU
+<U042B> <U0059>;"<U0059><U0060>"
+% CYRILLIC CAPITAL LETTER SOFT SIGN
+<U042C> <U02B9>;<U0060>
+% CYRILLIC CAPITAL LETTER E
+<U042D> <U00C8>;"<U0045><U0060>"
+% CYRILLIC CAPITAL LETTER YU
+<U042E> <U00DB>;"<U0059><U0055>"
+% CYRILLIC CAPITAL LETTER YA
+<U042F> <U00C2>;"<U0059><U0041>"
+% CYRILLIC SMALL LETTER A
+<U0430> <U0061>
+% CYRILLIC SMALL LETTER BE
+<U0431> <U0062>
+% CYRILLIC SMALL LETTER VE
+<U0432> <U0076>
+% CYRILLIC SMALL LETTER GHE
+<U0433> <U0067>
+% CYRILLIC SMALL LETTER DE
+<U0434> <U0064>
+% CYRILLIC SMALL LETTER IE
+<U0435> <U0065>
+% CYRILLIC SMALL LETTER ZHE
+<U0436> <U017E>;"<U007A><U0068>"
+% CYRILLIC SMALL LETTER ZE
+<U0437> <U007A>
+% CYRILLIC SMALL LETTER I
+<U0438> <U0069>
+% CYRILLIC SMALL LETTER SHORT I
+<U0439> <U006A>
+% CYRILLIC SMALL LETTER KA
+<U043A> <U006B>
+% CYRILLIC SMALL LETTER EL
+<U043B> <U006C>
+% CYRILLIC SMALL LETTER EM
+<U043C> <U006D>
+% CYRILLIC SMALL LETTER EN
+<U043D> <U006E>
+% CYRILLIC SMALL LETTER O
+<U043E> <U006F>
+% CYRILLIC SMALL LETTER PE
+<U043F> <U0070>
+% CYRILLIC SMALL LETTER ER
+<U0440> <U0072>
+% CYRILLIC SMALL LETTER ES
+<U0441> <U0073>
+% CYRILLIC SMALL LETTER TE
+<U0442> <U0074>
+% CYRILLIC SMALL LETTER U
+<U0443> <U0075>
+% CYRILLIC UNDEFINED
+<U0443><U0301> <U00FA>;"<U0075><U0060>"
+% CYRILLIC SMALL LETTER EF
+<U0444> <U0066>
+% CYRILLIC SMALL LETTER HA
+<U0445> <U0068>;<U0078>
+% CYRILLIC SMALL LETTER TSE
+<U0446> <U0063>;"<U0063><U007A>"
+% CYRILLIC SMALL LETTER CHE
+<U0447> <U010D>;"<U0063><U0068>"
+% CYRILLIC SMALL LETTER SHA
+<U0448> <U0161>;"<U0073><U0068>"
+% CYRILLIC SMALL LETTER SHCHA
+<U0449> <U015D>;"<U0073><U0068><U0068>"
+% CYRILLIC SMALL LETTER HARD SIGN
+<U044A> <U02BA>;"<U0060><U0060>"
+% CYRILLIC SMALL LETTER YERU
+<U044B> <U0079>;"<U0079><U0060>"
+% CYRILLIC SMALL LETTER SOFT SIGN
+<U044C> <U02B9>;<U0060>
+% CYRILLIC SMALL LETTER E
+<U044D> <U00E8>;"<U0065><U0060>"
+% CYRILLIC SMALL LETTER YU
+<U044E> <U00FB>;"<U0079><U0075>"
+% CYRILLIC SMALL LETTER YA
+<U044F> <U00E2>;"<U0079><U0061>"
+% CYRILLIC SMALL LETTER IO
+<U0451> <U00EB>;"<U0079><U006F>"
+% CYRILLIC SMALL LETTER DJE
+<U0452> <U0111>;"<U0064><U006A>"
+% CYRILLIC SMALL LETTER GJE
+<U0453> <U01F5>;"<U0067><U0060>"
+% CYRILLIC SMALL LETTER UKRAINIAN IE
+<U0454> <U00EA>;"<U0079><U0065>"
+% CYRILLIC SMALL LETTER DZE
+<U0455> <U1E91>;"<U007A><U0060>"
+% CYRILLIC SMALL LETTER BYELORUSSIAN-UKRAINIAN I
+<U0456> <U00EC>;<U0069>
+% CYRILLIC SMALL LETTER YI
+<U0457> <U00EF>;"<U0079><U0069>"
+% CYRILLIC SMALL LETTER JE
+<U0458> <U01F0>;<U006A>
+% CYRILLIC SMALL LETTER LJE
+<U0459> "<U006C><U0302>";"<U006C><U0060>"
+% CYRILLIC SMALL LETTER NJE
+<U045A> "<U006E><U0302>";"<U006E><U0060>"
+% CYRILLIC SMALL LETTER TSHE
+<U045B> <U0107>;"<U0074><U0073><U0068>"
+% CYRILLIC SMALL LETTER KJE
+<U045C> <U1E31>;"<U006B><U0060>"
+% CYRILLIC SMALL LETTER SHORT U
+<U045E> <U016D>;"<U0075><U0060>"
+% CYRILLIC SMALL LETTER DZHE
+<U045F> "<U0064><U0302>";"<U0064><U0068>"
+% CYRILLIC CAPITAL LETTER BIG YUS
+<U046A> <U01CD>;"<U004F><U0060>"
+% CYRILLIC SMALL LETTER BIG YUS
+<U046B> <U01CE>;"<U006F><U0060>"
+% CYRILLIC CAPITAL LETTER FITA
+<U0472> "<U0046><U0300>";"<U0046><U0068>"
+% CYRILLIC SMALL LETTER FITA
+<U0473> "<U0066><U0300>";"<U0066><U0068>"
+% CYRILLIC CAPITAL LETTER IZHITSA
+<U0474> <U1EF2>;"<U0059><U0068>"
+% CYRILLIC SMALL LETTER IZHITSA
+<U0475> <U1EF3>;"<U0079><U0068>"
+% CYRILLIC CAPITAL LETTER SEMISOFT SIGN
+<U048C> <U011A>;"<U0045><U0060>"
+% CYRILLIC SMALL LETTER SEMISOFT SIGN
+<U048D> <U011B>;"<U0065><U0060>"
+% CYRILLIC CAPITAL LETTER GHE WITH UPTURN
+<U0490> "<U0047><U0300>";"<U0047><U0060>"
+% CYRILLIC SMALL LETTER GHE WITH UPTURN
+<U0491> "<U0067><U0300>";"<U0067><U0060>"
+% CYRILLIC CAPITAL LETTER GHE WITH STROKE
+<U0492> <U0120>;"<U0047><U0048>"
+% CYRILLIC SMALL LETTER GHE WITH STROKE
+<U0493> <U0121>;"<U0067><U0068>"
+% CYRILLIC CAPITAL LETTER GHE WITH MIDDLE HOOK
+<U0494> <U011E>;"<U0047><U0048>"
+% CYRILLIC SMALL LETTER GHE WITH MIDDLE HOOK
+<U0495> <U011F>;"<U0067><U0068>"
+% CYRILLIC CAPITAL LETTER ZHE WITH DESCENDER
+<U0496> "<U017D><U0327>";"<U005A><U0048><U0060>"
+% CYRILLIC SMALL LETTER ZHE WITH DESCENDER
+<U0497> "<U017E><U0327>";"<U007A><U0068><U0060>"
+% CYRILLIC CAPITAL LETTER KA WITH DESCENDER
+<U049A> <U0136>;"<U004B><U0060>"
+% CYRILLIC SMALL LETTER KA WITH DESCENDER
+<U049B> <U0137>;"<U006B><U0060>"
+% CYRILLIC CAPITAL LETTER KA WITH STROKE
+<U049E> "<U004B><U0304>";"<U004B><U0060>"
+% CYRILLIC SMALL LETTER KA WITH STROKE
+<U049F> "<U006B><U0304>";"<U006B><U0060>"
+% CYRILLIC CAPITAL LETTER EN WITH DESCENDER
+<U04A2> <U1E46>;"<U004E><U0060>"
+% CYRILLIC SMALL LETTER EN WITH DESCENDER
+<U04A3> <U1E47>;"<U006E><U0060>"
+% CYRILLIC CAPITAL LIGATURE EN GHE
+<U04A4> <U1E44>;"<U004E><U0047>"
+% CYRILLIC SMALL LIGATURE EN GHE
+<U04A5> <U1E45>;"<U006E><U0067>"
+% CYRILLIC CAPITAL LETTER PE WITH MIDDLE HOOK
+<U04A6> <U1E54>;"<U0050><U0060>"
+% CYRILLIC SMALL LETTER PE WITH MIDDLE HOOK
+<U04A7> <U1E55>;"<U0070><U0060>"
+% CYRILLIC CAPITAL LETTER ABKHASIAN HA
+<U04A8> <U00D2>;"<U004F><U0060>"
+% CYRILLIC SMALL LETTER ABKHASIAN HA
+<U04A9> <U00F2>;"<U006F><U0060>"
+% CYRILLIC CAPITAL LETTER ES WITH DESCENDER
+<U04AA> <U00C7>;"<U0043><U0060>"
+% CYRILLIC SMALL LETTER ES WITH DESCENDER
+<U04AB> <U00E7>;"<U0043><U0060>"
+% CYRILLIC CAPITAL LETTER TE WITH DESCENDER
+<U04AC> <U0162>;"<U0054><U0060>"
+% CYRILLIC SMALL LETTER TE WITH DESCENDER
+<U04AD> <U0163>;"<U0074><U0060>"
+% CYRILLIC CAPITAL LETTER STRAIGHT U
+<U04AE> <U00D9>;<U0055>
+% CYRILLIC SMALL LETTER STRAIGHT U
+<U04AF> <U00F9>;<U0075>
+% CYRILLIC CAPITAL LETTER HA WITH DESCENDER
+<U04B2> <U1E28>;"<U0048><U0060>"
+% CYRILLIC SMALL LETTER HA WITH DESCENDER
+<U04B3> <U1E29>;"<U0068><U0060>"
+% CYRILLIC CAPITAL LIGATURE TE TSE
+<U04B4> "<U0043><U0304>";"<U0054><U0043><U005A>"
+% CYRILLIC SMALL LIGATURE TE TSE
+<U04B5> "<U0063><U0304>";"<U0074><U0063><U007A>"
+% CYRILLIC CAPITAL LETTER SHHA
+<U04BA> <U1E24>;"<U0053><U0048><U0060>"
+% CYRILLIC SMALL LETTER SHHA
+<U04BB> <U1E25>;"<U0053><U0048><U0060>"
+% CYRILLIC CAPITAL LETTER ABKHASIAN CHE
+<U04BC> "<U0043><U0306>";"<U0043><U0048><U0060>"
+% CYRILLIC SMALL LETTER ABKHASIAN CHE
+<U04BD> "<U0063><U0306>";"<U0063><U0068><U0060>"
+% CYRILLIC CAPITAL LETTER ABKHASIAN CHE WITH DESCENDER
+<U04BE> "<U00C7><U0306>";"<U0043><U0048><U0060>"
+% CYRILLIC SMALL LETTER ABKHASIAN CHE WITH DESCENDER
+<U04BF> "<U00E7><U0306>";"<U0063><U0068><U0060>"
+% CYRILLIC LETTER PALOCHKA
+<U04C0> <U2021>;<U0069>
+% CYRILLIC CAPITAL LETTER ZHE WITH BREVE
+<U04C1> "<U005A><U0306>";"<U005A><U0048><U0060>"
+% CYRILLIC SMALL LETTER ZHE WITH BREVE
+<U04C2> "<U007A><U0306>";"<U007A><U0068><U0060>"
+% CYRILLIC CAPITAL LETTER KHAKASSIAN CHE
+<U04CB> <U00C7>;"<U0043><U0048><U0060>"
+% CYRILLIC SMALL LETTER KHAKASSIAN CHE
+<U04CC> <U00E7>;"<U0063><U0068><U0060>"
+% CYRILLIC CAPITAL LETTER A WITH BREVE
+<U04D0> <U0102>;"<U0041><U0060>"
+% CYRILLIC SMALL LETTER A WITH BREVE
+<U04D1> <U0103>;"<U0061><U0060>"
+% CYRILLIC CAPITAL LETTER A WITH DIAERESIS
+<U04D2> <U00C4>;"<U0041><U0060>"
+% CYRILLIC SMALL LETTER A WITH DIAERESIS
+<U04D3> <U00E4>;"<U0061><U0060>"
+% CYRILLIC CAPITAL LETTER IE WITH BREVE
+<U04D6> <U0114>;"<U0045><U0060>"
+% CYRILLIC SMALL LETTER IE WITH BREVE
+<U04D7> <U0115>;"<U0065><U0060>"
+% CYRILLIC CAPITAL LETTER SCHWA
+<U04D8> "<U0041><U030B>";"<U0041><U0060>"
+% CYRILLIC SMALL LETTER SCHWA
+<U04D9> "<U0061><U030B>";"<U0061><U0060>"
+% CYRILLIC CAPITAL LETTER ZHE WITH DIAERESIS
+<U04DC> "<U005A><U0304>";"<U005A><U0048><U0060>"
+% CYRILLIC SMALL LETTER ZHE WITH DIAERESIS
+<U04DD> "<U007A><U0304>";"<U007A><U0068><U0060>"
+% CYRILLIC CAPITAL LETTER ZE WITH DIAERESIS
+<U04DE> "<U005A><U0308>";"<U005A><U0060>"
+% CYRILLIC SMALL LETTER ZE WITH DIAERESIS
+<U04DF> "<U007A><U0308>";"<U007A><U0060>"
+% CYRILLIC CAPITAL LETTER ABKHASIAN DZE
+<U04E0> <U0179>;"<U005A><U0060>"
+% CYRILLIC SMALL LETTER ABKHASIAN DZE
+<U04E1> <U017A>;"<U007A><U0060>"
+% CYRILLIC CAPITAL LETTER I WITH DIAERESIS
+<U04E4> <U00CE>;"<U0049><U0060>"
+% CYRILLIC SMALL LETTER I WITH DIAERESIS
+<U04E5> <U00EE>;"<U0069><U0060>"
+% CYRILLIC CAPITAL LETTER O WITH DIAERESIS
+<U04E6> <U00D6>;"<U004F><U0060>"
+% CYRILLIC SMALL LETTER O WITH DIAERESIS
+<U04E7> <U00F6>;"<U006F><U0060>"
+% CYRILLIC CAPITAL LETTER BARRED O
+<U04E8> <U00D4>;"<U004F><U0060>"
+% CYRILLIC SMALL LETTER BARRED O
+<U04E9> <U00F4>;"<U006F><U0060>"
+% CYRILLIC CAPITAL LETTER U WITH DIAERESIS
+<U04F0> <U00DC>;"<U0055><U0060>"
+% CYRILLIC SMALL LETTER U WITH DIAERESIS
+<U04F1> <U00FC>;"<U0075><U0060>"
+% CYRILLIC CAPITAL LETTER U WITH DOUBLE ACUTE
+<U04F2> <U0170>;"<U0055><U0060>"
+% CYRILLIC SMALL LETTER U WITH DOUBLE ACUTE
+<U04F3> <U0171>;"<U0075><U0060>"
+% CYRILLIC CAPITAL LETTER CHE WITH DIAERESIS
+<U04F4> "<U0043><U0308>";"<U0043><U0048><U0060>"
+% CYRILLIC SMALL LETTER CHE WITH DIAERESIS
+<U04F5> "<U0063><U0308>";"<U0063><U0068><U0060>"
+% CYRILLIC CAPITAL LETTER YERU WITH DIAERESIS
+<U04F8> <U0178>;"<U0059><U0060>"
+% CYRILLIC SMALL LETTER YERU WITH DIAERESIS
+<U04F9> <U00FF>;"<U0079><U0060>"
+% RIGHT SINGLE QUOTATION MARK
+<U2019> <U2035>;<U0027>
+
+translit_end
+
+END LC_CTYPE
@@ -62,6 +62,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -48,6 +48,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -46,6 +46,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -57,6 +57,7 @@
% Farsi yeh -> yeh
<U06CC> "<U064A>"
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -65,6 +65,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -57,6 +57,7 @@
% dong sign -> d// -> dd
<U20AB> "<U0111>";"<U0064><U0064>"
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -59,6 +59,7 @@
<U00C5> "A<U030A>";"A";"AU"
<U00E5> "a<U030A>";"a";"au"
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -54,6 +54,7 @@
% Accents are simply omitted if they cannot be represented.
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -64,6 +64,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -66,6 +66,7 @@
<U05F0> "<U05D5><U05D5>";"ww"
<U05F1> "<U05D5><U05D9>";"wj"
<U05F2> "<U05D9><U05D9>";"jj"
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -40,6 +40,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE
@@ -58,6 +58,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
class "hanzi"; /
@@ -68,6 +68,7 @@
translit_start
include "translit_combining";""
+include "translit_cyrillic";""
translit_end
END LC_CTYPE