W3C home > Mailing lists > Public > public-i18n-core@w3.org > January to March 2015

I18N-ISSUE-454 (BUG28156): Separate GBK and GB18030 even for decoding (toUnicode) [encoding]

From: Internationalization Working Group Issue Tracker <sysbot+tracker@w3.org>
Date: Mon, 30 Mar 2015 14:42:40 +0000
To: public-i18n-core@w3.org
Message-Id: <E1Ycatk-0009R4-VN@maia.w3.org>
I18N-ISSUE-454 (BUG28156): Separate GBK and GB18030 even for decoding  (toUnicode) [encoding]

http://www.w3.org/International/track/issues/454

Raised by: Richard Ishida
On product: encoding

https://www.w3.org/Bugs/Public/show_bug.cgi?id=28156

This issue tracks the bug listed above and was created as part of the WG CR process.

---

After bug 27235, GBK and GB18030 are distinct when encoding (fromUnicode). 

I guess the rationale for treating GBK and GB18030 identically when decodidng
(toUnicode) is that there are (significant) number of pages that are actually
in GB18030 but are mislabelled as GBK. 

I wonder if there's any statistics collected for that. I'm curious to know what
percentage of documents labelled as GBK are actually in GB18030. My suspicion
is that it's pretty low especially compared with 'ISO-8859-1 vs windows-1252',
'EUC-KR vs windows-949' (because it's so prevalent that the spec's EUC-KR is
actually windows-949, which I fully support), 'TIS 620 : ISO-8859-11 :
windows-864', and so forth. 

I'm raising this issue because 1) Blink, Webkit, Firefox (and I guess, IE, too)
have treated two encodings separately  2) Blink need to add extra code to treat
GBK/GB18030 as specified in the current spec. 

I believe that it's doable (I thought about how to do that yesterday), but I'm
not convinced that it's worth the effort / extra code.
Received on Monday, 30 March 2015 14:42:41 UTC

This archive was generated by hypermail 2.3.1 : Monday, 30 March 2015 14:42:42 UTC