- From: Martin J. Dürst <duerst@it.aoyama.ac.jp>
- Date: Mon, 18 Nov 2013 20:26:44 +0900
- To: "Henry S. Thompson" <ht@inf.ed.ac.uk>
- CC: Pete Cordell <petejson@codalogic.com>, John Cowan <cowan@mercury.ccil.org>, IETF Discussion <ietf@ietf.org>, JSON WG <json@ietf.org>, Anne van Kesteren <annevk@annevk.nl>, www-tag@w3.org, es-discuss <es-discuss@mozilla.org>
On 2013/11/18 20:11, Henry S. Thompson wrote: > Pete Cordell writes: > >> Given the history below, would it be sensible to accept BOMs for UTF-8 >> encoding, but not for UTF-16 and UTF-32? In other words, are BOMs needed >> and/or used in the wild for UTF-16 and UTF-32? >> >> Maybe the text can say something like "SHOULD accept BOMs for UTF-8, >> and MAY accept BOMs for UTF-16 and / or UTF-32"? > > My sense is that you'll see more UTF-16 BOMs than anything else. Yes indeed. BOM means Byte Order Mark. It's crucial for over-the-wire UTF-16. (It's irrelevant for in-memory UTF-16, but that's not what we are discussing.) To bring up the XML example again, XML actually strictly requires a BOM for UTF-16. The IETF definition of UTF-16 does not require a BOM for UTF-16. See http://tools.ietf.org/html/rfc2781, in particular http://tools.ietf.org/html/rfc2781#section-3.2, http://tools.ietf.org/html/rfc2781#section-3.3, and http://tools.ietf.org/html/rfc2781#section-4. For UTF-8, the BOM is not a Byte Order Mark, because such a mark isn't necessary at all. It may serve as a signature, but is not necessary, and in some circumstances counterproductive. As for what to say about whether to accept BOMs or not, I'd really want to know what the various existing parsers do. If they accept BOMs, then we can say they should accept BOMs. If they don't accept BOMs, then we should say that they don't. Regards, Martin. > UTF-32 support seems to be waning (at least in the browsers), but > UTF-16 is in pretty widespread use. John, do you think you can fool > google into counting BOMs for us?
Received on Monday, 18 November 2013 11:28:28 UTC