Re: [whatwg/encoding] Make TextEncoder and TextDecoder be transform streams (#127)

> Since it's a common enough request, would it be possible to include an example in the spec itself showing how to use streams to encode into an existing ArrayBuffer?

I'd like to reiterate the concerns about interoperability hazards associated with existing ArrayBuffer that I mentioned in [the issue dedicated to that request](https://github.com/whatwg/encoding/issues/69).

> I think we should indeed make split surrogates work.

While JS strings need to be able to exist in invalid-UTF-16 states when a JS program constructs a valid UTF-16 string, making unpaired surrogate preservation in crossing the WebIDL boundary into an explicit feature is a novel thing for the Web Platform and one that makes me uneasy.

So far, we have USVString, which explicitly replaces unpaired surrogates and various DOMString APIs that in UTF-16-based Web engines pass unpaired surrogates through, but that's mostly incidental and Web developers don't have good reasons to pass unpaired surrogates through, so not preserving unpaired surrogates appears to be feasible in terms of Web compat.

In Gecko, we were able to make CSSOM effectively change from DOMString to USVString without breaking the Web. If I've read the Servo source comments correctly, Servo panics if a DOMString contains unpaired surrogates and the absence of such crash reports suggests that the Web doesn't actually depend on unpaired surrogates crossing the WebIDL boundary (to the extent Servo has enough DOM features to be exercised with real Web content).

While of all the Web Platform APIs this might be the one where letting unpaired surrogates cross the WebIDL boundary might make sense as an explicit feature, I think we should consider a bit more carefully if that's necessary and if it's feasible to require textual streams not to split surrogate pairs across chunk boundaries.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/encoding/pull/127#issuecomment-359555507