[whatwg/streams] Switching to Web IDL (#963)

### Background

Streams as originally envisioned was meant as a ecosystem-agnostic specification at the level of ECMAScript. As such I wrote it with that specification style ([mostly](https://streams.spec.whatwg.org/#conventions)), instead of using Web IDL. See previous discussions in #45 and #732.

Since that time we've seen a shift in the ecosystem, toward using Web IDL more broadly:

- The URL and Encoding specifications (which use Web IDL), among others, are implemented in Node.js. That is, we have significant examples that Web IDL-based specifications can be implemented even in non-web environments.
- The Web Assembly specification settled on using Web IDL, instead of the ECMAScript style, despite being "engine level" and environment-agnostic. This is especially notable because most (all?) implementations of the Web Assembly API are done in the JS engine, instead of in the rendering engine with its IDL binding infrastructure. So this shows that IDL on the specification side is compatible with JS-engine-level implementations in a browser.
- The ECMAScript spec itself is [contemplating adopting an IDL](https://github.com/littledan/proposal-idl/blob/master/README.md), and at least one possible path there is to use an evolution of Web IDL.

Furthermore, not using Web IDL has caused us some pain. I think the pain breaks down in to three main areas:

1. Web platform implementers may be less familiar with this style of specification, which can be frustrating for them (e.g. #732). It can also be harder for them to implement in a way that matches the spec, using the tools available to them; they may need to use JavaScript engine APIs explicitly, or add lots of extended attributes to override Web IDL's default behavior if they use Web IDL anyway.
2. We lose some consistency with the rest of the platform. This category is about minor-ish technical things like over-generic methods (#961), non-enumerable methods, and especially argument coercion and options bag processing. We gain a little bit of consistency with ECMAScript, but ECMASCript isn't terribly internally consistent anyway, so on balance this is worse.
3. The ecosystem of tooling built up around Web IDL specs is not available to us. This involves things like automatic test coverage via idlharness.js (#962), generation of canonical TypeScript definitions (mentioned in #45), and integration with the Web Assembly host bindings proposal. It also causes us problems around how our spec integrates into the IDL ecosystem (https://github.com/whatwg/fetch/issues/780), and even our spec preprocessor (https://github.com/tabatkins/bikeshed/issues/1346). And proposals like [get-originals](https://github.com/domenic/get-originals) need to have a dedicated branch specifically for streams, instead of just treating these classes like other IDL interfaces.

As such, I think we should reconsider our avoidance of Web IDL. I'd like to use this issue to explore what a switch would look like.

### Plan

Concretely, I think there are a few things to consider:

1. Some patterns in the current spec are not expressible in Web IDL. I think this list is exhaustive:
   - Generic methods that don't check their `this` value: `pipeTo()` (#961) and the queuing strategy `size()` methods
   - Data properties instead of getters: `highWaterMark` on the queuing strategy classes
   - Non-enumerable methods and properties
   - Async iterators: landing in #954
   - Missing toStringTag: #952
2. A lot of argument processing in the current spec would change slightly if we switched to _idiomatic_ Web IDL (i.e., using types that weren't `any`). Especially options bag processing, when converted to dictionaries.
3. A lot of the normative spec text currently assumes it operates on ECMAScript objects, and would need to be carefully switched over. For example, [CreateReadableStream](https://streams.spec.whatwg.org/#create-readable-stream)'s object creation steps would need to move to a vaguer Web IDL style "create an X object". All the IsX() checks would become unnecessary. Etc.
4. Stylistically IDL specs use a different style, e.g. instead of internal slots they use "associated X", and abstract operations are written like "readable stream reader generic cancel steps" instead of "ReadableStreamReaderGenericCancel".

(1) and (2) are the toughest. I would be in favor of just switching wholesale to idiomatic Web IDL, and hoping that there are no web-compat issues. Specifically, moving away from current non-expressible patterns toward more standard ones, and changing argument processing to match how Web IDL dictionaries/optional/etc. work. I think it's likely web-compat will be fine, but who knows, enumerability of methods could throw someone off, or data properties vs. prototypes for the queuing strategies. We could go halfway, e.g. we could add a [NonEnumerable] attribute to Web IDL (it's already in most implementations), or continue defining data properties using ECMAspeak instead of moving `highWaterMark` to be a prototype getter. Thoughts welcome.

(3) is just a lot of work. It makes me think we may want to do this piecemeal, e.g. we could convert one class at a time.

(4) is silly and I think we should just keep the current style. Switching is not worth it.

### Thoughts?

I'd be interested to hear from various audiences. One thing I'm very concerned about is not making implementers feel like we are creating busywork for them. Or that for those implementers like @tschneidereit who implement streams in the JS engine, they now need to uproot all their work and move it to their rendering engine. I'm hopeful that the precedent of Web Assembly alleviates some of these concerns. So I'd appreciate hearing from folks like @youennf / @othermaciej, @tschneidereit / @jorendorff, @ricea / @yhirano, or @travisleithead about whether they'd feel positively or negatively about this sort of shift.

Thanks all, and my apologies for getting us into this situation so many years ago.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/streams/issues/963