Re: [whatwg/webidl] Add `async iterable<T>` type to WebIDL (PR #1397)

> I wonder how this ends up working in practice. Say you have Blob or async iterable<T> or DOMString. If you pass in a Blob with @@asyncIterable support, how is it handled? I guess @@asyncIterable comes later, probably before toString?

@annevk I have specified this behaviour in an update to the overload resolution algorithm in this PR already. The gist is: an `async iterable<T>` has the same behaviour for distinct types as `sequence<T>` (which one could think of as the eager synchronous version of `async iterator<T>`).

I have updated the graphic in the spec that explains this too: 
<img width="729" alt="image" src="https://github.com/whatwg/webidl/assets/7829205/faba6792-7c92-4022-b68d-1c68f698e93f">

In practice:

1. `async iterator<T>` can be distinguished from all types but `object` and `sequence<T>`
2. As with `sequence<T>`, a matching interface is considered more important than a platform object that implements the iterable, or async iterable protocol. So specifically for the `BodyInit` with `Blob` case: nothing changes.
3. As with `sequence<T>`, primitives like strings, even if they have `@@iterator` on their prototype, are not considered for `async iterator<T>` overloads, and are also not valid `async iterator<T>` values. `async iterator<T>` requires values are `Type(V) == Object`, in addition to having either `@@iterator` or `@@asyncIterator`.

---

> What is being proposed exactly, to what WebIDL type would it be converted? A reference to an [Iterator Record](https://tc39.es/ecma262/#sec-iterator-records)?

@petervanderbeken I am proposing the addition of an `async iterable<T>` type, which mirrors `sequence<T>` in many ways, but unlike it is:

- **lazy**: it can be possibly infinite length, with the JS -> IDL conversion not requiring synchronous transformation of all values of the underlying iterable. Instead values are lazily iterated off the underlying iterator, converted from JS to IDL, and yielded to IDL callers, as they request them.
- **asynchronous**: values can be produced asynchronously

If you imagine a matrix of JS stream primitives, with asynchronicity on one axis, and eagerness on the other, this fills out the square opposite of `sequence<T>`.

|    | **sync** | **async** |
| -- | -- | -- |
| eager | JS: `Array<T>`; IDL: `sequence<T>` | JS: `Promise<Array<T \| Promise<T>>>`; _`async sequence<T>`_ (doesn't exist) |
| lazy | JS: `{ @@iterator: Iterator<T> }`; IDL: _`iterable<T>`_ (doesn't exist) | JS: `{ @@asyncIterator: AsyncIterator<T> }`; IDL: `async iterable<T>` (this PR) | 

The use case for this type are Web APIs that take "streams" of values (of unknown length), that they then process in some way. A real Web API that does this right now is `ReadableStream.from()`. It takes objects that are iterables as a Web IDL `any` type, and [performs manual processing of this iterable](https://streams.spec.whatwg.org/#readable-stream-from-iterable). For one API, this is fine, but there are other proposed APIs that would also take async iterables as arguments:

- `new Response(asyncIterable)`: this overload of `Response`'s `BodyInit` aims to simplify interop of fetch bodies with Node's own stream APIs, which implement `@@asyncIterable` (see https://github.com/whatwg/fetch/issues/1291).
- `crypto.subtle.digest(algorithm, asyncIterable)`: this overload of subtle digest would allow digesting large sets of data without having to load them into memory entirely (see https://github.com/w3c/webcrypto/issues/73 and https://github.com/w3c/webcrypto/issues/73#issuecomment-1003741806).

Introduction of this `async iterable<T>` simplifies all of these, because the entirety of the stream iteration logic is now pulled into Web IDL, where all callers can share it.




-- 
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/webidl/pull/1397#issuecomment-2018649896
You are receiving this because you are subscribed to this thread.

Message ID: <whatwg/webidl/pull/1397/c2018649896@github.com>