[whatwg/streams] Utility buffer class for stream transformers (#1097)

I was hoping to start a discussion about the class `BufferTransformer` that I've just mocked up. Is this an antipattern? Could this be a useful helper class in the spec?

### BufferTransformer
```typescript
class Lock {
  promise: Promise<void>;
  open : () => void;
  close : () => void;
  constructor(){
    (this.close = () =>
      this.promise = new Promise(r =>
        this.open = r
      )
    )();
  }
}

class BufferTransformer<I, O> implements Transformer<I[], O[]> {
  private buffer: I[] = [];
  private lock: Lock = new Lock();
  private terminator: I;

  constructor(terminator: I){
    this.terminator = terminator;
  }

  transform(chunk: I[]){
    this.buffer = this.buffer.concat(chunk);
    this.lock.open();
  }

  flush(){
    this.lock.open();
  }

  private async has(i: number): Promise<boolean> {
    if(this.buffer.length > i) return true;
    this.lock.close();
    await this.lock.promise;
    return this.buffer.length > i;
  }

  async read(): Promise<I> {
    return await this.has(0) ? this.buffer.shift() : this.terminator;
  }

  async peek(i): Promise<I> {
    return await this.has(i) ? this.buffer[i] : this.terminator;
  }

  reconsume(i: I){
    this.buffer.unshift(i);
  }
}
```
An example of its usage is shown below, it takes a stream of characters, reads an unknown amount of them and enqueues only 1 Token. This seems to be the opposite of how I've seen a transformer used, where a fixed buffer of data is given to the transformer and an unknown amount of things are then queued.
 
### TokenizerTransformer
```typescript
class TokenizerTransformer extends BufferTransformer<string, Token> {
  async start(controller){
    let token: Token;
    while(token = await this.consumeToken()){
      controller.enqueue(token);
    }
  }
  
  async consumeToken(): Promise<Token>{
    await this.consumeComments();

    let c = await this.read();
    if(c === "") return;

    if(isWhitespace(c)){
      while(isWhitespace(await this.peek(0))){
        await this.read();
      }
      return {type: TokenType.whitespace};
    }
    
    if(isDigit(c)){
      this.reconsume(c);
      return this.consumeNumeric();
    }

    ...
  }

  ...
}
```


And another thing I was curious about is if I wanted to run this process on a string, is there no utility function or easier way than below to start the process?

```typescript
const stringToStream = str => new ReadableStream<string>({
  start(controller){
    str.split("").forEach(c => 
      controller.enqueue(c)
    );
    controller.close();
  }
});

stringToStream("foo").pipeThrough(new TokenizerTransformer)
````

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/streams/issues/1097