Re: [whatwg/streams] Enable WebIDL harness tests for reference implementation (#1044)

The good news: I already managed to get the tests to run locally, and I already found some missing things and broken things in the tests:
* web-platform-tests/wpt#24266
* web-platform-tests/wpt#24267

Now, I'd like to clean up and upstream my local patches. But first, I'd like your input on my suggested approach before I start making more pull requests. 😁 

In essence, there are two things missing to get the tests to run:
1. `wpt-runner` [needs to serve](https://github.com/web-platform-tests/wpt/blob/40724da33edf9653287985ff4f7b226ce60a0097/streams/idlharness.any.js#L2-L3) `/resources/idlharness.js` and `/resources/WebIDLParser.js` (which is actually [rewritten](https://github.com/web-platform-tests/wpt/blob/40724da33edf9653287985ff4f7b226ce60a0097/tools/serve/serve.py#L320) to `/resources/webidl2/lib/webidl2.js`) from `web-platform-tests`.
2. `idlharness.js` [needs to `fetch`](https://github.com/web-platform-tests/wpt/blob/master/resources/idlharness.js#L3483-L3494) WebIDL files from `web-platform-tests/interfaces`.

For (1), wpt-runner would need to either include its own copy of these files, or would need to know where to find them. The first option seems like a bad idea, since we'd have to regular make new releases just to stay up-to-date with WPT. For the second option, we could add an option where the user can define a list of URLs and file paths that the runner needs to serve. Then, [the streams spec](https://github.com/whatwg/streams/blob/c20f174396617787e1176e11796fc5039088a5ab/reference-implementation/run-web-platform-tests.js#L37) would pass in the URLs and paths of the WebIDL harness files:
```javascript
const failures = await wptRunner(testsPath, {
  rootURL: 'streams/',
  server: {
    // map of URL to file path
    "/resources/idlharness.js": path.resolve(__dirname, "web-platform-tests/resources/idlharness.js"),
    "/resources/WebIDLParser.js": path.resolve(__dirname, "web-platform-tests/resources/webidl2/lib/webidl2.js")
  },
  // ...
});
```

For (2), unfortunately `jsdom` does not yet support `fetch` out-of-the-box (jsdom/jsdom#1724). However, we don't need a full-fledged (...full-*fetch*ed?) `fetch` implementation: we only need to support what [idlharness's `fetch_spec`](https://github.com/web-platform-tests/wpt/blob/master/resources/idlharness.js#L3483-L3494) actually uses. So I think the simplest solution would be to inject a (very basic) `fetch` shim that only serves text files:
```javascript
const readFileAsync = promisify(fs.readFile);
const wptPath = path.resolve(__dirname, 'web-platform-tests');

const failures = await wptRunner(testsPath, {
  // ...
  setup(window) {
    window.queueMicrotask = queueMicrotask;
    window.fetch = async function (url) {
      const filePath = path.join(wptPath, url);
      if (!filePath.startsWith(wptPath)) {
        throw new TypeError('Invalid URL');
      }
      return {
        ok: true,
        async text() {
          return await readFileAsync(filePath, { encoding: 'utf8' });
        }
      };
    };
    window.eval(bundledJS);
  },
}
```

And that's it: with those things in place, plus the fixes to the web platform tests themselves, all IDL tests run smoothly.

So, what do you think? 🙂

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/streams/issues/1044#issuecomment-647189179