RE: MSE test suite status update

Hi all,

Here is an update on the status update.

> De : Francois Daoust [mailto:fd@w3.org]
[...]
> The evaluation
> and action plan are summarized in:
> 
> http://tidoust.github.io/media-source-testcoverage/
> 
> We expect to make progress on the action plan within the next couple of
> weeks.

Some actions are still on-going, but new tests are now pending review:
https://github.com/w3c/web-platform-tests/pulls?q=is%3Apr+is%3Aopen+label%3Amedia-source

I updated the implementation report:
http://w3c.github.io/test-results/media-source/all.html
http://w3c.github.io/test-results/media-source/less-than-2.html
http://w3c.github.io/test-results/media-source/complete-fails.html

In these reports:
- CH54 stands for Chrome Canary 54.0.2794.0 launched with the
    --enable-experimental-web-platform-features flag set
- ED14 stands for Microsoft Edge Home Insider Preview Build 14372
- FF50 stands for Firefox Nightly 50.0a1 (2016-07-12)
- SA09 stands for Safari 9.1.1 (11601.6.17)

Although the report still contains a few FAILs, it actually starts to look good! Main basic features defined in the spec seem supported across browsers in particular.

The latest version of Chrome Canary passes most of the tests in the test suite. For some reason, IDL tests fail (the IDL harness loops forever). I do not know why. I believe most of these tests would pass though. Remaining failures seem relatively minor:
- The exceptions raised are not always the right ones
- Chrome does not seem to support enabling/disabling audio/video tracks
- Some hiccups with the "sequence" mode as far as I can tell

Support in other browsers seems fine too, although some features are not yet supported, basically the same ones as previously reported:
1. the on{event} attributes
2. live seekable range
3. audioTracks and videoTracks in Firefox
4. resetting the delaying-the-load event in Edge (this makes most test files timeout but note individual tests may still pass)
5. TrackDefault (feature marked at-risk)

The first 2 features are probably worth focusing on in priority in implementations. One can probably argue that the rest should not block the spec from progressing forward along the Rec track.

Note that "appendStream" is still not tested and not supported across browsers.

Thanks,
Francois.

Received on Tuesday, 12 July 2016 16:45:46 UTC