Re: MSE testing status update

Thanks Mike. Today, I produced updated MSE test suite results [1] for:

* Chrome 43 (with --enable-experimental-web-platform-features)
* Spartan 15.10130.0.0
* Firefox 38.5 (with a few about:config settings that look like they
enabled MSE for the test suite URL as well as WebM in MSE)

Jerry, I went ahead and produced those Spartan results because I was
curious. Of course, please feel free to review or update them. Aside from
lack of WebM support, there are a large number of failures for Spartan that
may need higher prioritization. I'm doing similar for the Chrome failures
and expect to post in the next couple weeks an updated compilation of known
spec test coverage gaps so that we can accelerate implementation progress.

Notably, Firefox 38.5 has a relatively high test-passing rate, too, which I
believe is excellent news for the MSE spec!

Matt

[1]
https://github.com/w3c/test-results/commit/2a43bf44cd1d15736fb6463812c7be80170c1984

On Tue, Jun 23, 2015 at 4:25 AM, Michael[tm] Smith <mike@w3.org> wrote:

> Hi Matt,
>
> Matt Wolenetz <wolenetz@google.com>, 2015-06-16 07:58 -0700:
> > Archived-At: <
> http://www.w3.org/mid/CAADho6NHF=oOC0sNz_Tw1BuiKsF=JgJ8mLMjW+=VzZojtpd1FQ@mail.gmail.com
> >
> ...
> > Next steps will be to continue closing the test coverage gaps and
> updating
> > implementations (in my case, Chrome's implementation). Chrome, in
> > particular, would pass more of the existing tests if the
> > --enable-experimental-web-platform-features flag were used. I did not use
> > that flag when generating the test results.
>
> Actually, it’s fine to test with such flags/options enabled, and to
> generate reports of test results from test runs with those enabled.
>
> That’s because at least at this point the main goal, as far as W3C progress
> of the spec is concerned, is just to document that it’s possible to
> independently implement the spec (by having at least two UAs that have
> conforming implementations of each requirement in the spec).
>
> It’s not a goal that the tests must demonstrate that support for any
> particular requirement has actually shipped and is exposed to all users by
> default without being behind a flag or some other opt-in mechanism.
>
>   —Mike
>
> > [1] http://w3c.github.io/test-results/media-source/all.html
> > [2]
> http://lists.w3.org/Archives/Public/public-html-media/2014Dec/0012.html
>
> --
> Michael[tm] Smith https://people.w3.org/mike
>