W3C home > Mailing lists > Public > public-html-media@w3.org > September 2016

Re: MSE IDL tests (was: Re: HME WG meeting minutes, Tue Aug 30)

From: Matt Wolenetz <wolenetz@google.com>
Date: Thu, 8 Sep 2016 06:48:00 -0700
Message-ID: <CAADho6M464B9PTMBOMqJ=WVUPhz2U+BPTCHGAE_eHupvCUFuzg@mail.gmail.com>
To: Francois Daoust <fd@w3.org>
Cc: "Philippe Le Hegaret (plh@w3.org)" <plh@w3.org>, "<public-html-media@w3.org>" <public-html-media@w3.org>, Paul Cotton <Paul.Cotton@microsoft.com>
That makes sense. Thank you for finding and fixing the test problem.

On Sep 8, 2016 6:09 AM, "Francois Daoust" <fd@w3.org> wrote:

> Matt,
>
> In essence, I think that the test harness, called by the IDL harness,
> expects to get the list of tests before the "load" event fires on window.
> In other words, it expects all calls to "test" and "async_test" to have
> been made by then.
>
> In the MSE case, we need to wait for the MediaSource object to be ready
> before we may call the IDL harness, and the "load" event will have already
> fired at that point. The IDL harness tells the test harness to run the
> first test, and since the "load" event already fired, the test harness
> believes it is the only test to run, and switches to an "I'm done!" mode,
> getting ready to report the results. Problem is this is when it receives a
> number of additional tests to perform. This makes the test harness go
> berserk.
>
> The solution is relatively simple: we just need to be explicit as to when
> the list of tests is complete, which can be done with an initial call to
> "setup({ explicit_done: true });" followed by a call to "done()" after
> having run the IDL harness.
>
> Done in the following PR:
> https://github.com/w3c/web-platform-tests/pull/3664
>
> Francois.
>
>
> Le 07/09/2016 à 23:22, Matt Wolenetz a écrit :
>
>> Thanks Philippe. Help with the IDL failures would be appreciated -- it
>> looks like it's not specific to just Chrome. As mentioned in narrower
>> thread, I've updated and landed #3655, and will rerun Chrome's JSON (for
>> builds with, and without, ISO-BMFF support claimed by isTypeSupported())
>> once I land https://codereview.chromium.org/2315113002/ (which should
>> fix a bunch of the TypeError-related Chrome failures). Landing that
>> Chrome patch may not come before EOD today, though.
>>
>> On Wed, Sep 7, 2016 at 12:03 PM, Philippe Le Hégaret <plh@w3.org
>> <mailto:plh@w3.org>> wrote:
>>
>>
>>
>>     On 9/2/2016 7:24 PM, Matt Wolenetz wrote:
>>
>>>     I've made some progress, not as much as I had hoped. 2 of my PRs
>>>     are currently pending co-editor review. 5 pending PRs are still
>>>     awaiting my full review.
>>>
>>
>>     Thank you Matt for making progress!
>>
>>     I merged one of your PRs, still looking at #3655.
>>
>>     I regenerated test results and added a filter to eliminate
>>     VideoPlaybackQuality and TrackDefault related:
>>       http://w3c.github.io/test-results/media-source/less-than-2.html
>>     <http://w3c.github.io/test-results/media-source/less-than-2.html>
>>
>>     We get a lot of errors on the idlharness tests which I'll look at as
>>     well.
>>
>>     Philippe
>>
>>
>>     Detail:
>>>
>>>       * Prepped PR to cover 4 gaps from #823
>>>         : https://github.com/w3c/web-platform-tests/pull/3635
>>>         <https://github.com/w3c/web-platform-tests/pull/3635>
>>>           o *Awaiting @jdsmith3000's review to land it*
>>>           o crbug 623781 <http://crbug/623781> tracks fixing the
>>>             Chrome compliance issue behind at least the failure of 2
>>>             subtests (TypeError) in mediasource-addsourcebuffer.ht
>>>             <http://mediasource-addsourcebuffer.ht>ml
>>>           o Filed crbug 643788 <http://crbug/643788> to fix debug
>>>             build assertion in w-p-t
>>>             mediasource-removesourcebuffers.html and landed my fix for
>>>             it <https://codereview.chromium.org/2305023002>
>>>
>>>       * Reviewed and closed
>>>         ancient https://github.com/w3c/web-platform-tests/pull/939
>>>         <https://github.com/w3c/web-platform-tests/pull/939>
>>>           o Found only 1 trivial test coverage gap, prepped and merged
>>>             trivial PR to close that
>>>             gap: https://github.com/w3c/web-platform-tests/pull/3636
>>>             <https://github.com/w3c/web-platform-tests/pull/3636>
>>>       * Deferred to VNext ancient appendStream coverage test
>>>         PR https://github.com/w3c/web-platform-tests/pull/954
>>>         <https://github.com/w3c/web-platform-tests/pull/954>
>>>       * Deferred to VNext more strict isTypeSupported() coverage test
>>>         PR https://github.com/w3c/web-platform-tests/pull/1405
>>>         <https://github.com/w3c/web-platform-tests/pull/1405>
>>>      *
>>>
>>>         Commented on mechanical
>>>         PR https://github.com/w3c/web-platform-tests/pull/1816
>>>         <https://github.com/w3c/web-platform-tests/pull/1816> (doesn't
>>>         block MSE v1 PR, it's a cross-cutting infra PR)
>>>
>>>      *
>>>
>>>         In-progress:
>>>         Fixing https://github.com/w3c/web-platform-tests/pull/3082
>>>         <https://github.com/w3c/web-platform-tests/pull/3082>
>>>
>>>           o Filed, and in-progress fixing (short-term Chrome MSE piece
>>>             of) crbug 643846 <http://crbug/643846>
>>>      *
>>>
>>>         TODO 5 more pending PRs...
>>>
>>>
>>>     On Tue, Aug 30, 2016 at 6:42 PM, Matt Wolenetz
>>>     <wolenetz@google.com <mailto:wolenetz@google.com>> wrote:
>>>
>>>
>>>         On Tue, Aug 30, 2016 at 11:20 AM, Paul Cotton
>>>         <Paul.Cotton@microsoft.com <mailto:Paul.Cotton@microsoft.com>>
>>>         wrote:
>>>
>>>             *MSE test suite results*
>>>
>>>             <*/plh/*> Paul: still some outstanding pull requests
>>>
>>>             https://lists.w3.org/Archives/Public/public-html-media/2016A
>>> ug/0102.html
>>>             <https://lists.w3.org/Archives/Public/public-html-media/
>>> 2016Aug/0102.html>
>>>             is Matt's most recent update
>>>
>>>             <*/plh/*> Matt: going through them...
>>>
>>>             Outstanding PRs:
>>>             https://github.com/w3c/web-platform-tests/pulls?q=is%3Apr+
>>> is%3Aopen+label%3Amedia-source
>>>             <https://github.com/w3c/web-platform-tests/pulls?q=is%3Apr+
>>> is%3Aopen+label%3Amedia-source>
>>>
>>>             <*/plh/*> ... resolving the merge conflict
>>>
>>>             <*/plh/*> ... I'll send out an update later today
>>>
>>>
>>>         I've worked through about 5 of the 15 pending PRs today, and
>>>         will continue tomorrow. Current status:
>>>
>>>          *
>>>
>>>
>>>           * Merged
>>>             @tidoust's https://github.com/w3c/web-pla
>>> tform-tests/pull/3239
>>>             <https://github.com/w3c/web-platform-tests/pull/3239>
>>>           * Prepped and landed
>>>             my https://github.com/w3c/web-platform-tests/pull/3612
>>>             <https://github.com/w3c/web-platform-tests/pull/3612>
>>>           * Needed Chromium+W3C license stamp, so this
>>>             replaces/extends
>>>             @tidoust's https://github.com/w3c/web-pla
>>> tform-tests/pull/3264
>>>             <https://github.com/w3c/web-platform-tests/pull/3264>
>>>           * Updated and landed
>>>             my https://github.com/w3c/web-platform-tests/pull/3296
>>>             <https://github.com/w3c/web-platform-tests/pull/3296>
>>>           * Landed
>>>             @tidoust's https://github.com/w3c/web-pla
>>> tform-tests/pull/3233
>>>             <https://github.com/w3c/web-platform-tests/pull/3233>
>>>           * Prepped PR to fix the test introduced in #3233 to comply
>>>             with
>>>             w3c/media-source#154: https://github.com/w3c/web-pla
>>> tform-tests/pull/3613
>>>             <https://github.com/w3c/web-platform-tests/pull/3613>
>>>               o *Awaiting @jdsmith3000's review to land it*
>>>               o crbug 639144 <http://crbug/639144> tracks fixing the
>>>                 Chrome compliance issue behind failure of subtest 1/3
>>>                 in this PR
>>>               o crbug 373039 <http://crbug/373039> is one of a group
>>>                 of bugs tracking fixing the Chrome compliance issue
>>>                 behind failure of subtest 3/3 in this PR
>>>           * Reviewed
>>>             ancient https://github.com/w3c/web-platform-tests/pull/823
>>>             <https://github.com/w3c/web-platform-tests/pull/823>
>>>               o Found 4 gaps, but otherwise closed this PR since most
>>>                 of the coverage it includes is already covered
>>>                 elsewhere in our current suite
>>>          *
>>>
>>>             In-progress: put together PR to cover 4 gaps from #823
>>>
>>>               o crbug 623781 <http://crbug/623781> tracks fixing the
>>>                 Chrome compliance issue behind at least the failure of
>>>                 2 subtests (TypeError) in
>>>                 mediasource-addsourcebuffer.ht
>>>                 <http://mediasource-addsourcebuffer.ht>ml
>>>               o *In-progress: Investigating Chromium issue with PR I'm
>>>                 building to cover the 4 test gaps.*
>>>           * TODO 10 more pending PRs, followed by call to regenerate
>>>             reports and lists of known user agent bugs tracking test
>>>             failures.
>>>
>>>
>>>
>>
>>
Received on Thursday, 8 September 2016 13:48:30 UTC

This archive was generated by hypermail 2.3.1 : Thursday, 8 September 2016 13:48:31 UTC