- From: Matt Wolenetz <wolenetz@google.com>
- Date: Thu, 8 Sep 2016 06:48:00 -0700
- To: Francois Daoust <fd@w3.org>
- Cc: "Philippe Le Hegaret (plh@w3.org)" <plh@w3.org>, "<public-html-media@w3.org>" <public-html-media@w3.org>, Paul Cotton <Paul.Cotton@microsoft.com>
- Message-ID: <CAADho6M464B9PTMBOMqJ=WVUPhz2U+BPTCHGAE_eHupvCUFuzg@mail.gmail.com>
That makes sense. Thank you for finding and fixing the test problem.
On Sep 8, 2016 6:09 AM, "Francois Daoust" <fd@w3.org> wrote:
> Matt,
>
> In essence, I think that the test harness, called by the IDL harness,
> expects to get the list of tests before the "load" event fires on window.
> In other words, it expects all calls to "test" and "async_test" to have
> been made by then.
>
> In the MSE case, we need to wait for the MediaSource object to be ready
> before we may call the IDL harness, and the "load" event will have already
> fired at that point. The IDL harness tells the test harness to run the
> first test, and since the "load" event already fired, the test harness
> believes it is the only test to run, and switches to an "I'm done!" mode,
> getting ready to report the results. Problem is this is when it receives a
> number of additional tests to perform. This makes the test harness go
> berserk.
>
> The solution is relatively simple: we just need to be explicit as to when
> the list of tests is complete, which can be done with an initial call to
> "setup({ explicit_done: true });" followed by a call to "done()" after
> having run the IDL harness.
>
> Done in the following PR:
> https://github.com/w3c/web-platform-tests/pull/3664
>
> Francois.
>
>
> Le 07/09/2016 à 23:22, Matt Wolenetz a écrit :
>
>> Thanks Philippe. Help with the IDL failures would be appreciated -- it
>> looks like it's not specific to just Chrome. As mentioned in narrower
>> thread, I've updated and landed #3655, and will rerun Chrome's JSON (for
>> builds with, and without, ISO-BMFF support claimed by isTypeSupported())
>> once I land https://codereview.chromium.org/2315113002/ (which should
>> fix a bunch of the TypeError-related Chrome failures). Landing that
>> Chrome patch may not come before EOD today, though.
>>
>> On Wed, Sep 7, 2016 at 12:03 PM, Philippe Le Hégaret <plh@w3.org
>> <mailto:plh@w3.org>> wrote:
>>
>>
>>
>> On 9/2/2016 7:24 PM, Matt Wolenetz wrote:
>>
>>> I've made some progress, not as much as I had hoped. 2 of my PRs
>>> are currently pending co-editor review. 5 pending PRs are still
>>> awaiting my full review.
>>>
>>
>> Thank you Matt for making progress!
>>
>> I merged one of your PRs, still looking at #3655.
>>
>> I regenerated test results and added a filter to eliminate
>> VideoPlaybackQuality and TrackDefault related:
>> http://w3c.github.io/test-results/media-source/less-than-2.html
>> <http://w3c.github.io/test-results/media-source/less-than-2.html>
>>
>> We get a lot of errors on the idlharness tests which I'll look at as
>> well.
>>
>> Philippe
>>
>>
>> Detail:
>>>
>>> * Prepped PR to cover 4 gaps from #823
>>> : https://github.com/w3c/web-platform-tests/pull/3635
>>> <https://github.com/w3c/web-platform-tests/pull/3635>
>>> o *Awaiting @jdsmith3000's review to land it*
>>> o crbug 623781 <http://crbug/623781> tracks fixing the
>>> Chrome compliance issue behind at least the failure of 2
>>> subtests (TypeError) in mediasource-addsourcebuffer.ht
>>> <http://mediasource-addsourcebuffer.ht>ml
>>> o Filed crbug 643788 <http://crbug/643788> to fix debug
>>> build assertion in w-p-t
>>> mediasource-removesourcebuffers.html and landed my fix for
>>> it <https://codereview.chromium.org/2305023002>
>>>
>>> * Reviewed and closed
>>> ancient https://github.com/w3c/web-platform-tests/pull/939
>>> <https://github.com/w3c/web-platform-tests/pull/939>
>>> o Found only 1 trivial test coverage gap, prepped and merged
>>> trivial PR to close that
>>> gap: https://github.com/w3c/web-platform-tests/pull/3636
>>> <https://github.com/w3c/web-platform-tests/pull/3636>
>>> * Deferred to VNext ancient appendStream coverage test
>>> PR https://github.com/w3c/web-platform-tests/pull/954
>>> <https://github.com/w3c/web-platform-tests/pull/954>
>>> * Deferred to VNext more strict isTypeSupported() coverage test
>>> PR https://github.com/w3c/web-platform-tests/pull/1405
>>> <https://github.com/w3c/web-platform-tests/pull/1405>
>>> *
>>>
>>> Commented on mechanical
>>> PR https://github.com/w3c/web-platform-tests/pull/1816
>>> <https://github.com/w3c/web-platform-tests/pull/1816> (doesn't
>>> block MSE v1 PR, it's a cross-cutting infra PR)
>>>
>>> *
>>>
>>> In-progress:
>>> Fixing https://github.com/w3c/web-platform-tests/pull/3082
>>> <https://github.com/w3c/web-platform-tests/pull/3082>
>>>
>>> o Filed, and in-progress fixing (short-term Chrome MSE piece
>>> of) crbug 643846 <http://crbug/643846>
>>> *
>>>
>>> TODO 5 more pending PRs...
>>>
>>>
>>> On Tue, Aug 30, 2016 at 6:42 PM, Matt Wolenetz
>>> <wolenetz@google.com <mailto:wolenetz@google.com>> wrote:
>>>
>>>
>>> On Tue, Aug 30, 2016 at 11:20 AM, Paul Cotton
>>> <Paul.Cotton@microsoft.com <mailto:Paul.Cotton@microsoft.com>>
>>> wrote:
>>>
>>> *MSE test suite results*
>>>
>>> <*/plh/*> Paul: still some outstanding pull requests
>>>
>>> https://lists.w3.org/Archives/Public/public-html-media/2016A
>>> ug/0102.html
>>> <https://lists.w3.org/Archives/Public/public-html-media/
>>> 2016Aug/0102.html>
>>> is Matt's most recent update
>>>
>>> <*/plh/*> Matt: going through them...
>>>
>>> Outstanding PRs:
>>> https://github.com/w3c/web-platform-tests/pulls?q=is%3Apr+
>>> is%3Aopen+label%3Amedia-source
>>> <https://github.com/w3c/web-platform-tests/pulls?q=is%3Apr+
>>> is%3Aopen+label%3Amedia-source>
>>>
>>> <*/plh/*> ... resolving the merge conflict
>>>
>>> <*/plh/*> ... I'll send out an update later today
>>>
>>>
>>> I've worked through about 5 of the 15 pending PRs today, and
>>> will continue tomorrow. Current status:
>>>
>>> *
>>>
>>>
>>> * Merged
>>> @tidoust's https://github.com/w3c/web-pla
>>> tform-tests/pull/3239
>>> <https://github.com/w3c/web-platform-tests/pull/3239>
>>> * Prepped and landed
>>> my https://github.com/w3c/web-platform-tests/pull/3612
>>> <https://github.com/w3c/web-platform-tests/pull/3612>
>>> * Needed Chromium+W3C license stamp, so this
>>> replaces/extends
>>> @tidoust's https://github.com/w3c/web-pla
>>> tform-tests/pull/3264
>>> <https://github.com/w3c/web-platform-tests/pull/3264>
>>> * Updated and landed
>>> my https://github.com/w3c/web-platform-tests/pull/3296
>>> <https://github.com/w3c/web-platform-tests/pull/3296>
>>> * Landed
>>> @tidoust's https://github.com/w3c/web-pla
>>> tform-tests/pull/3233
>>> <https://github.com/w3c/web-platform-tests/pull/3233>
>>> * Prepped PR to fix the test introduced in #3233 to comply
>>> with
>>> w3c/media-source#154: https://github.com/w3c/web-pla
>>> tform-tests/pull/3613
>>> <https://github.com/w3c/web-platform-tests/pull/3613>
>>> o *Awaiting @jdsmith3000's review to land it*
>>> o crbug 639144 <http://crbug/639144> tracks fixing the
>>> Chrome compliance issue behind failure of subtest 1/3
>>> in this PR
>>> o crbug 373039 <http://crbug/373039> is one of a group
>>> of bugs tracking fixing the Chrome compliance issue
>>> behind failure of subtest 3/3 in this PR
>>> * Reviewed
>>> ancient https://github.com/w3c/web-platform-tests/pull/823
>>> <https://github.com/w3c/web-platform-tests/pull/823>
>>> o Found 4 gaps, but otherwise closed this PR since most
>>> of the coverage it includes is already covered
>>> elsewhere in our current suite
>>> *
>>>
>>> In-progress: put together PR to cover 4 gaps from #823
>>>
>>> o crbug 623781 <http://crbug/623781> tracks fixing the
>>> Chrome compliance issue behind at least the failure of
>>> 2 subtests (TypeError) in
>>> mediasource-addsourcebuffer.ht
>>> <http://mediasource-addsourcebuffer.ht>ml
>>> o *In-progress: Investigating Chromium issue with PR I'm
>>> building to cover the 4 test gaps.*
>>> * TODO 10 more pending PRs, followed by call to regenerate
>>> reports and lists of known user agent bugs tracking test
>>> failures.
>>>
>>>
>>>
>>
>>
Received on Thursday, 8 September 2016 13:48:30 UTC