Re: Test suite for the Presentation API (was: Test Facilitator for the Presentation API)

Hi Zhiqiang,

> On 17 Dec 2015, at 02:27, Zhang, Zhiqiang <zhiqiang.zhang@intel.com> wrote:
> 
> Hi Anssi, Louay,
> 
>> From: Kostiainen, Anssi
>> Sent: Wednesday, December 16, 2015 7:33 PM
>> To: Bassbouss, Louay <louay.bassbouss@fokus.fraunhofer.de>; Zhang,
>> Zhiqiang <zhiqiang.zhang@intel.com>
>> Cc: public-secondscreen@w3.org
>> Subject: Test suite for the Presentation API (was: Test Facilitator for the
>> Presentation API)
>> 
>> Hi All, Louay, Zhiqiang,
>> 
>>> On 15 Dec 2015, at 19:09, Bassbouss, Louay
>> <louay.bassbouss@fokus.fraunhofer.de> wrote:
>>> 
>>> I added a new Testing section [1] in the wiki that describes how to setup
>> the test environment and how to write tests for the Presentation API. As you
>> know, we will use the Web Platform Tests Project [2] as a Test Suite for the
>> Presentation API. I already submitted a new PR [3] with initial folder
>> structure and IDLharness for the Presentation API. once this PR is merged in
>> the main repository we will have an additional folder “presentation-api” in
>> the web-platform-tests project similar to other W3C API. inside this folder
>> we will have two subfolders “controlling-ua” and “receiving-ua” to test the
>> Presentation API on controlling and receiving user agents. The initial version I
>> submitted contains WebIDL tests of the Presentation API (each of the
>> “controlling-ua” and “receiving-ua” contains a test file “idlharness.html”).
>> After the PR is merged, you are welcomed to submit tests ;). I will share with
>> you more details about test strategy of the Presentation API in the next days.
>> Looking forward for your feedback and comments.
> 
> Great start.
> 
>> Louay - thank you for bootstrapping the testing effort for the Presentation
>> API! When the initial test suite PR has landed, I suggest you update the spec
>> adding a pointer to the test suite to give it more visibility (otherLinks in
>> respecConfig).
>> 
>> Some questions:
>> 
>> - In this initial contribution it is proposed the test suite tracks the latest spec
>> published on TR. If we follow this practice, we should make sure we publish
>> new WDs frequently enough to make the TR spec a good synchronisation
>> point (Echidna makes that very easy). Or alternatively, we could agree we
>> write tests against the latest ED spec. A downside is ED may have more spec
>> churn, so writing tests against it may be more risky for people who are not
>> following the spec development closely. However, my assumption is that
>> most of the test contributions would come from the participants of this
>> group though, so that would not be a major issue. Your thoughts?
> 
> Tests based on ED looks good to me.
I will submit a new PR targeting tests based on ED.
> 
>> - What are your thoughts, should we use Critic for this spec's test suite or just
>> stick with the GH based workflow? I'm not very familiar with Critic, so I'm
>> interesting in hearing feedback from people who are more familiar with Critic
>> (Zhiqiang?) on cases when it is particularly useful.
> 
> Both Critic and GH are OK and acceptable, feel free to use your familiar one or both. Anyone is welcome to review the tests.
I think all group members are more familiar with GH let´s use it also for test review.   Do you have any information you can share with the group about the review process (like what are the criteria to accept or a test, …).
> 
>> Zhiqiang - at our recent F2F you indicated interested to contribute to testing
>> of this spec. Could you please review [3] and provide feedback in the PR (or
>> in Critic if you think that'd be a helpful tool to use). Feel free to merge the PR
>> after review.
> 
> Yes, I will review it, but please expect this is to be done later next week. If someone reviews it before that time and at least leave a LGTM comment, I can help merge it without my review.
+1
> 
> Thanks,
> Zhiqiang
> 
Thx,
Louay

Received on Thursday, 17 December 2015 07:50:10 UTC