Test suite for the Presentation API (was: Test Facilitator for the Presentation API)

Hi All, Louay, Zhiqiang,

> On 15 Dec 2015, at 19:09, Bassbouss, Louay <louay.bassbouss@fokus.fraunhofer.de> wrote:
> 
> I added a new Testing section [1] in the wiki that describes how to setup the test environment and how to write tests for the Presentation API. As you know, we will use the Web Platform Tests Project [2] as a Test Suite for the Presentation API. I already submitted a new PR [3] with initial folder structure and IDLharness for the Presentation API. once this PR is merged in the main repository we will have an additional folder “presentation-api” in the web-platform-tests project similar to other W3C API. inside this folder we will have two subfolders “controlling-ua” and “receiving-ua” to test the Presentation API on controlling and receiving user agents. The initial version I submitted contains WebIDL tests of the Presentation API (each of the “controlling-ua” and “receiving-ua” contains a test file “idlharness.html”). After the PR is merged, you are welcomed to submit tests ;). I will share with you more details about test strategy of the Presentation API in the next days. Looking forward for your feedback and comments.

Louay - thank you for bootstrapping the testing effort for the Presentation API! When the initial test suite PR has landed, I suggest you update the spec adding a pointer to the test suite to give it more visibility (otherLinks in respecConfig).

Some questions:

- In this initial contribution it is proposed the test suite tracks the latest spec published on TR. If we follow this practice, we should make sure we publish new WDs frequently enough to make the TR spec a good synchronisation point (Echidna makes that very easy). Or alternatively, we could agree we write tests against the latest ED spec. A downside is ED may have more spec churn, so writing tests against it may be more risky for people who are not following the spec development closely. However, my assumption is that most of the test contributions would come from the participants of this group though, so that would not be a major issue. Your thoughts?

- What are your thoughts, should we use Critic for this spec's test suite or just stick with the GH based workflow? I'm not very familiar with Critic, so I'm interesting in hearing feedback from people who are more familiar with Critic (Zhiqiang?) on cases when it is particularly useful.

Zhiqiang - at our recent F2F you indicated interested to contribute to testing of this spec. Could you please review [3] and provide feedback in the PR (or in Critic if you think that'd be a helpful tool to use). Feel free to merge the PR after review.

Thanks,

-Anssi (WG chair)


> Thx,
> Louay
> 
> [1] https://www.w3.org/wiki/Second_Screen/Work_Mode#Testing

> [2] https://github.com/w3c/web-platform-tests

> [3] https://github.com/w3c/web-platform-tests/pull/2432

Received on Wednesday, 16 December 2015 11:33:24 UTC