Re: [widgets] Test Suite Creation

On Tue, Aug 4, 2009 at 3:02 PM, Dominique Hazael-Massieux<dom@w3.org> wrote:
> Le vendredi 31 juillet 2009 à 18:01 +0200, Marcos Caceres a écrit :
>> I've created the first draft of the test suite edition, it is
>> available here (it's ugly on purpose, I will remove the ugly
>> stylesheet soon):
>>
>> http://dev.w3.org/2006/waf/widgets/Overview_TSE.html
>>
>> Includes stable IDs on <p>'s, and HTML-class-based identification of
>> products to which assertions apply:
>>
>> either: "product-cc" or "product-ua"
>
> Great work, thanks! if you can join the MWTS call today, we can settle
> on who's going to update the test plan and how - your idea of separating
> the assertions from the associated metadata sounds good to me.

Ok, I'll join the call.

I have now gone through every assertion, made sure that products are
clearly identified. I believe I've weeded out all redundant
assertions. The spec is much leaner now. Also, all assertions are now
in active voice.

I've also address two bugs identified in [1] and [2].

At the bottom of the spec [3] one can now find two dynamically
generated assertion tables (requires a browser that supports Selectors
API, e.g., Opera 10beta, FF3.5, etc.). These assertion tables list
every testable assertion per product. This completes work items 1-4 of
my previous email.

What is now left is the following work items:

5. Design the template, naming convention, and metadata for tests.

I think for metadata we should have a simple XML file (or JSON) that
lists each test:

<testsuite src="http://dev.w3.org/2006/waf/widgets/Overview_TSE.html">
  <test for="ta-testID" src="someWidget.wgt"> Description of test </test>
 <test ... />
</testsuite>

I could then XHR the above file and mash it up with the spec [3].
Using the @for attribute, I can then list the test that apply to an
assertion in the appropriate place in the assertion table. This could
also make for a nice test harness to automate testing.

6. Create the tests for each assertion
Still not sure how we verify that a test correctly tests an assertion
(need QA process).


[1] http://lists.w3.org/Archives/Public/public-webapps/2009JulSep/0453.html
[2] http://lists.w3.org/Archives/Public/public-webapps/2009JulSep/0452.html
[3] http://dev.w3.org/2006/waf/widgets/Overview_TSE.html
-- 
Marcos Caceres
http://datadriven.com.au

Received on Tuesday, 4 August 2009 13:34:58 UTC