MWI Test Suite Framework

Mobile Web initiative, Test Suite Overview (Strawman Version)

A Mobile Web Test Suite, MWTS can be thought of as a collection of test suites from different, but related specifications (XHTMP-Print, SVG Tiny, CSS, etc) grouped under a <test-suite> element. A set of <test-suite-info>, <citations>, <roles>, <sources> and <test-collections> elements defines all the metadata that is required to define and execute each of the tests.

The <test-suite-info> element identifies the test suite. The <citations> element identifies each of the documents associated with the relevant specification or the specifications themselves.

The <sources> element identify each of the sources that will be used in test cases. Test cases will use these ID's to identify their source documents, or other pertinent data. The <roles> element enumerate a set of possible values that will be used by all the test cases. Test cases will use these ID's to indicate the role a source/output file will play during the test case execution.

<test-suite>

<test-suite-info>

<roles>

<test-collections>

</test-suite>

A <test-collections> element contains one or more <test-collection> elements, which in turn have a “name” and “url” attribute pair. This pair uniquely identifies a member Test suite (i.e. CSS 1).

<test-suite>

<test-collections>

<collection name = “CSS 1”

url = “http://www.w3.org/Style/CSS/Test/CSS1/current/”>

…

</collection>

<collection name = “XHTML Print”

url = “http://www.w3.org/MarkUp/Test/xhtml-print/current/”>

…

</collection>

</test-collections>

</test-suite>

Given that the MWTS is comprised of tests from different tests suites, whose organization may differ from one another, a <test-group> element identify any hierarchy particular to a test suite (a <test-group> may contain nested <test-group> elements in turn) as outlined below:

<test-group name = "PrinterConformance" url = "http://www.w3.org/MarkUp/Test/xhtm

print/current/tests/">

<title>XHTML Printer tests</title>

</GroupInfo>

The “name”, ”url” attributes from a <test-group> element provide a name and a namespace URI respectively for each of the test groups associated with the set. The <GroupInfo>/<title>/<description> elements provides more detailed documentation pertaining this set.

Associated with a <test-group> there are a number of test cases, which are the building blocks of the test suite as outlined by the fragment below:

<test-group>

…

<test-case name="sec21.htm" scenario="standard">

<description>Anchor Pseudo Classes Evaluation</description>

<spec-citation spec="CSS1" section-number="2.1" section-title="Anchor pseudo-classes"

section-pointer="Anchor-pseudo-classes"/>

<input-file role="principal-data">base</input-file>

</test-case>
…

</test-group>

A <test-case> is identified by a “name” attribute, which uniquely identifies the test case. A “scenario” attribute identify the scenarios that will be used in the test cases. This is more of a documentation item. The <description> element contains a brief explanation of the test purpose. The <spec-citation> element identifies part of the specifications being tested. The “name” attribute of the <test> element is almost always the same as the name of the test case. Its main purpose is to establish any conceptual/pragmatic differences between the test suite it originally belongs to and the MWTS. A <test-case> element may contain zero or more <input-file> elements. These IDREF's values provide any input sources required to execute the test case. A <test-case> element may also contain zero or more <output-file> elements, which identify the all expected results for this test case. Most of the cases may require a visual inspection of the results and thus this element may not be present.

The following non validated xml infoset reflect the ideas outlined above. Please note that not all elements/attributes were described.

<?xml version="1.0" encoding="UTF-8"?>

<test-suite xmlns=http://www.w3.org/2006/08/MWI-Catalog

CatalogDesignDate="2006-08-15" version="1.0.0 dev">

<test-suite-info>

<title>Mobile Web Initiative Test Suite 0.1</title>

Test Suite collections for Mobile Web Initiative

http://www.w3.org/Mobile/

</description>

</test-suite-info>

<citation-spec name="CSS1">

<description last-mod="2006-08-15">Cascading Sytlesheets Level/>

<spec-URI>http://www.w3.org/TR/CSS1</spec-URI>

</citation-spec>

<citation-spec name="SVG-Tiny">

<description last-mod="2006-08-15">Scalable vector Graphics, 1.1

specifications.

</description>

<spec-URI>http://www.w3.org/TR/SVG11/</spec-URI>

</citation-spec>

<citation-spec name="XHTML-Print">

<description last-mod="2006-08-15">XHTML Print Candidate

Recommendation</description>

<spec-URI>http://www.w3.org/TR/xhtml-print/</spec-URI>

</citation-spec>

</citations>

<description last-mod="2005-04-14">Basic stylesheet used by all

tests.

</description>

</source>

<description last-mod="2005-04-14">A link to test2 from this

file

</description>

</source>

<description last-mod="2005-04-14">Imported cascading stylesheet

for "sec11".

</description>

</source>

<description last-mod="2005-04-14">Imported cascading stylesheet for

"sec11.

</description>

</source>

<description last-mod="2005-04-14">Imported cascading stylesheet for

"sec11".

</description>

</source>

</sources>

A test case that is expected to produce valid results.

</description>

</scenario>

A test case that is expected to produce to produce some kind of an

error (parse or runtime).

</description>

</scenario>

</scenarios>

<test-suite-collections>

<collection

name = "CSS 1"

url = “http://www.w3.org/Style/CSS/Test/CSS1/current/">

<test-group name = "Basic-Concepts">

<title>Basic Concepts for the CSS level 1 Test Suite</title>

</GroupInfo>

<test-case name="sec11.html" scenario="standard">

<description>Basic Containment test for CSS level 1.</description>

<spec-citation

spec="CSS1" section-number="1.1"

section-title="Containment in HTML"

section-pointer="basic-concepts"/>

<input-file role="principal-data">base</input-file>

<input-file role="principal-data">linktest2</input-file>

<input-file role="principal-data">imptest1</input-file>

<input-file role="principal-data">imptest1a</input-file>

<input-file role="principal-data">imptest2</input-file>

</test-case>

</test-group>

<test-group name = "Pseudo-Classes-and-Pseudo-Elements">

<title> Pseudo-Classes and Pseudo-Elements</title>

</GroupInfo>

<test-case name="sec21.htm" scenario="standard">

<description>Anchor Pseudo Classes Evaluation</description>

<spec-citation

spec="CSS1" section-number="2.1"

section-title="Anchor pseudo-classes"

section-pointer="Anchor-pseudo-classes"/>

<input-file role="principal-data">base</input-file>

</test-case>

</test-group>

</collection>

<collection

name = "SVG Tiny"

url =

http://www.w3.org/Graphics/SVG/Test/20030813/htmlframe/tinyndex.html/">

<test-group

name = "All-Tests"

url ="http://www.w3.org/Graphics/SVG/Test/20030813/htmlframe/">

<title>SVG Tiny tests</title>

</GroupInfo>

<test-case name="tiny-animate-elem-02-t.html" scenario="standard">

<description>Test 'additive' and 'accumulate'attributes./>

<spec-citation

spec="SVG"

section-number="19.2"
section-title="Animation Elements"

section-pointer="AnimationElements"/>

</test-case>

</test-group>

</collection>

<collection

name = "XHTML Print"

url = "http://www.w3.org/MarkUp/Test/xhtml-print/current/">

<test-group

name = "PrinterConformance"

url = "http://www.w3.org/MarkUp/Test/xhtml-print/current/tests/">

<title>XHTML Printer tests</title>

</GroupInfo>

<test-case name="2_3_1_BF-02.htm" scenario="standard">

<description>If a printer encounters an image in a format it does not

support, it SHALL render any alternate content

provided.</description>

<spec-citation

spec="XHTML-Print"

section-number="2.3.1"

section-title="Formatting/Rendering Rules"

section-pointer="S2.3"/>

</test-case>

</test-group>

</collection>

</test-suite-collections>

</test-suite>