Re: requirements for test sample management system

The W3C Mobile Web Activity has a mobile test harness [1], which may be 
relevant to our discussions on this topic.

Also there was a W3C QAWG Test FAQ document [2], which may contain 
information to stimulate further discussions on this topic.

The Matrix of W3C Specifications [3], although older, may contain links to 
other development work which may be of interest.

Thanks and best wishes
Tim Boland NIST



[1]: http://www.w3.org/2007/03/mth/harness
[2]: http://www.w3.org/QA/WG/2005/01/test-faq
[3]: http://www.w3.org/QA/TheMatrix

At 12:48 PM 9/29/2008 +0200, you wrote:

>Hi,
>
>I have an (unrecorded) action item to write up draft requirements for a 
>test sample management system. This has become more urgent because of the 
>migration of the wiki.
>
>Here's a list of requirements:
>* an overview (which I will call "status list" from here on) of all the 
>test samples; this overview should show:
>   - test sample ID,
>   - a link to the HTML version of the XML metadata,
>   - links to the structure reviews (possibly with identification of the 
> reviewers or reviewers),
>   - links to the content reveiws (possibly with identification of the 
> reviewers or reviewers),
>   - the status of the test sample,
>   - the submitter: organization (if any) and contact information (e-mail 
> is sufficient),
>   - links to issues that need to be fed back to WCAG WG.
>   (This mirrors what we have at 
> <http://www.w3.org/2006/tsdtf/TestSampleStatusList>.)
>   It should also be possible to filter and sort this overview by at least 
> ID and status.
>   Nice to have in the status list:
>     - filtering based on "features" used in the test samples,
>     - filtering based on techniques and failures covered by the test samples.
>   (The status list at <http://tinyurl.com/ynu7q4> has some sorting 
> options but no filtering.)
>
>* an HTML view of the metadata of each test sample;
>   see the links in the column "test sample ID" at 
> <http://tinyurl.com/ynu7q4>.
>   - This HTML view also contains links to both the metadata file and the 
> HTML file or files.
>   - For the sake of greater automation of the structure reviews, this 
> view should also link
>     to validation reports (or funcationality to generate them) for both 
> XSD validation and
>     ISO Schematron validation.
>   - The section "relevant technique/failure" should also provide the 
> title of the technique
>     or failure instead of only the identifier.
>   - Links to WBS questionnaires where the test sample was surveyed.
>
>
>Notes regarding the status list:
>- The XML metadata contains an element for status 
>(\testCaseDescription\formalMetadata\status)
>   but reviewers have only edited the status in the wiki. Ideally, status 
> is tracked in the XML
>   metadata and the status list pulls the status from the XML, so that the 
> status list does not
>   get out of sync with the XML metadata.
>- The status list in the wiki contains several tables: one for the current 
>WCAG 2 draft,
>   "the previous table" (for test samples that map to the May 2007 draft 
> and test samples
>   for the April 2006 draft which still needed to be updated to a more 
> recent draft), and
>   a table with "obsolete test samples" (test samples with the status 
> "deprecated" because there
>   was no matching technique or failure, and JSP test samples).
>   So we would also need to be able to filter and sort the test samples by 
> WCAG 2 draft.
>
>
>
>What we already have:
>- XSLT to generate HTML view of each test sample's XML metadata.
>- Ant script (using more XSLT) to generate test sample status list.
>- XSD for the XML metadata (TCDL 2.0; see RDDL file at 
><http://bentoweb.org/refs/TCDL2.0/>),
>   but is there a PHP implementation?
>   (I don't see this at 
> <http://phpxmlclasses.sourceforge.net/show_doc.php?class=class_xml_check.html>
>- ISO Schematron (see how-to doc at 
><http://bentoweb.org/refs/TCDL2.0/tsdtf_schematron.html>),
>   but there seems to be no PHP implementation. Maybe we can use the Amara 
> XML Toolkit
>   (<http://uche.ogbuji.net/tech/4suite/amara/>), an open-source 
> collection of Python tools that
>   has an ISO Schematron implementation. Amara 1.2a2 requires Python 2.4 
> and 4Suite XML 1.0
>   (<http://cheeseshop.python.org/pypi/4Suite-XML>), which is also open 
> source.
>   Alternatively, we can just rely on the XSTL implementation (discussed 
> in my how-to doc) and
>   PHP's XSLT support: 
> <http://phpxmlclasses.sourceforge.net/show_doc.php?class=class_xslt.html>.
>   If we choose the option of using the XSLT produced by the XSLT 
> implementation
>   (e.g. to use it in PHP code), we need to take into account that there 
> is currently no stylesheet
>   that produces an HTML report at 
> <http://www.schematron.com/implementation.html> or at
>   <http://www.schematron.com/validators.html>.
>
>
>This is just a first draft. Please comment on the list.
>
>We can then put the requirements on a page under 
><http://www.w3.org/WAI/ER/tests/>.
>
>
>Best regards,
>
>Christophe
>
>
>
>--
>Christophe Strobbe
>K.U.Leuven - Dept. of Electrical Engineering - SCD
>Research Group on Document Architectures
>Kasteelpark Arenberg 10 bus 2442
>B-3001 Leuven-Heverlee
>BELGIUM
>tel: +32 16 32 85 51
>http://www.docarch.be/
>---
>Please don't invite me to LinkedIn, Facebook, Quechup or other "social 
>networks". You may have agreed to their "privacy policy", but I haven't.
>
>
>Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm
>
>

Received on Tuesday, 30 September 2008 13:39:28 UTC