- From: Christophe Strobbe <christophe.strobbe@esat.kuleuven.be>
- Date: Tue, 30 Sep 2008 16:07:44 +0200
- To: public-wai-ert-tsdtf@w3.org
Hi, At 15:38 30/09/2008, Tim Boland wrote: >The W3C Mobile Web Activity has a mobile test harness [1], which may >be relevant to our discussions on this topic. The test harness, and especially the way a reviewer checks test results (i.e. you get output in red or green or compare something against a reference image), reminds me of an important difference between the WCAG 2.0 test suites I've worked on and most of the W3C test suites I've seen: most W3C test suites move along a single axis: the list of features of a technology; a test suite for WCAG 2 has two such axes: the accessibility requirements on the one hand, and the features of a technology (sometimes multiple technologies, like HTML and JavaScript) on the other.[1] This difference has influenced the design of the test samples, the metadata format, and the processes and tools around these things. [1] (That is also one of the reasons why reviewing test samples during the conference call isn't the quick and dirty process that Michael would like it to be; the other reason is that we need more concrete review recommendations that the group can quickly accept or reject.) Best regards, Christophe >Also there was a W3C QAWG Test FAQ document [2], which may contain >information to stimulate further discussions on this topic. > >The Matrix of W3C Specifications [3], although older, may contain >links to other development work which may be of interest. > >Thanks and best wishes >Tim Boland NIST > > > >[1]: http://www.w3.org/2007/03/mth/harness >[2]: http://www.w3.org/QA/WG/2005/01/test-faq >[3]: http://www.w3.org/QA/TheMatrix > >At 12:48 PM 9/29/2008 +0200, you wrote: > >>Hi, >> >>I have an (unrecorded) action item to write up draft requirements >>for a test sample management system. This has become more urgent >>because of the migration of the wiki. >> >>Here's a list of requirements: >>* an overview (which I will call "status list" from here on) of all >>the test samples; this overview should show: >> - test sample ID, >> - a link to the HTML version of the XML metadata, >> - links to the structure reviews (possibly with identification >> of the reviewers or reviewers), >> - links to the content reveiws (possibly with identification of >> the reviewers or reviewers), >> - the status of the test sample, >> - the submitter: organization (if any) and contact information >> (e-mail is sufficient), >> - links to issues that need to be fed back to WCAG WG. >> (This mirrors what we have at >> <http://www.w3.org/2006/tsdtf/TestSampleStatusList>.) >> It should also be possible to filter and sort this overview by >> at least ID and status. >> Nice to have in the status list: >> - filtering based on "features" used in the test samples, >> - filtering based on techniques and failures covered by the >> test samples. >> (The status list at <http://tinyurl.com/ynu7q4> has some sorting >> options but no filtering.) >> >>* an HTML view of the metadata of each test sample; >> see the links in the column "test sample ID" at >> <http://tinyurl.com/ynu7q4>. >> - This HTML view also contains links to both the metadata file >> and the HTML file or files. >> - For the sake of greater automation of the structure reviews, >> this view should also link >> to validation reports (or funcationality to generate them) for >> both XSD validation and >> ISO Schematron validation. >> - The section "relevant technique/failure" should also provide >> the title of the technique >> or failure instead of only the identifier. >> - Links to WBS questionnaires where the test sample was surveyed. >> >> >>Notes regarding the status list: >>- The XML metadata contains an element for status >>(\testCaseDescription\formalMetadata\status) >> but reviewers have only edited the status in the wiki. Ideally, >> status is tracked in the XML >> metadata and the status list pulls the status from the XML, so >> that the status list does not >> get out of sync with the XML metadata. >>- The status list in the wiki contains several tables: one for the >>current WCAG 2 draft, >> "the previous table" (for test samples that map to the May 2007 >> draft and test samples >> for the April 2006 draft which still needed to be updated to a >> more recent draft), and >> a table with "obsolete test samples" (test samples with the >> status "deprecated" because there >> was no matching technique or failure, and JSP test samples). >> So we would also need to be able to filter and sort the test >> samples by WCAG 2 draft. >> >> >> >>What we already have: >>- XSLT to generate HTML view of each test sample's XML metadata. >>- Ant script (using more XSLT) to generate test sample status list. >>- XSD for the XML metadata (TCDL 2.0; see RDDL file at >><http://bentoweb.org/refs/TCDL2.0/>), >> but is there a PHP implementation? >> (I don't see this at >> <http://phpxmlclasses.sourceforge.net/show_doc.php?class=class_xml_check.html> >>- ISO Schematron (see how-to doc at >><http://bentoweb.org/refs/TCDL2.0/tsdtf_schematron.html>), >> but there seems to be no PHP implementation. Maybe we can use >> the Amara XML Toolkit >> (<http://uche.ogbuji.net/tech/4suite/amara/>), an open-source >> collection of Python tools that >> has an ISO Schematron implementation. Amara 1.2a2 requires >> Python 2.4 and 4Suite XML 1.0 >> (<http://cheeseshop.python.org/pypi/4Suite-XML>), which is also >> open source. >> Alternatively, we can just rely on the XSTL implementation >> (discussed in my how-to doc) and >> PHP's XSLT support: >> <http://phpxmlclasses.sourceforge.net/show_doc.php?class=class_xslt.html>. >> If we choose the option of using the XSLT produced by the XSLT >> implementation >> (e.g. to use it in PHP code), we need to take into account that >> there is currently no stylesheet >> that produces an HTML report at >> <http://www.schematron.com/implementation.html> or at >> <http://www.schematron.com/validators.html>. >> >> >>This is just a first draft. Please comment on the list. >> >>We can then put the requirements on a page under >><http://www.w3.org/WAI/ER/tests/>. >> >> >>Best regards, >> >>Christophe >> >> >> >>-- >>Christophe Strobbe > -- Christophe Strobbe K.U.Leuven - Dept. of Electrical Engineering - SCD Research Group on Document Architectures Kasteelpark Arenberg 10 bus 2442 B-3001 Leuven-Heverlee BELGIUM tel: +32 16 32 85 51 http://www.docarch.be/ --- Please don't invite me to LinkedIn, Facebook, Quechup or other "social networks". You may have agreed to their "privacy policy", but I haven't. Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm
Received on Tuesday, 30 September 2008 14:08:47 UTC