action: ID/naming/referencing mechanism

Hi,

I had an action item [1] to send a summary of the 
ID/naming/referencing mechanism of the tests, and collect the open 
issues (or previously discussed issues that were not ideally resolved).


We currently use the following naming convention:
sca.b.c_lz_nnn, where the sections separated by underscores have the 
following meaning:
- sca.b.c: the number of the success criterion,
- lz: the level of the success criterion,
- nnn: the number of the test case.

This value is used in the id attribute of each TCDL file, the name of 
the metadata file (with the addition of the extension ".xml"), the 
name of the test files (with the addition of the extension ".html", 
unless there is more than one test file), and in the names of the 
supporting files (with the addition of appriopriate extensions). [2]


The downside of using a naming convention based on SC numbering is 
that the names go out of date if a success criterion is given a new number.
The alternative would be to use the IDs of success criteria instead 
of their numbers. [3]
The downside of using SC IDs for test sample IDs is that it would 
then require much more work to find the SC number (for the HTML view 
of the metadata, sorting, etcetera).[4]


Using the same naming convention for both the metadata and the test 
files makes it easy to see which files belong together. [14 November 
2006: RESOLUTION: test files used for more than one success criteria 
will be duplicated.] The downside is that we are required to 
duplicate test files if the same technique can be used to meet 
several success criteria. If the naming convention for the test files 
were based on the technique/failure IDs, it would be possible to 
reuse test files.
(Based on Vangelis' comment at [5].)


It is currently hard to guess to which "test suite" (combination of 
technology + WCAG draft, e.g. XHTML test suite for 17 May 2007 WD 
versus SVG test suite for 17 May 2007 WD) a test sample belongs 
without looking at the folder hierarchy. A test suite ID would be 
helpful here. [6]


Reflecting the expected result in the file names was rejected because 
it could be easily abused by tool developers. [7]


We can use this message as input for tomorrow's discussion, but don't 
hesitate to comment on the list.


Best regards,

Christophe


[1] <http://www.w3.org/2008/01/08-tsdtf-minutes.html#action04>
[2] <http://www.w3.org/WAI/ER/tests/usingTCDL#naming>
[3] <http://www.w3.org/2007/08/14-tsdtf-minutes#item01>
[4] 
<http://lists.w3.org/Archives/Public/public-wai-ert-tsdtf/2007Oct/0005.html>
[5] 
<http://lists.w3.org/Archives/Public/public-wai-ert-tsdtf/2006Sep/0043.html>
[6] 
<http://lists.w3.org/Archives/Public/public-wai-ert-tsdtf/2007Dec/0000.html>
[7] 
<http://lists.w3.org/Archives/Public/public-wai-ert-tsdtf/2006Dec/0038.html>


---
Please don't invite me to LinkedIn, Facebook, Quechup or other 
"social networks". You may have agreed to their "privacy policy", but 
I haven't.

-- 
Christophe Strobbe
K.U.Leuven - Dept. of Electrical Engineering - SCD
Research Group on Document Architectures
Kasteelpark Arenberg 10 bus 2442
B-3001 Leuven-Heverlee
BELGIUM
tel: +32 16 32 85 51
http://www.docarch.be/ 


Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm

Received on Monday, 18 February 2008 16:07:30 UTC