Re: Markup Validator's test suite - using EARL?

Le Lundi, 29 sept 2003, à 22:13 America/Montreal, Olivier Thereaux a 
écrit :
> After a better look at how MUTAT [2] works, I started wondering 
> whether we could actually use EARL for the test descriptions.

do you mean EARL for the report of passing the Test Suite?
	or to describe the test?

We will have to consider something where the success of passing the 
test suite is not the fact of being valid.

Test A -> Has to be always invalid -> Success if invalid
								   -> Failed  if valid
Test B -> Has to be always valid   -> Success if invalid
								   -> Failed  if invalid

So I guess something like

===================
#   Mockup for a test file
      @prefix : <http://www.w3.org/QA/2003/09/validator/vocab#> .
      @prefix dc: <http://purl.org/dc/elements/1.1/> .

<> dc:title "W3C Markup Validator Test Suite";
    dc:description """
	List of test cases for a Validator Test suite.
	Describe here the organisation.
	""" .

<http://validator.w3.org/dev/tests/2003/09/test001.html>
	a :testCase;
	:ExpectedResult <http://validator.w3.org/valid>;
	dc:title "Simple XHTML 1.0 Strict Document";
	dc:description """
		XHTML 1.0 Strict document has given in the
		W3C XHTML 1.0 Specification.
	""";
	:reference <http://www.w3.org/TR/2000/REC-xhtml1-20000126/#strict> .

<http://validator.w3.org/dev/tests/2003/12/test069.html>
	a :testCase;
	:ExpectedResult <http://validator.w3.org/invalid>;
	dc:title "Invalid HTML Document";
	dc:description """
		HTML Document which must be invalid all the time
		no doctype
	""";
	:reference 
<http://lists.w3.org/Archives/Public/public-qa-dev/Dec2003/001.html> .

<http://validator.w3.org/dev/tests/2003/12/test070.html>
	a :testCase;
	:ExpectedResult <http://validator.w3.org/nocharset>;
	dc:title "Wrong Charset document";
	dc:description """
		The document has no charset. The validator must report it.
	""";
	:reference 
<http://lists.w3.org/Archives/Public/public-qa-dev/Dec2003/004.html> .
=============================


I put together some examples. It doesn't mean it has to be like that.

:validatorResult points to an URI
	valid
	invalid
	nocharset

Which is the expected result of the validator. So if the test says that 
the result should be "nocharset" and the validator validates the file 
without complaining about the charset. The test didn't pass.

Are the information about the test itself should be inside the test 
itself? At first we could say yes thinking in terms of HTML, but it 
will mean we will have to define of describing metadata for each kind 
of markup. It might be easier to put this information in the n3 file.

reference could be one or more reference to understand the context of 
the test itself. People who have submitted, references in the specs, 
discussions about the tests.


There is quite a huge number of tests in the HTML 4.01 Test Suite as 
well.

http://www.w3.org/MarkUp/Test/HTML401/

--
Karl Dubost - http://www.w3.org/People/karl/
W3C Conformance Manager
*** Be Strict To Be Cool ***