Use cases of EARL

This discussion has started on an internal mailing list. The QA WG is 
in the process of creating an implementation report for Specification 
Guidelines [1]. Then EARL seemed to be a sensible choice for it.

So I tried, and I came to the conclusion that things were missing in 
EARL 1.0 [2] to make it usable in this particular usage of reporting 
something.

The discussion is an excerpt of emails exhanged with Chaals. Please 
feel free to jump in the discussion.

The most important part is in the bottom of the mail, where some 
possible use cases are given.


[1] http://www.w3.org/TR/2004/WD-qaframe-spec-20041122/
[2] http://www.w3.org/TR/2002/WD-EARL10-20021206/

=======================================
Date: Sun, 19 Dec 2004 08:39:21 -0500 (EST)
From: Charles McCathieNevile <charles@w3.org>

Cool...

On Thu, 16 Dec 2004, Karl Dubost wrote:

> I will still use the same case because I think it's more practical. I
> think I start to have something which is really meaningful.

Agree. Can we also discuss this on some public list?

> I have defined an n3 and rdf file for specgl-ics
>
> It's a compilation of the test cases for specGL
> http://www.w3.org/QA/WG/2004/12/specgl/0.1.rdf
> http://www.w3.org/QA/WG/2004/12/specgl/0.1.n3

Major comment: I wouldn't use earl:note for giving the text - I would 
use
rdfs:label xml:lang="en"

This makes  it easier to build some system that wants to make its user
intereface in multiple languages. (What we do with Hera - although we
actually don't yet use RDF for the back-end because we don't know enough
about working with RAP, that's the plan).

> I have also started a report for Xinclude, only a few cases inside for
> now (for testing purpose)
>
> http://www.w3.org/QA/WG/2004/12/xinclude-report.n3
> http://www.w3.org/QA/WG/2004/12/xinclude-report.rdf
>
> A few issues. I didn't find in the EARL vocabulary how to say that a
> Test Case is mandatory or that is not normative.
> 	In specGL:
> 		Requirement = Normative
> 		Good Practice = Informative

True, this is an outstanding issue. In my collected random stuff at
http://www.w3.org/2001/sw/Europe/200407/earl there is an approach to 
doing
this in OWL - you say that conforming to some set of requirements (e.g. 
all
the normaticve things in SpecGL) implies conformance to some other
requirement (e.g. SpecGL itself). Shadi and I talked about this a lot 
too - I
think he has some code that handles it. (I ran into the fact that I 
would
have to write a bunch of cwm rules to handle basic OWL, and got as far 
as
noting that copying the rules Jena uses would make it easier before I 
ran out
of steam).

> The TestSubject class has also only a few classes of products.
> 	Tool
> 	UserAgent
> 	WebContent
>
> I wonder if a technical specification is WebContent or another kind of
> of thing.

IMHO another kind of thing.

> And I don't think that the difference is really between Tool
> and UserAgent. This class has really to be refined, you could redefined
> Sub-Classes after.

Yep.

> 	Person: someone who will be evaluated against tests.
> 		- Computing engineer
> 		- Webmaster
> 		- Administrative Assistant
> 		- Plumber
> 	Product: Something which has to be checked if it respect criterias
> 		- Software: A piece of code whatever the nature
> 			- Authoring Tool
> 			- User Agent
> 			- Bot
> 			- etc.
> 		- Content: writing on the web or not that should be evaluated
> 			- Book
> 			- TV Program
> 			- Speech
> 			- etc.
>
> I know my categories seem to be broad.If we want to make it a
> successful language and shows that it can be used for many things. We
> may want to foster implementations by creating a multi-purpose
> reporting language.

Well, it is multipurpose. It is just not yet very clear how much it 
should
include about the thing that you are testing (should it include 
vocabulary
for describing a Person, or would FOAF do...?)

> For example it could be used
> 	- by a teacher to report the scores of a student,
> 	- for a professional test,
> 	- to test the quality of a product
>
> the vocabulary of tests being outside of EARL and free to be designed
> by the people who have specific criterias.

Yep. This is something I have in mind as a general principle (see the
discussions in 2002 :-) that I think has broader acceptance than just 
me.

Feel free to copy this message anywhere you like in the interests of a
broader discussion.


-- 
Karl Dubost - http://www.w3.org/People/karl/
W3C Conformance Manager
*** Be Strict To Be Cool ***