Re: earl

On Thu, Sep 14, 2006 at 06:33:24PM -0400, Kendall Clark wrote:
> 
> Folks,
> 
> I promised the SWCG that I'd try to find someone on DAWG to review  
> the EARL schema:
> 
> http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20060905g

== Example 3 ==
<http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20060905#example-3>
  <earl:Assertion rdf:ID="#assertion">
should be "assertion" (kudos to emacs22 nxml-mode).

There are a few other mods I had to make. See the attached xslt for
how I extracted the examples, and the attached rdf file for what my
final edit was. (Ignore the html mods in the parseType="Literal" --
they were just so I keep the text brief.)


== 2.1 Assertion ==
<http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20060905#example-3>
It may be worth associating the mode with the assertor, eg

<earl:Assertion rdf:ID="#assertion">
  <earl:assertedBy rdf:resource="#assertor" />
  <earl:subject rdf:resource="#subject" />
  <earl:test rdf:resource="#testcase" />
  <earl:result rdf:resource="#result" />
  <!-- removed mode -->
  <!-- earl:mode rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#mixed"/ -->
</earl:Assertion>

<earl:compoundAssertor rdf:ID="assertor">
  <dc:title xml:lang="en">Bob using Two Cool Tools</dc:title>
  <dc:description xml:lang="en">Bob doing semi-automated testing</dc:description>
  <earl:mainAssertor rdf:resource="#bob"/>
  <earl:helpAssertor rdf:resource="#tool"/>
  <earl:helpAssertor rdf:resource="#validator"/>
  <earl:mode rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#mixed"/> 
</earl:compoundAssertor>

<foaf:Person rdf:ID="bob">
  <foaf:name>Bob B. Bobbington</foaf:name>
  <foaf:mbox rdf:resource="mailto:bob@example.org"/>
  <foaf:mbox_sha1sum>1a9daad476f0158b81bc66b7b27b438b4b4c19c0</foaf:mbox_sha1sum>
  <earl:mode rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#semiauto"/> 
</foaf:Person>

<earl:Software rdf:ID="tool">
  <dc:title xml:lang="en">Cool Tool</dc:title>
  <dc:description xml:lang="en">My favorite tool!</dc:description>
  <foaf:homepage>http://example.org/tools/#cool</foaf:homepage>
  <dct:hasVersion>1.0.3</dct:hasVersion>
  <earl:mode rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#semiauto"/> 
</earl:Software>

<earl:Software rdf:ID="validator">
  <dc:title xml:lang="en">Cool Tool</dc:title>
  <dc:description xml:lang="en">My favorite tool!</dc:description>
  <foaf:homepage>http://example.org/tools/#cool</foaf:homepage>
  <dct:hasVersion>1.0.3</dct:hasVersion>
  <earl:mode rdf:resource="http://www.w3.org/WAI/ER/EARL/nmg-strawman#automatic"/> 
</earl:Software>

This might be worth 10 minutes thought, but feel free to dismiss as it
may not have much payback.


== Confidence Level ==
<http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20060905#example-3>
[[
Confidence in the result by an Assertor. This may be used where a tool
wants to assert that it has different levels of confidence about two
possible results (for example low confidence that a test has been
passed, but also high confidence that it is not applicable). The
values of the confidence level should be defined either as RDF values,
or using a datatype, in order to allow others to work interoperably
with them.
]]

I don't see much value to a standardized predicate with an
application-specific range. I would guess you serve more needs if you
specify a fairly arbitrary, human-friendly range, say, 0-100, and let
the apps that need their own range use their own predicate in addition
to the standard one:
<earl:result rdf:about="#result">
  <earl:confidence rdf:datatype="http://www.w3.org/2001/XMLSchema#int">87</earl:confidence>
  <my:confidence rdf:datatype="http://www.w3.org/2001/XMLSchema#int">65535</earl:confidence>
</earl:result>

what is an "RDF values"? (There's only one instance of the term in the
spec.)


== 2.6.3. Instance Location ==
<http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20060905#instancelocation?
[[
An Instance Location is a an area within the Test Subject that
triggered or primarily influenced the Test Result. These areas could
be described using a variety of different types of pointers that re
grouped by Pointer Collection classes. Each instance of such a
collection must describe exactly one occurence of an area.
]]

Readers could use more guidance on "area". Perhaps you get the most
bang for the buck by saying it's logical section of a document
identifiable by at least one of these addressing schemes.

Hard case: identifying the location of a cognitive barrier introduced
by the intereaction of a flowed paragraph and the picture it's flowing
around. Worth it? I dunno. 10 minutes thought by the experts.

[[
earl:htmlPointer
    An HTML Pointer string expression
]]

Is that a URL reference (#label thingy)?

I don't think you need both earl:charSnippet and earl:byteSnippet.
Rather, I think that these two identify quoted-printable and base64
RFC2045 § 6 Content-Transfer-Encodings. Please consider merging them
into a earl:snippit property with an object with an
ear:encodingMechanism with values like #quoted-printable, #eightBit,
#base64.

_:result earl:instance [
  earl:snippet [
    a earl:SnippetPointer ;
    earl:encodingMechanism smtp:base64 ;
    earl:content "A32F48ED3F2" ] ;
  earl:snippet [
    a earl:SnippetPointer ;
    earl:encodingMechanism smtp:quoted-printable ;
    earl:content "Hi mom!" ] ;
] .


== 2.8. Web Content ==
<http://www.w3.org/WAI/ER/EARL10/WD-EARL10-Schema-20060905#webcontent>
[[
earl:httpResponse
    HTTP response (except the actual payload of the date) sent by the
    server to the client
]]

s/date/data/
rfc2616 says that a response is made of:
       Response      = Status-Line               ; Section 6.1
                       *(( general-header        ; Section 4.5
                        | response-header        ; Section 6.2
                        | entity-header ) CRLF)  ; Section 7.1
                       CRLF
                       [ message-body ]          ; Section 7.2
Infortunately, there's no convenient "headers" production, but you can
s/payload/message-body/ in order to tie back to the RFC terminology.
(I assume you you want the Status-Line included.)

[[
Note: an instance of the Web Content class may include several
request/response sequences to document the content and language
negotiation that took place before the content was finally sent.
]]
Ooo, attacking the non-REST interactions. That *is* ambitious.
Is that a requirement? You can never be sure you've captured enough in
your log for someone to duplicated the experience, even if they can go
back in time. If you want to give folks a rough tool that they can use
to captures *some* of these, I'd get rid of the dc:format (or tie it
redundantly to the indivitual requests and responses), and make the 

[ a earl:TestSubject ;
  earl:messageSequence ( [ Server "example.org" ; 
			   port "80" ; 
			   request "GET /loginPage..." ; 
			   response "200..." ]
			 [ server "example.org" ; 
			   port "80" ; 
			   request "GET /loginPage...?name=ericP..." ; 
			   response "200...cookie..." ]
			 [ server "example.org" ; 
			   port "80" ; 
			   request "GET /service...cookie..." ; 
			   response "200..." ] ) ] .
for documents that can't be identified by URI, and 
[ a earl:TestSubject ;
  earl:uri <http://example.org/service> ]
for documents that can be.

2.8 also could use an example.


== Query-ability ==

Looking at this from the SPARQL perspective ,this is all trivially
query-able. For example, from the attached data extracted from the
examples wiht the attached earl-ex.xslt , I queried

[[
PREFIX earl: <http://www.w3.org/WAI/ER/EARL/nmg-strawman#>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
SELECT ?name ?title ?val ?conf
 WHERE { ?assert earl:subject [ dc:title ?title ] ; 
		 earl:test [ dc:identifier "http://example.org/tests/html/#282" ] ; 
		 earl:result [ earl:validity ?val ; earl:confidence ?conf ] ; 
		 earl:assertedBy [ earl:mainAssertor [ foaf:name ?name ] ] }
]]

and got back:

+-------------------+--------------+---------+----+
|               name|         title|      val|conf|
|-------------------|--------------|---------|----|
|"Bob B. Bobbington"|"Login Applet"|earl:fail|  87|
+-------------------+--------------+---------+----+


== General Comments ==

I was impressed by this spec, though you tackle some difficult stuff
near the end. Have fun!


== Mapping to DAWG tests ==
should be done soon
-- 
-eric

home-office: +1.617.395.1213 (usually 900-2300 CET)
	    +33.1.45.35.62.14
cell:       +33.6.73.84.87.26

(eric@w3.org)
Feel free to forward this message to any list for any purpose other than
email address distribution.