EARL properties details

Starting from the f2f notes at
  http://www.w3.org/WAI/ER/2001/03/01-f2f-minutes.html

and detailing a bit (in custom data typing notation).

Let's discuss that and then we can come up with a notation.


The basic statement we want to express is that

 in some context
 some resource (e.g. a web page, a browser)
  evaluate-to
 some result (e.g. fails to meet a checkpoint, 
                   correctly implements css2 display property)

of course, we want to attach more information to the statement's
resource and result and to the context in which the test was run (when,
who). In the following, I'm using CAPITAL to note compound types
defined later on.


For the context:
  date (of the evaluation)
  who-claims: PERSON (running the test)
  platform: PLATFORM (on which the test was run)
 
For the resource being evaluated we mostly want to have
  unique id: URL
    e.g.  http://example.org/page#img[3]
          http://foo.com/svgplayer1.23
  nature of resource: ENUM [web-content, tool]
  optionally: 
    a version number
    a date (date the resource was last modified, released)
    a snapshot: URL (copy of resource at the time it was evaluated)

For the result, we want to have 
  value: ENUM [pass, fail, not-tested, not-applicable, cant-tell]
  confidence: ENUM [high, medium, low]  
  comments: string 
  test-case: TESTCASE 
  
then we have some compound subtypes:
  PERSON: regular stuff: NAME, EMAIL, URL
  PLATFORM: TOOL, OS, CCPP-SETTING
  TESTCASE: this one deserves more details, it could come from a
            separated test framework/harness formalism, used  
            to build the test itself
      suite: URL (w3.org/tr/wcag10, w3.org/style/css/test)
      individual id: URL, (checkpoint id, svg test suite assertion)
      mode: ENUM [manual, auto, heuristic]
      purpose: STRING
      expected result: STRING (to be compared with result comments above)
      operator-instructions: STRING
      reproducable step: STRING

PS: the repair information, if any, could be added on top of that, by
pointing to the statement as a whole and associating it with a repair
info block.


Three examples:


1) a web page fails a couple of wcag checkpoints
 
on 2001/03/14, danield@w3.org, using linux, netscape 4.75 says that:

 web-content at http://example.org/page#img[3] 
   last modified on 2001/01/01
   a snapshot of which was saved at http://store.com/example.org/page#img[3]
 fails
   with a high confidence 
   comments: alt text provided is just filename
 http://w3.org/tr/wcag#cp1.1, 
   according to the test suite provided www.cast.org/bobby
   manual check
   expected result: presence of alt text describing function

and that:

 web-content at http://example.org/page#name
   last modified on 2001/01/01
   a snapshot of which was saved at http://store.com/example.org/page#name
 fails
   with a medium confidence 
   comments: use a font instead of h3
 http://w3.org/tr/wcag#cp2.1, 
   according to the test suite provided www.cast.org/bobby
   manual check
   expected result: use proper structure instead of style
 

2) an svg viewer correctly implements circle filling, rect filling, etc

on 2001/03/14, joe@zzz.com says that

 the tool identified by http://foo.com/svgplayer1
   version 1.23
   released 2000/12/24
 passes 
   with a medium confidence 
   comments: circle looks ok, but there's an extra pixel there
 http://w3.org/tr/svg/ts/assertion1, 
  part of http://w3.org/svg/testsuite1.0 
  manual check
  purpose: testing circle filling
  expected result: circle should be all red, with a black border 

and that

 the same tool (pointer to resource described above)
 passes 
   with a high confidence 
   comments: rect looks just fine
 http://w3.org/tr/svg/ts/assertion2, 
  part of http://w3.org/svg/testsuite1.0 
  manual check
  purpose: testing rect filling
  expected result: rect should be all red, with a white border 


3) a web page has an HTML syntax error

on 2001/03/14 sean@w3.uk, says that:
 web-content at http://example.org/page
 fails
   comments: syntax error, line12, missing ul
 http://w3.org/html4/testassertion123
 according to http://validator.w3.org/html
 auto mode 
 purpose: checking html4 dtd content model
 expected result: see w3.org/tr/html4#ul

Received on Wednesday, 14 March 2001 10:49:06 UTC