Re: Questions and proposals related to strawman schema from F2F

On Tue, 10 Sep 2002, Wendy A Chisholm wrote:

> If I remember correctly:
> * automatic - the test is cut and dry.  e.g. either the img element has an 
> alt attribute or it does not
> * heuristic - we can check for patterns but this might not be exhaustive or 
> it might generate a false positive.  e.g. the alt-text of this image seems 
> to be ok since it does not contain the following suspicious phrases: 
> ".gif", "insert alt-text here", "KB", "image", etc.

I don't like that distinction.  It's still automatic; it's just making
a diagnosis with less than 100% Confidence.

Now, if you were to define "heuristic" as a two-stage test starting
with the above but only generating a result after another test (eg
human confirms the diagnosis), it would make more sense.

> I haven't seen heuristic used too much and it might not be worth 
> keeping.

That's another option, and I won't be sorry to see the back of it.

> >I think that EARL should distinguish between groups of testcases and
> >groups of the criteria that the testcases test.
> 
> agreed.

We're wandering into test-definition-language territory here IMO.
ISTR posting on the subject a couple of months back.

> >The line between a testcase and a criterion is also a bit blurry - are
> >WCAG checkpoints considered to be testcases or criteria?
> 
> In WCAG 2.0 we have 3 levels:
> 1. Guidelines
> 2. Checkpoints
> 3. Success criteria
> 
> a success criterion is the child of one checkpoint and the great-grandchild 
> of one guideline.
> a checkpoint is the child of one guideline and at least one child (success 
> criterion).
> a guideline has siblings (other guidelines) and more than one child 
> (checkpoint).
> the root is the document "WCAG 2.0"

The language you're using there suggests that WCAG 2.0 will be expressed
in a machine-readable form.  I predict that if you do that, the world
will be plagued with inconsistencies between that and the accompanying
text (whatever that may be).  Though it might still be a worthwhile
exercise, and might leave it better-specified than WCAG 1.

> We could come up with the following.  I'm not sure how I feel about 
> this.  I didn't need to use Criterion, CriterionGroup, containsCriterion, 
> or tests.  I really like the idea of tests, tho...
> png of the graph is at: 
> http://lists.w3.org/Archives/Public/www-archive/2002Sep/0110.html
> 
> <rdf:RDF xmlns="http://www.w3.org/2001/03/earl/0.95#"
>      xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
>      xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">
>   <Assertor>
>    <name>Bob B. Bobbington</name>
>    <email rdf:resource="mailto:bob@example.org"/>
>    <asserts>
>      <Assertion>
>        <subject rdf:resource="http://www.w3.org"/>
>       <result rdf:resource="http://www.w3.org/2001/03/earl/0.95#passes"/>
>        <testcase rdf:resource="http://example.org/#WCAG20-Checkpoint-1"/>

Practical issues:
  * Confidence ?
  * Granularity (testcases <-> checkpoints isn't a 1-1 mapping in either
			direction, so the above is rather idealised)
  * Granularity of Subject: want to make assertions about things within page
  * Overloaded URI syntax.  An rdf:resource is a unique identifier, but
    the subject in the above is not unique (unless by coincidence).
    RDF explicitly _doesn't_ dereference URIs, but the above is only
    meaningful if you _do_ dereference the subject.

Ah, I see your testcase _is_ example.org .. OK, then you're presumably
pointing to a test description, which is good.  But then the
"WCAG20-Checkpoint-1" is a bit misleading.

>      </Assertion>
>    </asserts>
>   </Assertor>
> 
>   <TestGroup rdf:about="http://example.org/#WCAG20-Checkpoint-1">
>      <containsTestCase rdf:resource="http://example.org/#success-criterion-1"/>
>      <containsTestCase rdf:resource="http://example.org/#success-criterion-2"/>
>   </TestGroup>
>   </rdf:RDF>

Wouldn't that be better shoved off into a schema at http://example.org/ ?
(Cone to think of it, is the above even a valid reference otherwise)?


> If each version and os release of a UA is identifiable by a unique URI then 
> that should use a URI.  What is the URI for Opera v5 for Mac OS X? is it 
> http://www.opera.com/download/get.pl?platform=mac&force=5.0 ?  That seems 
> to be the only thing I could find on their site.

No, you can't use a vendor-URI as an identifier for its products, unless
the vendor itself publishes a classification incorporating them.  Opera
would be within their rights to use that URI for something totally
different, and using it in RDF has just confused things.

You could publish a schema with URIs for each identified UA, though.

> Should we include earl properties to uniquely identify test subjects (i.e. 
> normalization algorithm, hash algorithm, hash values for web content  and 
> os, version for tools)?

That seems to be a reference to my ramblings on robust metadata.

Above, I complain of your overloaded URI syntax.  Since the subject of
the assertion is in fact something you get by dereferencing the URI, it
is not well-specified by the above.  This is why you need additional
metadata, as discussed in my F2F talk, etc.

You will recollect that we have three basic tasks here:
  (1) Dealing with granular content - Generalised Pointers
      (e.g. ID, Line+Column, or XPointer into a normalisation).
  (2) Dealing with changing content (and whether an assertion stands
      or is invalidated after any particular change).
  (3) Dealing with Content Negotiation and multi-valued content.

Now, as far as I am concerned, EARL has little value if we can't use
subjects within a page, so we certainly need to be able to express
such a thing.  There are a few instances that should be pretty well
standard - notably those I have identified above - so perhaps we
could include EARL terms for them.  A further instance should be
a transformation that reduces the markup by factoring out that
which is irrelevant to an assertion: this could be coupled with
a checksum/hash on the reduction to detect change.

The EARL generated by Accessibility Valet uses a different approach:
all assertions reference a (slightly-reduced) normalisation that is
itself included in the report (as an alternative to referencing
instructions for reconstructing it).

Likewise, content-negotiation is important, so EARL should have terms
for recording an HTTP transaction.  In the case of a tool that always
uses a minimilastic GET (e.g. most of mine), this could be recorded
as properties of the tool (Assertor).  A tool that varies its HTTP
requests (eg cg-eye) might need to record it as properties of an
assertion.

Changing content is more iffy.  We can record the measures that
are defined in HTTP (date, length, MD5, etag), and these will indicate
whether a document has changed since an assertion.  But do we want
to be able to express anything more specific?  Jim and I already do
that in our tools, but we're dealing with particular issues that
probably don't generalise enough to merit EARL terms for them,
except as noted above.

-- 
Nick Kew

Received on Thursday, 19 September 2002 17:42:02 UTC