Re: Questions and proposals related to strawman schema from F2F

On Thu, 19 Sep 2002, Nick Kew wrote:
>
>On Tue, 10 Sep 2002, Wendy A Chisholm wrote:
>
>> If I remember correctly:
>> * automatic - the test is cut and dry.  e.g. either the img element has an
>> alt attribute or it does not
>> * heuristic - we can check for patterns but this might not be exhaustive or
>> it might generate a false positive.  e.g. the alt-text of this image seems
>> to be ok since it does not contain the following suspicious phrases:
>> ".gif", "insert alt-text here", "KB", "image", etc.
>
>I don't like that distinction.  It's still automatic; it's just making
>a diagnosis with less than 100% Confidence.
>
>Now, if you were to define "heuristic" as a two-stage test starting
>with the above but only generating a result after another test (eg
>human confirms the diagnosis), it would make more sense.
>
Oh. I had a different understanding, which was that a heuristic result was
one which was derived, rather than directly tested.

For example, in EART there are some number of applicable atomic tests to be
passed, where failing any one of those implies failing WCAG checkpoint 1.1.
In Bristol we discussed being able to say that these tests were collectively
tests which implied checkpoint 1.1.

So if you tested against something automatically (with a tool) you could
assert that. If you had some results that implied another result, (for
example a set of results for the collection of tests you have for checkpoint
1.1) then you would assert conformance, but heuristically.

If we know the original results used to derive the heuristic result we can
combine that with checking parts of a document only, rather than re-running
several tests taht are unnecesary when one does become necessary (see Nick's
final paragraph at the end of this mail).

>> >I think that EARL should distinguish between groups of testcases and
>> >groups of the criteria that the testcases test.
>>
>> agreed.
>
>We're wandering into test-definition-language territory here IMO.
>ISTR posting on the subject a couple of months back.

I disagree with the original proposition. The criteria are a testcase, and to
do the testing we make some other testcases available that might collectively
imply meeting the original testcase. If we don't distinguish we can gather
together results collected at an arbitrarily fine level of detail (from
"Abdullah says this is WCAG triple-A conformant" to "the text referred to by
this longdesc attribute doesn't match that for another longdesc attribute for
the same image"). If we distinguish different levels of grouping we need to
define at which level something exists, which gets messy.

>> >The line between a testcase and a criterion is also a bit blurry - are
>> >WCAG checkpoints considered to be testcases or criteria?

[snip]
>> that should use a URI.  What is the URI for Opera v5 for Mac OS X? is it
>> http://www.opera.com/download/get.pl?platform=mac&force=5.0 ?  That seems
>> to be the only thing I could find on their site.
>
>No, you can't use a vendor-URI as an identifier for its products, unless
>the vendor itself publishes a classification incorporating them.  Opera
>would be within their rights to use that URI for something totally
>different, and using it in RDF has just confused things.
>
>You could publish a schema with URIs for each identified UA, though.
>
>> Should we include earl properties to uniquely identify test subjects (i.e.
>> normalization algorithm, hash algorithm, hash values for web content  and
>> os, version for tools)?
>
>That seems to be a reference to my ramblings on robust metadata.
>
YHep. I think we need to use something like your hashes and pointers system
to identify content and give a sense of how robust the identification is.

[snip]
>Changing content is more iffy.  We can record the measures that
>are defined in HTTP (date, length, MD5, etag), and these will indicate
>whether a document has changed since an assertion.  But do we want
>to be able to express anything more specific?  Jim and I already do
>that in our tools, but we're dealing with particular issues that
>probably don't generalise enough to merit EARL terms for them,
>except as noted above.
>

Received on Sunday, 22 September 2002 18:07:44 UTC