Re: Questions and proposals related to strawman schema from F2F from Nick Gibbins on 2002-09-26 (w3c-wai-er-ig@w3.org from September 2002)

From: Nick Gibbins <nmg@ecs.soton.ac.uk>
Date: Thu, 26 Sep 2002 16:40:40 +0100
To: Charles McCathieNevile <charles@w3.org>
Cc: Nick Kew <nick@webthing.com>, Wendy A Chisholm <wendy@w3.org>, <w3c-wai-er-ig@w3.org>
Message-ID: <15763.10872.33275.305083@localhost.localdomain>

Charles McCathieNevile <charles@w3.org> writes:
> On Thu, 19 Sep 2002, Nick Kew wrote:
>> On Tue, 10 Sep 2002, Wendy A Chisholm wrote:

>>> If I remember correctly:
>>> * automatic - the test is cut and dry.  e.g. either the img
>>>   element has an alt attribute or it does not
>>> * heuristic - we can check for patterns but this might not be
>>>   exhaustive or it might generate a false positive.  e.g. the
>>>   alt-text of this image seems to be ok since it does not contain
>>>   the following suspicious phrases: ".gif", "insert alt-text
>>>   here", "KB", "image", etc. 

>> I don't like that distinction.  It's still automatic; it's just
>> making a diagnosis with less than 100% Confidence.

>> Now, if you were to define "heuristic" as a two-stage test starting
>> with the above but only generating a result after another test (eg
>> human confirms the diagnosis), it would make more sense.

> Oh. I had a different understanding, which was that a heuristic
> result was one which was derived, rather than directly tested.

Okay, that makes sense.

If we go with that definition, then heuristic/not heuristic seems to
be orthogonal to manual/automatic; a machine can derive a result from
previous results as well as can a person.

In the example below, there are assertions about two different
testcases. The first testcase has been performed manually by a person,
and has found to be failing.

The second testcase has not been performed, but the deduction that it
too must be a fail has also been performed manually. The link to the
previous assertion is made using the earl:derivedFrom property.

Assertions which state that they are derived from the values of other
assertions are in some sense heuristic, going by Chaals'
definition. Conversely, an assertion which does not state that it is
derived from other assertions is not heuristic.

<?xml version='1.0' encoding='ISO-8859-1'?>
<!DOCTYPE rdf:RDF [
         <!ENTITY earl 'http://www.w3.org/WAI/ER/EARL/nmg-strawman#'>
         <!ENTITY rdf 'http://www.w3.org/1999/02/22-rdf-syntax-ns#'>
]>
<rdf:RDF xmlns:earl="&earl;"
         xmlns:rdf="&rdf;">
  <earl:Assertion rdf:about="http://example.org/#assertion-1">
    <earl:subject rdf:resource="http://example.org/#someID02495012470"/>
    <earl:result rdf:parseType="Resource">
      <earl:validity rdf:resource="&earl;fail"/>
      <earl:confidence rdf:resource="&earl;high"/>
      <earl:message>too many angle brackets in line 23</earl:message>
    </earl:result>
    <earl:mode rdf:resource="&earl;manual"/>
    <earl:testcase rdf:resource="http://example.org/#tc-1"/>
    <earl:assertedBy rdf:resource="http://example.org/#assertor12345678" />
  </earl:Assertion>

  <earl:Assertion rdf:about="http://example.org/#assertion-2">
    <earl:subject rdf:resource="http://example.org/#someID02495012470"/>
    <earl:result rdf:parseType="Resource">
      <earl:validity rdf:resource="&earl;fail"/>
      <earl:confidence rdf:resource="&earl;high"/>
      <earl:message>malformed element in line 23</earl:message>
    </earl:result>
    <earl:mode rdf:resource="&earl;manual"/>
    <earl:testcase rdf:resource="http://example.org/#tc-2"/>
    <earl:assertedBy rdf:resource="http://example.org/#assertor12345678" />
    <earl:derivedFrom rdf:resource="http://example.org/#assertion-1"/>
  </earl:Assertion>

  <earl:Person rdf:about="http://example.org/#assertor12345678">
    <earl:name>Bob B. Bobbington</earl:name>
    <earl:email rdf:resource="mailto:bob@example.org"/>
  </earl:Person>

  <earl:WebContent rdf:about="http://example.org/#someID02495012470">
   <earl:reprOf rdf:resource="http://www.w3.org/" />
   <earl:date>2001-05-17T23:07:35Z</earl:date>
  </earl:WebContent>

  <earl:Testcase rdf:about="http://example.org/#tc-1">
    <earl:testId rdf:resource="http://example.org/MyTestCaseThingy-1" />
  </earl:Testcase>

  <earl:Testcase rdf:about="http://example.org/#tc-2">
    <earl:testId rdf:resource="http://example.org/MyTestCaseThingy-2" />
  </earl:Testcase>
</rdf:RDF>

This could also give us a way to assert that a group of testcases have
passed or failed as a whole. In the example below, there are two
assertions about testcases, one which fails and one which passes (both
are evaluated manually, as before). The assertion that is made about
the group containing these two testcases indicates that it is derived
from the sense of its constituent assertions (eg. the group failure
value depends on the individual passes or fails), and that it was made
automatically.

<?xml version='1.0' encoding='ISO-8859-1'?>
<!DOCTYPE rdf:RDF [
         <!ENTITY earl 'http://www.w3.org/WAI/ER/EARL/nmg-strawman#'>
         <!ENTITY rdf 'http://www.w3.org/1999/02/22-rdf-syntax-ns#'>
]>
<rdf:RDF xmlns:earl="&earl;"
         xmlns:rdf="&rdf;">
  <earl:Assertion rdf:about="http://example.org/#assertion-1">
    <earl:subject rdf:resource="http://example.org/#someID02495012470"/>
    <earl:result rdf:parseType="Resource">
      <earl:validity rdf:resource="&earl;fail"/>
      <earl:confidence rdf:resource="&earl;high"/>
      <earl:message>too many angle brackets in line 23</earl:message>
    </earl:result>
    <earl:mode rdf:resource="&earl;manual"/>
    <earl:testcase rdf:resource="http://example.org/#tc-1"/>
    <earl:assertedBy rdf:resource="http://example.org/#assertor12345678" />
  </earl:Assertion>

  <earl:Assertion rdf:about="http://example.org/#assertion-2">
    <earl:subject rdf:resource="http://example.org/#someID02495012470"/>
    <earl:result rdf:parseType="Resource">
      <earl:validity rdf:resource="&earl;pass"/>
      <earl:confidence rdf:resource="&earl;high"/>
      <earl:message>headings are in big friendly letters</earl:message>
    </earl:result>
    <earl:mode rdf:resource="&earl;manual"/>
    <earl:testcase rdf:resource="http://example.org/#tc-2"/>
    <earl:assertedBy rdf:resource="http://example.org/#assertor12345678" />
  </earl:Assertion>

  <earl:Assertion rdf:about="http://example.org/#assertion-3">
    <earl:subject rdf:resource="http://example.org/#someID02495012470"/>
    <earl:result rdf:parseType="Resource">
      <earl:validity rdf:resource="&earl;fail"/>
      <earl:confidence rdf:resource="&earl;high"/>
    </earl:result>
    <earl:mode rdf:resource="&earl;automatic"/>
    <earl:testcase rdf:resource="http://example.org/#tg-1"/>
    <earl:derivedFrom rdf:resource="http://example.org/#assertion-1"/>
    <earl:derivedFrom rdf:resource="http://example.org/#assertion-2"/>
    <earl:assertedBy rdf:resource="http://example.org/#assertor12345678" />
  </earl:Assertion>


  <earl:Person rdf:about="http://example.org/#assertor12345678">
    <earl:name>Bob B. Bobbington</earl:name>
    <earl:email rdf:resource="mailto:bob@example.org"/>
  </earl:Person>

  <earl:WebContent rdf:about="http://example.org/#someID02495012470">
   <earl:reprOf rdf:resource="http://www.w3.org/" />
   <earl:date>2001-05-17T23:07:35Z</earl:date>
  </earl:WebContent>

  <earl:TestCase rdf:about="http://example.org/#tc-1">
    <earl:testId rdf:resource="http://example.org/MyTestCaseThingy-1" />
  </earl:TestCase>

  <earl:TestCase rdf:about="http://example.org/#tc-2">
    <earl:testId rdf:resource="http://example.org/MyTestCaseThingy-2" />
  </earl:TestCase>

  <earl:TestGroup rdf:about="http://example.org/#tg-1">
    <earl:containsTestCase rdf:resource="http://example.org/#tc-1"/>
    <earl:containsTestCase rdf:resource="http://example.org/#tc-2"/>
  </earl:TestGroup>
</rdf:RDF>

The examples above assume the extra RDFS fragment for the schema is as
follows:

<rdf:Property rdf:about="&earl;mode">
  <rdfs:domain rdf:resource="&earl;Assertion"/>
  <rdfs:range rdf:resource="&earl;Mode"/>
</rdf:Property>

<rdfs:Class rdf:about="&earl;Mode/>

<earl:Mode rdf:about="&earl;manual"/>
<earl:Mode rdf:about="&earl;automatic"/>

<rdf:Property rdf:about="&earl;derivedFrom">
  <rdfs:domain rdf:resource="&earl;Assertion"/>
  <rdfs:range rdf:resource="&earl;Assertion"/>
</rdf:Property>

[ The mode of an assertion could alternatively be represented by
  subclassing Assertion, which is arguably more elegant - it may be
  worth discussing this. ] 

Does this do what we need?
-- 
Nick Gibbins                                            nmg@ecs.soton.ac.uk
IAM (Intelligence, Agents, Multimedia)             tel: +44 (0) 23 80592831
Electronics and Computer Science                   fax: +44 (0) 23 80592865
University of Southampton

Received on Thursday, 26 September 2002 11:40:44 UTC