Re: What is a subject of a test?

I have been thinking a bit about how we model the assertions and locations  
of errors.

In Johanes' proposal below, there are essentially two assertions, both  
about the same subject and test case. It says that the subject fails the  
test twice.

I think we should adopt the Annotea approach of having a property that  
talks about the location within a test subject of a particular failure,  
but I don't thin that this should be part of the message, rather a  
property of the assertion itself. So the full statement would be

Assertion
   subject #foo
   location #pointer(somewhere)
   testcase #bar
   result Fail
   ...

Assertion
   subject #foo
   location #pointer(elsewhere)
   testcase #bar
   result Fail
   ...

If we collapse these together to give

Assertion
   subject #foo
   location #pointer(somewhere)
   location #pointer(elsewhere)
   testcase #bar
   result Fail
   ...

then we simplfy the syntax, but any processor needs to know that if there  
are multiple contexts/locations there are multiple errors. This gets  
complicated because in some cases a result will actually rely on multiple  
contexts for a single result. For example, an assertion that a client-side  
image map has redundant text links (which is a WCAG checkpoint) may want  
to refer to both the relevant places - the image map and the text links.

So I suggest we avoid the short syntax in the interests of easier  
processing. The alternative is to analyse the testcase, and determine  
whether it expects multiple contexts, or whether multiple contexts means  
multiple results.

This needs to be considered in the light of dynamically generated content  
where pieces are put together to form a whole. I'll address that  
seperately, because I think it is a pretty complex case, but not  
intractable.

what do people think?

Cheers

Chaals

On Thu, 31 Mar 2005 19:58:11 +1000, Johannes Koch  
<johannes.koch@fit.fraunhofer.de> wrote:

>
> Shadi Abou-Zahra wrote:
>
> Johannes? said
>>> Version 2 looks strange to me because the machine-processable  
>>> information (testCase, subject, validity) are the same and only the  
>>> (human-readable) message differs.
>>>
>>> Would it be necessary for the EARL spec to clearly define how to do
>>  that
>>> in order to make interchange of EARL reports possible?
>>   Yes, I think you are absolutely right that we need to clearly define
>> this situation in the specification.
>>  However, it seems to me that the description of location within the
>> message part of the assertion is not an ideal approach because it could
>> not be processed automatically anymore. Maybe we would need more than
>> one "location" attribute or even multiple "subject". We should discuss
>> this.
>
> Proposal:
> Assertion
>    |- testCase: HTML 4.01 Strict
>    |- subject: document URL
>    |- result
>       |- validity: fail
>       |- message
>       |  |- location
>       |  |  |- line1
>       |  |  |- column1
>       |  |- text: image lacks alt attribute
>       |- message
>          |- location
>          |  |- line2
>          |  |- column2
>          |- text: image lacks alt attribute

-- 
Charles McCathieNevile                      Fundacion Sidar
charles@sidar.org   +61 409 134 136    http://www.sidar.org

Received on Friday, 1 April 2005 13:27:57 UTC