W3C home > Mailing lists > Public > public-wai-ert@w3.org > October 2005

Re: Proposal for earl:Location

From: Christophe Strobbe <christophe.strobbe@esat.kuleuven.be>
Date: Wed, 12 Oct 2005 14:31:26 +0200
Message-Id: <6.0.0.22.2.20051012135747.02e86778@mailserv.esat.kuleuven.be>
To: public-wai-ert@w3.org

Hi Shadi,

At 16:00 11/10/2005, Shadi Abou-Zahra wrote:
>Hi Christophe,
>
>Christophe Strobbe wrote:
>>>* What is the impact and severity of inconsistency regarding the location?
>>
>>One obvious consequence is that it becomes hard to compare and benchmark 
>>evaluation tools, but that is admittedly a very specific use case of EARL.
>
>One could still compare if tools produce the same results as well as the 
>number of instances of that result per test-subject (count the number of 
>location properties).

You're right.

>Is it really essential to compare the specific locations that actually 
>triggered the results as well or is the location more suitable for human 
>consumption?

I think it would make comparisons more reliable.
Imagine that there are six instances of a similar construct in a page:
- tool A says that instances 1, 2 and 4 pass, and that instances 3, 5 and 6 
fail CP 2.2 (colour contrast),
- tool B says that instances 1, 2 and 3 pass, and that instances 4, 5 and 6 
fail CP 2.2,
so the number of instances is the same in each report and only the 
locations can tell you if both tools find the same problems. But this case 
is only relevant to makers of test suites for benchmarks and can probably 
be ignored in evaluations of real websites.

Regards,

Christophe Strobbe



>Regards,
>  Shadi

-- 
Christophe Strobbe
K.U.Leuven - Departement of Electrical Engineering - Research Group on 
Document Architectures
Kasteelpark Arenberg 10 - 3001 Leuven-Heverlee - BELGIUM
tel: +32 16 32 85 51
http://www.docarch.be/ 


Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm
Received on Wednesday, 12 October 2005 12:32:51 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Tuesday, 8 January 2008 14:18:26 GMT