RE: some initial questions from the previous thread

Some other possible questions: Does an evaluation methodology necessarily involve a user carrying out a predefined task involving websites?  What exactly are we evaluating against (how do any business rules, mission definition/completion requirements, etc. influence an evaluation - "context" of evaluation)?  Do we need any formalisms or ontologies to adequately express any evaluation parameters/context information?

Thanks and best wishes
Tim Boland

PS - apologies in advance if these questions have already been answered.. 


-----Original Message-----
From: public-wai-evaltf-request@w3.org [mailto:public-wai-evaltf-request@w3.org] On Behalf Of Shadi Abou-Zahra
Sent: Monday, August 22, 2011 7:35 AM
To: Eval TF
Subject: some initial questions from the previous thread

Dear Eval TF,

 From the recent thread on the construction of WCAG 2.0 Techniques, here 
are some questions to think about:

* Is the "evaluation methodology" expected to be carried out by one 
person or by a group of more than one persons?

* What is the expected level of expertise (in accessibility, in web 
technologies etc) of persons carrying out an evaluation?

* Is the involvement of people with disabilities a necessary part of 
carrying out an evaluation versus an improvement of the quality?

* Are the individual test results binary (ie pass/fail) or a score 
(discrete value, ratio, etc)?

* How are these test results aggregated into an overall score (plain 
count, weighted count, heuristics, etc)?

* Is it useful to have a "confidence score" for the tests (for example 
depending on the degree of subjectivity or "difficulty")?

* Is it useful to have a "confidence score" for the aggregated result 
(depending on how the evaluation is carried out)?


Feel free to chime in if you have particular thoughts on any of these.

Best,
   Shadi

-- 
Shadi Abou-Zahra - http://www.w3.org/People/shadi/
Activity Lead, W3C/WAI International Program Office
Evaluation and Repair Tools Working Group (ERT WG)
Research and Development Working Group (RDWG)

Received on Wednesday, 24 August 2011 13:35:05 UTC