- From: Vivienne CONWAY <v.conway@ecu.edu.au>
- Date: Tue, 13 Sep 2011 15:48:04 +0800
- To: Detlev Fischer <fischer@dias.de>, "public-wai-evaltf@w3.org" <public-wai-evaltf@w3.org>
Hi Detlev & TF'ers Detlev, as usual, you are making me think way too hard. Just kidding of course. Yes, of course I agree with your 3 top points. And Yes, I think I am probably being overly optimistic thinking that if it's designed properly everyone will get the same outcome for the same site. If I'm perfectly honest, I may not even get the same answer twice for the same site. I'm going to have to bow to your superior reasoning on this one. At the moment, we have no idea (at least till we build something) whether it is replicable. Perhaps we need to propose a test we all carry out on a certain page and using our own techniques to test it against WCAG 2.0 AA amd see the answers? This might give us an idea of how replicable our methods are. Regards Vivienne L. Conway ________________________________________ From: public-wai-evaltf-request@w3.org [public-wai-evaltf-request@w3.org] On Behalf Of Detlev Fischer [fischer@dias.de] Sent: Tuesday, 13 September 2011 3:35 PM To: public-wai-evaltf@w3.org Subject: Do we share an understanding of "requirement"? Hi everyone, I am getting quite concerned myself now, so please forgive me if I break my promise to “stay shtum” to kick off a discussion about we mean when we are using the term *requirement*. 1) Do we agree that we should not include requirements for attributes which we have not shown to be *feasible*? 2) Do we agree that a requirement identifies a *necessary* attribute, capability, characteristic, or quality of a system in order for it to have value and utility to a user? 3) Do we further agree that requirements should be *verifiable*, i.e. that tests can eventually prove that the thing built (our methodology, in this case) meets the requirements we have specified? If we agree on these three points (and I hope we do) then R03: Unique interpretation and R04: Replicability should be first of all feasible; they should be shown to be necessary (e.g., the methodology would have reduced credibility without them); finally, they should also be verifiable (e.g. replicability and uniqueness of interpretation can be proven in independent tests of a real-world sites). If you agree so far, were do we stand in this? *Feasible:* I have not read a single statement on this mailing list so far that has offered any evidence that replicability and unique (unambiguous) interpretation are feasible - especially if the methodology stays on a fairly generic level (i.e., if it does not prescribe the tools to be used, a step-by-step procedure, and detailed instructions for evaluating test results). *Verifiable:* We do not know yet, we have not built anything so far that we could use to carry out tests independently and then compare results. So let’s move on to second-best, the various methods we currently use. I would ask all of you to report on any tests that were carried out by two independent testers and arrived at the same result. No one has come forward and claimed it has happened, or even, that it can be done. *Necessary:* Some of you may believe that replicability and uniqueness of interpretation are necessary because the methodology would be less credible without them. But unless the methodology mandates that tests are actually replicated, the claim of replicability is just a red herring. I think that any claims that cannot be verified in practical application seriously undermine the credibility of a methodology. Detlev -- --------------------------------------------------------------- Detlev Fischer PhD DIAS GmbH - Daten, Informationssysteme und Analysen im Sozialen Geschäftsführung: Thomas Lilienthal, Michael Zapp Telefon: +49-40-43 18 75-25 Mobile: +49-157 7-170 73 84 Fax: +49-40-43 18 75-19 E-Mail: fischer@dias.de Anschrift: Schulterblatt 36, D-20357 Hamburg Amtsgericht Hamburg HRB 58 167 Geschäftsführer: Thomas Lilienthal, Michael Zapp --------------------------------------------------------------- This e-mail is confidential. If you are not the intended recipient you must not disclose or use the information contained within. If you have received it in error please return it to the sender via reply e-mail and delete any record of it from your system. The information contained within is not the opinion of Edith Cowan University in general and the University accepts no liability for the accuracy of the information provided. CRICOS IPC 00279B
Received on Tuesday, 13 September 2011 07:48:46 UTC