- From: Detlev Fischer <fischer@dias.de>
- Date: Tue, 13 Sep 2011 09:35:02 +0200
- To: public-wai-evaltf@w3.org
Hi everyone, I am getting quite concerned myself now, so please forgive me if I break my promise to “stay shtum” to kick off a discussion about we mean when we are using the term *requirement*. 1) Do we agree that we should not include requirements for attributes which we have not shown to be *feasible*? 2) Do we agree that a requirement identifies a *necessary* attribute, capability, characteristic, or quality of a system in order for it to have value and utility to a user? 3) Do we further agree that requirements should be *verifiable*, i.e. that tests can eventually prove that the thing built (our methodology, in this case) meets the requirements we have specified? If we agree on these three points (and I hope we do) then R03: Unique interpretation and R04: Replicability should be first of all feasible; they should be shown to be necessary (e.g., the methodology would have reduced credibility without them); finally, they should also be verifiable (e.g. replicability and uniqueness of interpretation can be proven in independent tests of a real-world sites). If you agree so far, were do we stand in this? *Feasible:* I have not read a single statement on this mailing list so far that has offered any evidence that replicability and unique (unambiguous) interpretation are feasible - especially if the methodology stays on a fairly generic level (i.e., if it does not prescribe the tools to be used, a step-by-step procedure, and detailed instructions for evaluating test results). *Verifiable:* We do not know yet, we have not built anything so far that we could use to carry out tests independently and then compare results. So let’s move on to second-best, the various methods we currently use. I would ask all of you to report on any tests that were carried out by two independent testers and arrived at the same result. No one has come forward and claimed it has happened, or even, that it can be done. *Necessary:* Some of you may believe that replicability and uniqueness of interpretation are necessary because the methodology would be less credible without them. But unless the methodology mandates that tests are actually replicated, the claim of replicability is just a red herring. I think that any claims that cannot be verified in practical application seriously undermine the credibility of a methodology. Detlev -- --------------------------------------------------------------- Detlev Fischer PhD DIAS GmbH - Daten, Informationssysteme und Analysen im Sozialen Geschäftsführung: Thomas Lilienthal, Michael Zapp Telefon: +49-40-43 18 75-25 Mobile: +49-157 7-170 73 84 Fax: +49-40-43 18 75-19 E-Mail: fischer@dias.de Anschrift: Schulterblatt 36, D-20357 Hamburg Amtsgericht Hamburg HRB 58 167 Geschäftsführer: Thomas Lilienthal, Michael Zapp ---------------------------------------------------------------
Received on Tuesday, 13 September 2011 07:35:27 UTC