- From: Detlev Fischer <fischer@dias.de>
- Date: Tue, 13 Sep 2011 09:35:02 +0200
- To: public-wai-evaltf@w3.org
Hi everyone,
I am getting quite concerned myself now, so please forgive me if I break
my promise to “stay shtum” to kick off a discussion about we mean when
we are using the term *requirement*.
1) Do we agree that we should not include requirements for
attributes which we have not shown to be *feasible*?
2) Do we agree that a requirement identifies a *necessary* attribute,
capability, characteristic, or quality of a system in order for
it to have value and utility to a user?
3) Do we further agree that requirements should be *verifiable*, i.e.
that tests can eventually prove that the thing built (our
methodology, in this case) meets the requirements we have specified?
If we agree on these three points (and I hope we do) then R03: Unique
interpretation and R04: Replicability should be first of all feasible;
they should be shown to be necessary (e.g., the methodology would have
reduced credibility without them); finally, they should also be
verifiable (e.g. replicability and uniqueness of interpretation can be
proven in independent tests of a real-world sites).
If you agree so far, were do we stand in this?
*Feasible:* I have not read a single statement on this mailing list so
far that has offered any evidence that replicability and unique
(unambiguous) interpretation are feasible - especially if the
methodology stays on a fairly generic level (i.e., if it does not
prescribe the tools to be used, a step-by-step procedure, and detailed
instructions for evaluating test results).
*Verifiable:* We do not know yet, we have not built anything so far
that we could use to carry out tests independently and then compare
results. So let’s move on to second-best, the various methods we
currently use. I would ask all of you to report on any tests that were
carried out by two independent testers and arrived at the same result.
No one has come forward and claimed it has happened, or even, that it
can be done.
*Necessary:* Some of you may believe that replicability and uniqueness
of interpretation are necessary because the methodology would be less
credible without them. But unless the methodology mandates that tests
are actually replicated, the claim of replicability is just a red
herring. I think that any claims that cannot be verified in practical
application seriously undermine the credibility of a methodology.
Detlev
--
---------------------------------------------------------------
Detlev Fischer PhD
DIAS GmbH - Daten, Informationssysteme und Analysen im Sozialen
Geschäftsführung: Thomas Lilienthal, Michael Zapp
Telefon: +49-40-43 18 75-25
Mobile: +49-157 7-170 73 84
Fax: +49-40-43 18 75-19
E-Mail: fischer@dias.de
Anschrift: Schulterblatt 36, D-20357 Hamburg
Amtsgericht Hamburg HRB 58 167
Geschäftsführer: Thomas Lilienthal, Michael Zapp
---------------------------------------------------------------
Received on Tuesday, 13 September 2011 07:35:27 UTC