RE: Do we share an understanding of "requirement"?

Hi Detlev & TF'ers

Detlev, as usual, you are making me think way too hard.  Just kidding of course.
Yes, of course I agree with your 3 top points.  And Yes, I think I am probably being overly optimistic thinking that if it's designed properly everyone will get the same outcome for the same site.  If I'm perfectly honest, I may not even get the same answer twice for the same site.  I'm going to have to bow to your superior reasoning on this one.  At the moment, we have no idea (at least till we build something) whether it is replicable.  Perhaps we need to propose a test we all carry out on a certain page and using our own techniques to test it against WCAG 2.0 AA amd see the answers?  This might give us an idea of how replicable our methods are.


Regards

Vivienne L. Conway
________________________________________
From: public-wai-evaltf-request@w3.org [public-wai-evaltf-request@w3.org] On Behalf Of Detlev Fischer [fischer@dias.de]
Sent: Tuesday, 13 September 2011 3:35 PM
To: public-wai-evaltf@w3.org
Subject: Do we share an understanding of "requirement"?

Hi everyone,

I am getting quite concerned myself now, so please forgive me if I break
my promise to “stay shtum” to kick off a discussion about we mean when
we are using the term *requirement*.

1) Do we agree that we should not include requirements for
    attributes which we have not shown to be *feasible*?

2) Do we agree that a requirement identifies a *necessary* attribute,
    capability, characteristic, or quality of a system in order for
    it to have value and utility to a user?

3) Do we further agree that requirements should be *verifiable*, i.e.
    that tests can eventually prove that the thing built (our
    methodology, in this case) meets the requirements we have specified?

If we agree on these three points (and I hope we do) then R03: Unique
interpretation and R04: Replicability should be first of all feasible;
they should be shown to be necessary (e.g., the methodology would have
reduced credibility without them); finally, they should also be
verifiable (e.g. replicability and uniqueness of interpretation can be
proven in independent tests of a real-world sites).

If you agree so far, were do we stand in this?

*Feasible:* I have not read a single statement on this mailing list so
far that has offered any evidence that replicability and unique
(unambiguous) interpretation are feasible  -  especially if the
methodology stays on a fairly generic level (i.e., if it does not
prescribe the tools to be used, a step-by-step procedure, and detailed
instructions for evaluating test results).

*Verifiable:*  We do not know yet, we have not built anything so far
that we could use to carry out tests independently and then compare
results. So let’s move on to second-best, the various methods we
currently use. I would ask all of you to report on any tests that were
carried out by two independent testers and arrived at the same result.
No one has come forward and claimed it has happened, or even, that it
can be done.

*Necessary:*  Some of you may believe that replicability and uniqueness
of interpretation are necessary because the methodology would be less
credible without them. But unless the methodology mandates that tests
are actually replicated, the claim of replicability is just a red
herring. I think that any claims that cannot be verified in practical
application seriously undermine the credibility of a methodology.

Detlev

--
---------------------------------------------------------------
Detlev Fischer PhD
DIAS GmbH - Daten, Informationssysteme und Analysen im Sozialen
Geschäftsführung: Thomas Lilienthal, Michael Zapp

Telefon: +49-40-43 18 75-25
Mobile: +49-157 7-170 73 84
Fax: +49-40-43 18 75-19
E-Mail: fischer@dias.de

Anschrift: Schulterblatt 36, D-20357 Hamburg
Amtsgericht Hamburg HRB 58 167
Geschäftsführer: Thomas Lilienthal, Michael Zapp
---------------------------------------------------------------

This e-mail is confidential. If you are not the intended recipient you must not disclose or use the information contained within. If you have received it in error please return it to the sender via reply e-mail and delete any record of it from your system. The information contained within is not the opinion of Edith Cowan University in general and the University accepts no liability for the accuracy of the information provided.

CRICOS IPC 00279B

Received on Tuesday, 13 September 2011 07:48:46 UTC