Re: Do we share an understanding of "requirement"?

Hi all,

This is a useful discussion that both Eric and I have been watching 
closely; I do not suggest anyone be shtum (quite).

I also do not think it is a matter of dropping or keeping R03 and R04 
but finding a wording that better explains what we essentially mean. It 
seems that there is general agreement that we want less ambiguity and a 
higher degree of replicability, but that there are no absolutes in this 
endeavor. I hope we can find a wording along these lines.

Best,
   Shadi


On 13.9.2011 09:35, Detlev Fischer wrote:
> Hi everyone,
>
> I am getting quite concerned myself now, so please forgive me if I break
> my promise to “stay shtum” to kick off a discussion about we mean when
> we are using the term *requirement*.
>
> 1) Do we agree that we should not include requirements for
> attributes which we have not shown to be *feasible*?
>
> 2) Do we agree that a requirement identifies a *necessary* attribute,
> capability, characteristic, or quality of a system in order for
> it to have value and utility to a user?
>
> 3) Do we further agree that requirements should be *verifiable*, i.e.
> that tests can eventually prove that the thing built (our
> methodology, in this case) meets the requirements we have specified?
>
> If we agree on these three points (and I hope we do) then R03: Unique
> interpretation and R04: Replicability should be first of all feasible;
> they should be shown to be necessary (e.g., the methodology would have
> reduced credibility without them); finally, they should also be
> verifiable (e.g. replicability and uniqueness of interpretation can be
> proven in independent tests of a real-world sites).
>
> If you agree so far, were do we stand in this?
>
> *Feasible:* I have not read a single statement on this mailing list so
> far that has offered any evidence that replicability and unique
> (unambiguous) interpretation are feasible - especially if the
> methodology stays on a fairly generic level (i.e., if it does not
> prescribe the tools to be used, a step-by-step procedure, and detailed
> instructions for evaluating test results).
>
> *Verifiable:* We do not know yet, we have not built anything so far that
> we could use to carry out tests independently and then compare results.
> So let’s move on to second-best, the various methods we currently use. I
> would ask all of you to report on any tests that were carried out by two
> independent testers and arrived at the same result. No one has come
> forward and claimed it has happened, or even, that it can be done.
>
> *Necessary:* Some of you may believe that replicability and uniqueness
> of interpretation are necessary because the methodology would be less
> credible without them. But unless the methodology mandates that tests
> are actually replicated, the claim of replicability is just a red
> herring. I think that any claims that cannot be verified in practical
> application seriously undermine the credibility of a methodology.
>
> Detlev
>

-- 
Shadi Abou-Zahra - http://www.w3.org/People/shadi/
Activity Lead, W3C/WAI International Program Office
Evaluation and Repair Tools Working Group (ERT WG)
Research and Development Working Group (RDWG)

Received on Tuesday, 13 September 2011 07:50:53 UTC