Re: Testability and normative requirements

(Mark, a question for you in the text)


Le 06-03-08 à 23:40, Dominique Hazael-Massieux a écrit :
> As discussed during the IG F2F [1], I started a Wiki page on this  
> topic:
> http://esw.w3.org/topic/NormativeButNotTestable

And the text is for now
[[[
When writing a specificiation, it is sometimes tempting to use non- 
testable text for normative requirements; indeed, making sure a text  
is testable requires much more work than using simple text without  
caring about testability (see also TestableOrNot).

But using non testable text as normative requirements has many  
drawbacks:

     1. mandating something that cannot be tested is a no-op; how can  
you check whether something was indeed implemented if you cannot test  
it?

     2. non-testable requirements means that implementations are  
likely to differ on the said requirement, meaning that  
interoperability will be loose at best

     3. leaving a requirement in a fuzzy non testable state means  
leaving the disambiguation work to the implementors, making it much  
more costly and much more likely to generate confusion for the end users
]]]
-- Normative But Not Testable - ESW Wiki
http://esw.w3.org/topic/NormativeButNotTestable
Tue, 11 Apr 2006 06:02:39 GMT

Dom has requested for example of non testable requirements.


Provided by Ian Hickson
[[[
  [...] user agents must make a best attempt to render all characters,
  regardless of the value specified by lang.
  -- HTML4 section 8.1.

  Those browsers that interpret soft hyphens must observe the following
  semantics: If a line is broken at a soft hyphen, a hyphen character
  must be displayed at the end of the first line. If a line is not
  broken at a soft hyphen, the user agent must not display a hyphen
  character.
  -- HTML4 section 9.3.3.

  User agents must know where to render the header and footer.
  -- HTML4 section 11.2.1.
]]]

-- Re: Testability and normative requirements from Ian Hickson on  
2005-12-21 (www-qa@w3.org from December 2005)
http://lists.w3.org/Archives/Public/www-qa/2005Dec/0008
Thu, 22 Dec 2005 20:40:48 GMT

I think there's a missing part in the wiki text for now to give a bit  
more context:

What is the meaning of "normative"? Why do I ask something which  
seems obvious?

    * A normative requirement is a requirement defined by one of the  
RFC2119 keywords.
or
    * A normative requirement is a requirement defined by MUST keywords.

(Here I have chosen RFC 2119 for simplicity but it could be any kind  
of requirements rules defined in the Conformance section of a  
specification).

RFC 2119 uses:
	- absolute requirement (MUST, REQUIRED, SHALL)
	- absolute prohibition (MUST NOT, SHALL NOT)
	- particular item      (SHOULD, RECOMMENDED)
         - particular behaviour (SHOULD NOT, NOT RECOMMENDED)
         - item                 (MAY)

which shows btw that RFC 2119 is not really consistent. I remember  
that at the WWW2002 Conference, Mark Skall had presented a paper  
about the problems of RFC 2119.

Mark, do you still have this paper and could you send the text in a  
mail on this list?

Ok going (slowly) to my point. IMHO, a non testable requirement  
doesn't have the same implications if it's a
	- MUST-non_testable-requirement
	- SHOULD-non_testable-requirement
except if we consider that a SHOULD is not a requirement, but I don't  
believe so. It has other implications.

So a thumb rule which comes to my mind is

	* If your requirement is not testable, make it optional.


 From the Mailing List Front
"Non testable":   10 occurences
http://www.w3.org/Search/Mail/Public/search?keywords=%7Bnon+testable%7D

"Not testable":  217 occurences
http://www.w3.org/Search/Mail/Public/search?keywords=%7Bnot+testable%7D

I think it is related to another issue:
	Meaning versus Behavior
	http://esw.w3.org/topic/MeaningVsBehavior



-- 
Karl Dubost - http://www.w3.org/People/karl/
W3C Conformance Manager, QA Activity Lead
   QA Weblog - http://www.w3.org/QA/
      *** Be Strict To Be Cool ***

Received on Tuesday, 11 April 2006 06:51:59 UTC