Testability again (was: [QA Review] CharMod for the Web 1.0: Fundamentals WD 25 Feb 2004)

* Karl Dubost wrote:
>This is a review of

>http://www.w3.org/TR/2004/WD-charmod-20040225

>C001   [S]   [I]   [C]   Specifications,  software and content MUST NOT 
>assume that there is a one-to-one  correspondence between characters 
>and the sounds of a  language.
>
>===> How do you test that for each implementations [S][I][C]? What will 
>be the three tests that you will be able to create to demonstrate the 
>implementability of this during the CR period where you will seek for 
>implementation? If you can't design a test for it, it means that your 
>assertion is not testable, therefore not implementable.

I am confused. How is a "test" relevant in this regard? You would test
a specification by reading it which would not involve the creation of
tests. Maybe you could point at the tests the QA WG created for SpecGL
to give an idea how to design such tests for Charmod? It also seems that
in order to create a test for software or content one would need an
interface to these things, an API or a specific document format, none of
which is defined in Charmod and hence cannot be directly tested through
Charmod. My understanding is rather that specifications that require
Charmod conformance are responsible to demonstrate interoperability. You
would not create standalong charmod conforming software or content.

I also wonder what the QA WG means by "testable", my LC comment

  http://www.w3.org/QA/WG/2004/02/cr-issues.html#x30

still appears to be unresolved. This makes communication quite
difficult, I can hardly argue with you whether certain things are
testable if we disagree about the definition of "testable".

Based on your comments I would say that I strongly disagree with your
definition. As far as I am concerned, human inspection of a spec makes
it possible to determine with complete reliability (!) whether the spec
adheres to Charmod or not (minor issues aside). Much like with SpecGL.
So, what am I missing here?

> I think one of 
>the problems comes from the "assume".
>	Imagine a language where you have "a one-to-one correspondence between 
>characters and the sounds of a  language". If the software implements 
>only this language because it's a specific use for only this language. 
>It means that it's not conformant to C001, even if this software does 
>the correct thing.

No, you cannot know and assume the same thing at the same time. If you
know that the only language you implement has this characteristic then
you do not assume this characteristic and are thus conformant.

>*KD-004
>C008   [S]   [I]   Specifications and implementations of sorting and 
>searching algorithms SHOULD accommodate all characters in Unicode.
>
>===> What's happening if you implement all western languages but not 
>asian because the context of applications do not make it necessary. Do 
>I still have to implement everything? If not how can I be conformant?

If you have fully understood and carefully weighted the implications of
the deviation, you may deviate. This is covered by RFC 2119 which is
properly referenced on Charmod. You will likely consider the definition
of "SHOULD" in RFC 2119 not testable, Charmod already kindly addressed
this, 

[...]
  A specification conforms to this document if they:
[...]
  2. documents the reason for any deviation from criteria where
     the imperative is SHOULD, SHOULD NOT, or RECOMMENDED,
[...]

Did the QA WG resolve this in their CRs in a more "testable" or
otherwise generally better manner?

Received on Sunday, 11 April 2004 18:11:54 UTC