Re: new Guidelines drafts posted

Responding to Alex Rousskov's comments:
>> Checkpoint 1.1. Define the scope of the specification.
>>
>> To fulfill this checkpoint, a specification MUST define what
>> scenarios are in scope...

>...It is not clear whether ALL or just
>some in- and out-of-scope scenarios must/should be defined....
>If ALL scenarios must be defined, then the checkpoint becomes untestable
>because it is impossible, under general assumptions, to know whether all
>scenarios have been covered by the scope definition....

Agreed. And some specs define facilities that get put to additional uses
later on. I think this checkpoint is trying to require the WG to explain
why it spent all the effort to write a spec, so they only need to
provide a list of scenarios sufficient to convey their intent.

>> Guideline 2. Identify what needs to conform and how.

>What negative impact will removing this guideline from SpecGL have?

This GL arose because a spec that defines the behavior of a consumer
of the described XML may be seen as imposing some constraints on the
producers of that XML. Case in point: XSLT defines an XML vocabulary
and what a particular type of consumer (XSLT processor) must do with
it. There is a benefit to saying explicitly that only XSLT processors
can be judged on their conformance to the spec, not the various
stylesheet generators and editors that are out on the market. (BTW,
the XSLT 1.0 spec fulfills this checkpoint.)

>> Checkpoint 3.1. Specify any universal requirements for minimum
>> functionality. [Priority 1]

>I also question the priority level assigned to this checkpoint.
>Again, given a good conformance clause, having "minimum
>functionality" documented is a nice-to-have not must-have.

Think about creating a test suite. Think about vendors bragging
that their products are conformant. Think about interoperability.
If this checkpoint is met, then the WG can identify a set of test
cases that all conformant products must pass.

>> Checkpoint 4.3. If profiles are chosen, define their relationships
>> and interaction with other dimensions of variability. [Priority 2]
>> To fulfill this checkpoint, a specification MUST identify all the
>> relations and interactions between profiles and the other
>> dimensions of variability. It is not applicable if profiles are not
>> used.

>This checkpoint is not testable. The authors may claim that all
>interactions are identified, but it is not possible, in general, to
>verify that the authors did not miss any implicit interactions.

I wouldn't want to abandon this checkpoint just because it could be
difficult. If the WG overlooked an interaction in Working Drafts,
it should be revealed when developers try to implement the spec and
when testers try to write test cases. My advice to the person trying
to assess the spec on this checkpoint is to review as follows:
For each profile, check whether the profile requires particular
choices for any discretionary items, check whether the profile has
levels, check whether the profile requires particular levels of any
common features, check whether the profile constrains extensibility,
[and so on].

>If we are only concerned about explicit interactions, then all
>explicitly defined interactions are, of course, "identified" (i.e.,
>any spec would satisfy the above checkpoint).

I hope that's true. I worry that in the complicated specs, there
could be enough verbiage to prove that an interaction exists, but
not enough to know all the applicable rules.

>> Checkpoint 5.1. If modules are chosen, indicate any mandatory
>> conditions or constraints on their usage. [Priority 1]

>This checkpoint is not testable.

Unlike profiles, modules are always completely enumerated in the
spec. This guideline is simply allowing for the possibility that
a particular module could be a "core module" required of all
products in a class. That seems testable to me. Maybe this
guideline needs a rewrite.

>> Checkpoint 7.1. Identify each deprecated feature.

>This checkpoint is not testable. See comments for checkpoint 4.3
>above.

In other words, you think a feature could be *implicitly*
deprecated? How would that occur? Would it make it harder to write
test cases for that spec, thus violating the guideline?

>> Checkpoint 8.2. For implementation dependencies, address the
>> allowable differences between implementations [Priority 1]

>This checkpoint is not testable.

A non-ideal way to test the spec is to attempt to code an
implementation, and while coding keep a log of all instances
where the spec left externally-detectable behavior unspecified.
(Sounds like current practice.)

>> Checkpoint 9.4. Use a standard mechanism to define the extension.

>What is a "standard mechanism/way"?

I agree that it's badly worded. An example would be a function
   boolean  function-available(string function-name)
that is included in the standard's own expression language. It
probably should say an "interoperable way" to check whether an
extension is present.
.................David Marston

Received on Sunday, 5 January 2003 22:33:03 UTC