- From: by way of the Lastcall Form <pfawcett@real.com>
- Date: Thu, 13 Mar 2003 05:03 +0900
- To: www-qa@w3.org
Here is a last call comment from Peter Fawcett (pfawcett@real.com)
on QA Framework : Operational Guidelines (and Examples and Techniques)
received by the LC form system.
Submitted on behalf of: QAWG sub group on Test Suite Process Document
Comment type: Substantive
The comment applies to: "Overall"
Comment title: OpsGL Appendix 1 - Process Document Template
Comment:
- "test material" is referred to in multiple ways in the document
including but not limited to "test material" and "test material type".
We could not determine a difference in concept from context nor could we
come up with an example that needed both.
- The language in Rec 1 seems odd. Especially the second sentence.
"The Checklist should be developed in a public framework.
The development of this framework itself is a central area of
interest. The QAWG welcomes the participation of interested
parties in developing the Checklist."
- The third sentence in Rec 3 also seems a bit odd
- We found a number of uses of "test materials" and
"test suite" that did not use the "fill-in" markup.
- DocumentFramework should be Document Framework
- In Test materials contrabuition section in the a/b choice rather than
saying tests or series.. say contributions.
- In Test Materials contribution section there is a missing <span></span>
around address.
above]</span> mailing list: [address].</p>
- We found the first item in the list under Test Materials Contribution to be
some what redundant. At least in our case as any submissions would already
go to the list as it is.
The test or series of tests, get submitted to the framework. Submitter
should also send a notification to the mailing list that will be set up
for the Checklist to indicate that he/she has submitted a test to the
class="fill-this-in">[test material name] framework.
- There is a typo in section Receipt and review of test contributions
"according tot he" should be "according to the"
- In Receipt and Review section list item 1
it currently says:
"since it tests a feature that is not in the specification"
should it say something more like:
"since it tests a recommendation that is not in the specification"
as 'feature' is not a universally applicable term.
- In Receipt and Review section list item 3:
This item makes the assumption that the test cases will be written
in some xml based language.
How ever can we make this assumption? What about languages like
rdf that don't use an xml syntax?
- this same section (item 3) also has a hanging ']</span>' on it.
- In Receipt and Review section list item 4:
"<li><em>Scenarios</em> that underlay the particular test layout and its
intended scope.</li>"
We found the use of "test layout" to be out of place in the document.
it is the only place that this term is used. We believe that it could
just as easily use language that's more standard to the document.
- In Receipt and Review section list item 5:
The point of this checkpoint is valid, in that if the submitted test doesn't follow
the Development Guide it should be rejected, how ever the notion of the Development
Guide is introduced here (in this document). We feel that it should be listed
earlier in the document as well. perhaps in the requirements, that the test
materials be developed in accordance with the Development Guide.
- Typo in last sentence of Receipt and Review section:
is "publication arereurned to"
should be: "publication are returned to"
- In Test status and review procedures Section:
In the second paragraph it reads:
"the status of the test is changed to reflect the fact that its
validity has been disputed"
Earlier we say that "status is changed to "accepted"
we should use the same verbiage here like: "state changed to "disputed""
- In Test status and review procedures Section:
The last sentence of the second paragraph has very odd wording.
- In Test status and review procedures Section:
second paragraph uses term "Task Force" in terms of the maintainers of the test
materials. This is another concept that is first introduced in this context.
This too would work better if the concept of appointing a "Test Task Force" was
done at an earlier step.
- In Test status and review procedures Section in the first item of the list.
The label "stable" may not be the best word for case 1 as it doesn't
really reflect what is meant. "Error Free" or "Valid" would work better.
- In Test status and review procedures Section in the third item of the list.
It reads: "questioned the correctness, consistency, clarity, etc"
we think that correctness does not belong.
- In Test status and review procedures Section in the second and third item of the list.
"A consensus exists in the community"
What community? The w3c, the 'whole internet' the WG (as this is specifically
talking about the task force.)
We are also fairly uncomfortable with the notion that the task force can be
over ridden by the (not clearly defined) community.
Either the taskforce agrees or disagrees and the case accepted or not.
Unless "community" means WG in which case this might make sense...
- In "Section Status of the test suite according to the above"
We feel that the wording of this section heading could be improved quite a lot.
We assume this means that if you conform to all of the above. or you've done/are
doing all of the above.
- In "Section Status of the test suite according to the above" second paragraph
It reads:
It is proposed that the W3C WG representative act as moderator and controller
for incoming tests. The moderator is chosen by the [WG name] WG. All tests
should be kept for archive purposes, whether they get published or not.
much of this is repeated from above. The new information is that a moderator
should be chosen. This is another case where it may be beneficial to introduce
the concept of a Moderator earlier in the document.
This is already covered above to some degree. But with moderator.
- Both the Documentation and "See above" sections do not have references back to
a GL document.
Proposed resolution :
]]
--
This comment was submitted through the lastCall form system,
designed by Martin Duerst and Adapted for the QAWG by Olivier Thereaux.
Received on Wednesday, 12 March 2003 15:03:43 UTC