TestGL Issues

All,

Below you'll find rationales for how we adressed some of the TestGL 
issues sent to the QA WG some days ago.

11: We still need Alex's minutes for the HTTP example
12: I don't understand what "explicitly tests other specifications" 
means or where it should be stated. Ckpt stands well as is. It probably 
alludes to interdependance but has nothing to do with explicitly 
_testing_ other specifications
13: relies on whatever wording is supplied for issues 2-5
14: closed, still appears in the issues list.
16: closed, still appears in the issues list.
18: I don't understand what the last bullet in the original issue 
amounts to (a new bullet should be added to include conditional tests 
as an optioanl metadata; how can tests be metadata?)
18: I have to disagree with one point which I didn't see before: 3rd 
bullet in 3.1 mentions additional information for tests. this can 
actually be set on a higher level for an entire set of tests. for the 
needs of this version, I think we're OK with what we have, but let's 
elaborate on that in future versions
20: done, still appear in the issues list.
25: should be marked as deleted, since it got rolled into issue 23
29: should be marked as deleted, since it got rolled into issue 28
32: The explanation of my coming up with the issue is: It's one thing 
to test the tests that go into a test suite (validation), it's another 
thing to test the test suite before release (correctness fo different 
kind). However, from the point of view of the test suite user, the WG 
inner workings on testing tests (and providing metadata etc.) is not 
relevant, therefore I drop this issue (what I mean is actually covered 
by issue 31)
38: should indeed be dropped, since what we want is covered in other 
checkpoints
41: Agree with that it is not reduntant and leave wording as is
43: I don't understand. In any case, I think previous issue resolves 
some points that I suppose this issue adresses.
46: cannot see how proposed wording helps much. Propose to leave as is.
48: I don't understand
50: I can't find a convincing argument for either changing the wording 
or providing counter arguments, given that the issue's originator 
believes that "to test for conformance [...] is trivially incorrect". 
Since what we do _is_ conformance testing related, I think there is not 
much room for debate.
51: The checkpoint wording does not imply a minimum of number of people 
responsible for metadata, just that the WG must do it.
52: This metric is definitely not useless but not ideally defined. I 
suggest that we keep the issue open until a more concrete definition of 
coverage information can be given. What we aim at is, simply put, how 
much of the specified functionality has corresponding tests.
53: Checkpoint deleted
54: Misreading. The checkpoint does not stipulate that the WG come up 
with a catch-all solution, rather that, as a design issue, the test 
execution process be automated and require as little manual 
interventation as possible. This is especially important since the 
outcome of the test suite must be deterministic and repeatable.
55: Patrick, can we change priorities here?
57: Checkpoint wording clarified

Received on Monday, 23 February 2004 14:04:47 UTC