- From: Rob Lanphier <robla@real.com>
- Date: Thu, 28 Feb 2002 10:39:22 -0800 (Pacific Standard Time)
- To: Alex Rousskov <rousskov@measurement-factory.com>
- cc: Lofton Henderson <lofton@rockynet.com>, Al Gilman <asgilman@iamdigex.net>, "www-qa@w3.org" <www-qa@w3.org>
On Thu, 28 Feb 2002, Alex Rousskov wrote: > Personally, I doubt it is possible to produce [a checklist to > objectively rate test suites]. My > doubts are based on my inability to suggest such a checklist for test > tools I am familiar with. They are too different in scope, > functionality, and intended audience to be rated using common > criteria. I was going to object to this point, but in writing my response, I find myself in the middle. I think Alex is right in saying that this group won't be able to construct a checklist that is safe to mechanically apply to test suites to determine their conformance level. There's going to be qualitative judgements made. That said, the W3C makes qualitative judgements all of the time, any time a specification is promoted to "Recommendation" status. So, there may need to be various levels of blessing of test suites, and a process for getting those test suites blessed. Part of the process would most likely involve measuring the test suite against a checklist, but there should be a process to ensure that we arrive at a consensus judgement regarding the satisfaction of that checklist. Rob
Received on Thursday, 28 February 2002 13:45:55 UTC