Re: Proposal on forming of W3C Test Group (action item A-2002-03-1-3)

DImitris,

Thanks for the proposal and starting this dialog.  I have a couple of 
comments in-line, to your and Kirill's comments...

At 04:58 PM 4/4/02 +0200, Dimitris Dimitriadis wrote:
>Hi Kirill
>Thanks for your comments. As I noted, this was a primer, it is certain to 
>change as more comments are made. My replies inline:
>
>On Thursday, April 4, 2002, at 04:44 , Kirill Gavrylyuk wrote:
>>Hi, Dimitris!
>>Have 2 initial questions/suggestions:
>>- Why should this be independent group? If you're saying that
>>...it is important that there is a group that can take over after the
>>current QA WG finishes the more process-oriented work.....
>>Isn't it what the original QA WG was chartered for? Why not just have
>>the Testing Group as a task force inside QA WG?

I'm a little uncertain about the scope of the activities and deliverables 
of the proposed group, so maybe defining it more closely -- Kirill's second 
point below -- should be a prerequisite to deciding about organization.

But what I think I'm reading is that you're proposing that the scope 
includes the 4th parts of the Frameworks ("Test Materials"), and related 
QAWG charter deliverables (e.g., a couple of the last bullets at [1]).  If 
so, here are a couple of observations:

-- I'm not sure about the characterization of "jobs of two groups different 
in principle".  I view it more as a practical necessity that, with limited 
resources, we are tackling a big set of problems in a logical 
order:  Process, Specs, then Test Materials.  It could be that more 
parallel development is possible, but I don't see that we have the resources.

-- I myself view the "Test" as the most fun part, and hope that we (or at 
least I) will be able to spend more time there, after we have some momentum 
and maturity on the first parts.

>In order to facilitate information exhcange between other interested 
>parties (other WGs) I think it's simpler to go directly to a WG instead of 
>going to a task force set up in a WG (in my experience, at least).
>
>Of course, and especially given that formal aspects are not that important 
>after all, setting up a task force may be another option to consider. I 
>lean toward separating the two, though, especially insofar as enhancing 
>the two group's (overlapping but not identical) goals is concerned.

On the one hand, at the QA Workshop we did leave the door open to creating 
more WGs within the QA Activity.  On the other hand, there is a lot of 
overhead.  A good part of QAWG's energy for its first few months was 
absorbed in logistics, procedures, methods, tools (e.g., Issues list, doc 
technologies, etc), Web site, establishing liaisons, ...

For that reason alone, I'd favor an initial "task group" approach, unless 
it could be demonstrated that having a separate new WG under the QA 
Activity would, by itself, lead to a big increase in new members.  I.e., 
will lots of people join and be active in a new Test WG, who wouldn't join 
a "Test Task Group" within QAWG?


>>- I'd empathize somewhere in the proposal what are the specific
>>deliverables of the Testing Group and what is the expected term of
>>functioning. You're saying "to aid in producing test materials" and "to
>>help in producing specification authoring tools", but I'd like it to
>>have more finite term and measurable goals.
>[dd] Great suggestion, thanks. I left it out because I wanted the WG to 
>consider even the basic draft, but I will incorporate this information in 
>the next iteration.

I'm interested in these details, also.

Thanks for the effort to start sorting this out.  Whatever the ultimate 
organizational details, it clearly is the next major piece of the QA's 
work.  It also relates to a question that I have been thinking about for a 
while:  with our current QAWG scope and near-future plans, are there 
omissions that interest people?  I.e., are there topics and subjects that 
people, including potential new participants, might want to work on?

-Lofton.

[1] http://www.w3.org/QA/WG/charter#Scope


>>Thanks
>>
>>-----Original Message-----
>>From: Dimitris Dimitriadis [mailto:dimitris@ontologicon.com]
>>Sent: Thursday, April 04, 2002 5:20 AM
>>To: www-qa-wg@w3.org
>>Subject: Proposal on forming of W3C Test Group (action item
>>A-2002-03-1-3)
>>
>>
>>QA WG,
>>
>>Below please find my outline of a proposal for forming a dedicated Test
>>Group within or in parallel to the QA activity. For simplicity's sake I
>>write in in email form, after receiving comments I will circulate a more
>>
>>usual html draft.
>>
>>Forming a QA Test Group
>>
>>Rationale: Simplify production of Test Suites (with the required minial
>>set of quality requirements), enhance coordination between WGs and allow
>>
>>for easier asserting of cross-specification functionality. Also to
>>increase the practice of the guidelines produced by the QA WG.
>>
>>Introduction: Most of the QA-related test activities that have been
>>produced up until this date in connection to W3C specs have either been
>>inernally produced within the particular specification's WG, or
>>incorporated in coordination with external parties. This has lead to
>>quality testing frameworks on the one hand, but in some cases very
>>different and incosistent, on the other.
>>
>>Proposal: According to the work that this WG produces, the W3C and its
>>WGs shouls produce test suites that are checked against the same
>>guidelines and checkpoints. This implies that they should share some
>>basic functionality and design. In order to achieve this and ease the
>>burden on the WGs that are to produce the Test Suites, I propose that
>>the W3C form a special Test Group which has the characteristics below:
>>
>>1. Have full and normative knowledge of the various QA-related
>>frameworks within the W3C, especially with regard to tests and
>>conformance issues (as opposed to specification authoring, for example)
>>2. Help WG representatives to produce Test Material (mentioned, but not
>>formalized so far in the QA WG work)
>>3. Aid in producing Test Materials to be used for testing
>>interdependencies between implementations of specifications
>>4. Help in producing specification authoring tools that allow easier
>>generation of test materials (particulalry important given the current
>>idea of enhancing granularity of schemas used to write W3C
>>specifications)
>>
>>Organization: The Test Group should ideally be another (technically
>>oriented) WG within the QA activity. The reason it shoudl be a WG is
>>that it needs a chair, for coordination, W3C staff allocated in order to
>>
>>make sure the technical architecture is there, as well as W3C member
>>organizations and/or invited experts.
>>
>>Also, it is important that there is a group that can take over after the
>>
>>current QA WG finishes the more process-oriented work. It is certain
>>that there will be many issues witht the testing frameworks that will be
>>
>>produced, and even more certain that various issues will arise in
>>connection with conformance claims that will be made by various
>>implementors.
>>
>>Concluding, I not that one was of viewing this group's work (but not
>>exhaustive) is as the more techically oriented parts of the current QA
>>WG documents: making what we say there actually happen.
>>
>>I look forward to the WG's comments.
>>
>>Kind regards,
>>
>>/Dimitris
>

Received on Monday, 8 April 2002 13:25:24 UTC