Draft minutes of QA F2F meeting in Boulder, 21 Oct AM

QA Working Group Face-to-Face (at NIST Boulder)
Tuesday, 21-October-2003, Morning
------------------------------------------------
Scribe: Patrick Curran

Attendees:
(PC) Patrick Curran (Sun Microsystems)
(KD) Karl Dubost (W3C, WG co-chair)
(DH) Dominique Hazaël-Massieux (W3C)
(LH) Lofton Henderson (CGMO - WG co-chair)
(SM) Sandra Martinez (NIST)
(LR) Lynne Rosenthal (NIST - IG co-chair)
(MS) Mark Skall (NIST)
(VV) Vanitha Venkatraman (Sun Microsystems)
(OT) Olivier Thereaux (W3C, IG co-chair)

Regrets:
(DD) Dimitris Dimitriadis (Ontologicon)
(AT) Andrew Thackrah (Open Group)

Absent:
(KG) Kirill Gavrylyuk (Microsoft)

Guest:
(AR) Alex Rousskov (Measurement Factory)

Agenda: http://www.w3.org/QA/2003/10/f2f


Summary of new action items

AI-20031021-1: LH to talk to Daniel about meeting with Arnaud from IBM 
to encourage their participation in QAWG
AI-20031021-2: KD to speak with Steve Bratt to encourage Microsoft's 
participation in QAWG
AI-20031021-3: PC to follow up with Oracle to encourage their 
participation in QAWG
AI-20031021-4: KD is appointed director of recruiting - he will follow 
up on recruiting efforts

Minutes:

1. Discussion on membership & recruiting

General agreement that we need to recruit more members.
Brief discussion of possible activities for Advisory Committee meeting 
(November 2003, Japan).
No conclusions.

KD:  We have no Japanese member in QAWG - could we try to recruit someone?
All: we need to encourage more participation generally
LH: need to issue a call for participation when we re-charter; this 
should be more than pro-forma
DH: could we get someone from IBM?
OT: Boeing may be interested - I am working with them...
LH: those of us who are active in "real" working groups should try to 
recruit from these groups
KD: Now that Peter is not participating, will this cause problems for 
Patrick on TestGL?
PC: Yes - let's discuss when we get to Test GL
LR: When PC visits NIST, can meet with Mary Brady and review the doc; 
will get good feedback
DH: how will we follow-up on these efforts?

2. Inreach/Outreach

OT leads the discussion

Definitions: Inreach = working with people inside W3C, Outreach = 
outside W3C

All: We haven't done much inreach since the TP earlier this year. (We 
have done some reviews.)
LR: CC/PP represents a success story. They were reluctant to listen to 
us initially, but LR
reviewed their spec, recruited them and they added a conformance section.
OT: Is not sure that they were really committed to QAWG processes.
LR: They did address the classes of product, and they re-wrote a section 
of the spec
when it became obvious that it wasn't clear and understandable.
DH/LR: The review process is valuable
LR: Also reviewed UUAG - they were 95% compliant. This is probably 
because Iain Jacobs,
the spec lead, is very experienced
OT: CC/PP has a very long history - several drafts. They thought they 
were done a year ago.
There were already several implementations, yet did not have conformance 
requirements in
spec. Some other WGs who are not so far along do not seem to see the 
importance of this.
DH: We do a lot of reviews, but there isn't much follow-up
KD: We are only reviewing Last Calls, yet we could be more effective if 
we engaged earlier.
We don't even have enough resources to do this...
MS: Early engagement is good
OT: Don't review early drafts - WGs will be resistant
LH: It would be better if we reviewed draft before Last Call
KD: Re-emphasises that early engagement would help
AR: IETF is considering how to improve spec quality. People seem to 
believe that early
review will help. However, this does take more resources.
OT: If we agree that early reviews are best, how to track them? Last 
Calls, CR are
announced, but working drafts are not so widely published.
MS: We have asked in OpsGL that each WG should be assigning a point of 
contact. We should
ping them.
DH: Indeed - if people were following our Ops GLs, we would be OK
LR: We should focus on a few WGs, and build up good relationships with them
LH: Does IETF have anything like a QAWG?
AR: To migrate from draft to RFC specs must be reviewed by "ISG".
LH: what would they think of SpecGL?
LR: they probably havn't heard of it. There are minimal requirements. 
Things are changing.
ISG gets overloaded with poor-quality submissions. They are discussing 
how to improve the
situation. They are thinking of assigning a group of people who would 
review drafts.
Public list of specs that have been reviewed (by us) will encourage 
others to participate.
LR: It would be good if we published a list of common errors
DH: So - what to do?
LR: Target a small number of working groups
OT: Agreed - target a small number of WGs early
PC: Early engagement should focus on OpsGL process stuff rather than 
reviewing specs
KD: This would help to dispel the myth that we use waterfall model

OT: Summarizing - in early stages we are trying to influence behaviour 
rather than review
early drafts of specs.

DH: We need to work on our communications with WGs. We would be more 
effective in reviewing
Last Calls if we had built up a good working relationship along the way, 
and had helped them
to incorporate good practices.

VV: We should assign an individual liaison to work with them.

LH: Summarizing - pick a focused subset, assign a contact person in the 
WG, and a liaison
in QAWG

AR: Doesn't think that we have resources to assign a dedicated liaison - 
why not just review
their docs?

DH: closer relationships will help.
LR: sounds like we're trying to do too much...

Moving discussion to how we validate our specs

LH: all we need is two groups implementing each checkpoint (doesn't have 
to be one group
that implements "several" checkpoints).
KD: no - W3C management wants us to demonstrate that two WGs have "good 
coverage"
OT: we must to this if we want our guidelines to become mandatory
KD: we must prove the maximum rather than the minimum - it's difficult 
to recruit - he needs help
AR: should work with new WGs that are just forming
KD: mail is not effective - need phone or face-to-face
KD: there may be an opportunity to meet with several WGs at the Japan AC 
meeting

LH: Our OpsGL CR call obligates us to work closely with six WGs, so we 
seem to have
already committed ourselves to the "focus and liaise" approach

DH: given this, should we assign people to work with WGs? Getting WGs 
into compliance
with OpsGL and SpecGL would be good, but higher priority should be 
involvement,
education. OpsGL & SpecGL are tools to help get us there. First identify 
the WGs,
contact them, build up a relationship. (The docs can be overwhelming.)
If nobody is interested in doing the work this is all academic.

KD: Assigned liaison can follow the WGs activities via mailing list, 
jump in if they see
a problem.

OT: We really need people to participate.
KD: This is not optional - we have to do it, otherwise we'll have to 
drop Ops/SpecGL.

LH: Is on SVG, but is falling behind
DH: Will work with Multimodal working group
SM: XML core
PC: Leonid Arbouzov (works for PC) is on XML schema
LR: Tim Bolan (NIST) and LR will do CSS
KD: Will do HTML and I18N
LH: Dave Marston might we willing to do Query

DH: Should we ask for monthly reports from these liaisons?

All agree

LH: We're not necessarily volunteering for formal Ops/Spec reviews. 
However, we're more
interested in building a relationship, teaching people good processes.

LH: We will put these reviews on agenda for last telecon of each month

[break...]

2 continued - Outreach

LR: Asks whether "month in QA" is useful
OT: Yes - it's useful, but during the past few months we haven't had 
much to report
When we did have stuff to report, this did seem to generate responses. 
However, we
don't know whether this would have happened anyway.
LR: If we do want to continue, should we be generating materials in some 
other manner
than reviewing our archives?
OT: We could make it irregular - publish only when we have news
LH: How many hits do we get?
OT: Doesn't know - people aren't eagerly awaiting it
LR: Thinks we should publish on a regular schedule
LR: Agrees to continue to own this. If people have suggestions for 
additional material
(other than reviewing the archives) please let her know.  It can take 
her several hours
to put one of these together.

Agreed: LR will continue to do this, but it's OK to skip a month 
occasionally.
She will also try to include material other than archives.
AR: call it "QA Times", so it doesn't have to be on a rigid schedule?

OT leads the discussion

OT: Things are going slowly
DH: Asks AR how it looks from his perspective as an outsider.
AR: Doesn't see much communication
OT: QA tips, everything published on our webpage, counts
AR: People don't know the webpage exists
DH: We have various mechanisms - mailing list archives, "google effect", 
we do presentations
occasionally
AR: You don't have outreach - you publish materials - but you're not 
actively trying to reach
people
KD: We should make announcements when stuff is published - via relevant 
mailing lists
We could do a better job of publicising our tools. Also, we need to 
document them better.
It would be good if we could get contacts inside "developer programs" at 
Sun, Apple,
Microsoft and other big companies

LH: What outreach projects are in progress right now?
OT: the collection if "QA tips", the "WaSP ask W3C" project, and the Web 
quality documents
of the QA library, which we are working on consolidating with 
contributions from IG participants.
LR: We also participate in external activities (such as the voting 
systems panel that's coming
up) - we should highlight this more
LH: discussions on IG lists (eg PNG v. GIF) - do we highlight them in QA 
Times?
LR: yes
LR: people should let her know about presentations
OT: if materials are public, we can put them into our library

3. Plans for next Technical Plenary (1-5 March 2004, Cannes-Mandelieu, 
France)

When should we plan our f2f meetings?
Unsure yet who is planning to attend

LR: should we plan a f2f in January?
DH: not sure we'll have enough to work on
MS: maybe TestGL?

LH: should we plan a Wednesday presentation at TP?
MS: we should - this is an opportunity to pitch to WGs
LR: can we get a WG to join with us to talk about what worked and what 
didn't?
DH: we should choose the group early, and work closely with them
DH: we should request a slot on Wednesday
OT: should have a backup plan if we don't have enough material, or can't 
find a
willing WG? Could we participate in an existing session?
DH: a panel might be a better idea than having a single WG
DH: what kind of outreach activities do we want to engage in?
LR: should we try to engage with additional WGs?
PC: focus on deepening relationships with groups we're already engaged 
with,
rather than a shotgun approach of meeting with others
DH: issue an open invitation - let people know that we're willing to 
meet with anyone
who's interested
LH: should we do a more technical presentation? perhaps on test-case 
description language,
or something similar?
DH: perhaps we could demo something...
LH: a detailed review of checkpoints would not be very interesting to people
LH: should we plan on working on TestGL during this time? We're pretty 
good at processing
issues during telecons - may not be best use of our time to do this. 
Should spend time
consolidating our CR experience for OpsGL and SpecGL.
LH: On Thursday and Friday, QAWG members who are planning to stay should 
make themselves
available to meet with interested WGs.
LH: should meeting with interested WGs on Monday/Tuesday take precedence 
over our work?
DH: decide this on a case-by-case basis

Received on Thursday, 30 October 2003 20:15:42 UTC