Raw minutes from 20 September UAWG teleconference

20 September 2001 UAWG teleconference

Agenda announcement: 
Participants: Jon Gunderson (Chair), Ian Jacobs (Scribe), David
Poehlman, Mickey Quenzer, Harvey Bingham, Karl DuBost, Jim Allan,
Tim Lacy 

Regrets: Al Gilman, Eric Hansen, Loretta Reid
Absent: Denis Anson, Gregory Rosmaita, Rich Schwerdtfeger

Previous meeting: 6 Sep 2001

Next meeting: 27 September

Reference document 12 September Candidate Recommendation:

1.Test Suites

 /* Karl Dubost joins the meeting. Karl is the W3C Conformance
 Manager */

 KD: W3C just started the Quality Assurance (QA) Activity. We are not
 yet organized, and will be developing guidelines.

 KD: First of all, let me say that your conformance section is very
 good, and I appreciate the way that you have modularized conformance.

 KD: I think it's very hard to do a full/complete test suite. If you
 want to organize your work, you should first find the testable
 assertions in the Recommendation. Build test suites from these. 
 For example, see the SVG test suite guidelines written by Lofton

 KD: You need:

  - A process for contributing to the test suite, modifying
    a test, approving a test, comments on the test suite.
  - Copyright for the test suite.
  - Need a dated version of a test suite (not the individual
    test cases).
  - You need to explain the tests (how to use the test suite).

 KD: It's usually good to start a test suite earlier in the process.
 HB: I expect that the development of a serious test suite is a very
 challenging job. 
 TL: It's extremely challenging to do test suites. The test of
 testable scenarios is basically limitless.

 IJ: I imagine that some of our checkpoints will be
 format-independent, and those tests will be independent of tests
 that are format-specific (e.g., HTML test files). See the
 MathML test suite, for example:

 KD: Your tests should allow some independence (different software can
 satisfy the checkpoints in different ways). If you show examples,
 make sure that you indicate that there may be other ways to satisfy a

 JG: I think resources (within this WG) are an issue. I'd like to
 focus on the format-specific tests first.

 IJ Summarizing:
  1) Process for building and managing the test suite
  2) Tools for building a test suite

 KD: We don't have any tools for test suites within W3C. MathML uses
 an XML format, transformed by XSLT. DOM WG is using a mailing list
 for input, and using CVS as a repository for the tests.

 KD: First QA meeting is in Bruxelles in November.
     Refer to QA home page: http://www.w3.org/QA/

 KD: The life of each test is important. If you delete a test, need to
 explain why (so that not discussed over and over). Tracking test
 cases is very important.

 KD: Need to distinguish conformance to the test suite from
 conformance to the technical report. Test suites don't cover all

 IJ: At some point, we might create format-specific bindings
 that are formal documents to which one can conform.

 KD: This is like the DOM specification / test suites. DOM is an API,
 the test suites are bindings in a particular programming language.
 That's why the DOM WG has a formal process for building the
 test suite.

 IJ: I sent mail to www-math about how they built their test suite.

 TL: We have internal tools that we use to manage, say, 18,000
 individual test cases. There is some software that distributes the
 tests to the clients, and the results are compared
 automatically. Much of what we do is to verify that changes don't
 introduce problems. We'll run a scenario manually and then code it
 up. When we verify that it's automated, then that case is run every
 couple of days.

 IJ: How much could you automate in our case (e.g., here's a NOFRAMES
 element test file, and you should have access to the following
 content...")? It may be possible to encode our evaluations in such as
 way that N evaluations can be compared.

 TL: I was hoping to have someone from IE who does test suites
 come to the face-to-face meeting. That person should be able
 to attend a future meeting.

 KD: If you have any questions on this in the future, send mail
 to the public mailing list www-qa@w3.org.

 /* KD leaves */

 JG: Here's a model:
  1) Developers can use the test suites to determine what
    they do or need to do.

  2) Evaluations will tell developers what they don't do.
  3) Techniques will tell them some ways to do things.

 MQ: I have no idea on how to start an evaluation. 

 IJ: I agree - I think a "how to do an evaluation" manual would be

 MQ: If we have different expectations when doing an evaluation, we
     will have different results.

 IJ: I agree with both points:

  - "How to do an evaluation" is a good idea.
  - Test suites will provide us with a common baseline.

 IJ: I expect to be working on the test suites and tools. Who wants to
 work on the manual?
 Check out "Geting started: Making a Web site accessible;"

 IJ: For instance, some checkpoints are hard to evaluate unless you
 talk to the developer (or get information from their site). That
 should be called out to reviewers. Other checkpoints are
 format-specific and we should be able to provide a discrete list of
 elements/attributes. A "how to evaluate" document should explain how
 to deal with hard checkpoints, how to deal with documentation (e.g.,
 12.1, which points off to WCAG), etc.

 JA: How should we coordinate with developers? How do we get the right
 developer on the phone, for example?

 JG: We already know people in many organizations. I think that it's
 key to get developer input in these evaluations.

 IJ: I recommend that if you want to do an evaluation, to coordinate
 on our mailing list - someone may know who to talk to, someone may
 have already done an evaluation, etc.

 Action JG: Draw up a "how to do an evaluation" draft.

 IJ will be coming up to speeds on test suites in the background.

 HB: Chris Ridpath and Sean Palmer could provide input about the AERT
 test files.

2.Rescheduling FTF meeting

TL: Microsoft is willing to host the meeting.

JG: Thank you, Tim.

JG: In light of the events in New York, people may be less willing to
travel, and companies may be changing their travel policies. 

MQ: Seattle is great for me (I don't have to fly).

JA: Unrelated to last week, the state has curtailed all out-of-state
travel. The school will give me time to go to the meeting, but won't
purchase the ticket for me. I can't get reimbursed for travel.

HB: I would prefer by telephone.

DP: I am still contemplating the impact of last week on my long
distance travel. I wouldn't rule this out, but want to give it some

TL: Personally, I don't love travel anyway....my Microsoft budget
would make it difficult for me to organize travel.

MQ: What about video conferencing?

IJ: That is something that W3C is looking into for this type of
meeting. Jon, you might want to raise this in the Coordination Group.

IJ: I think there are advantages to meeting face-to-face.

JG: I think there are advantages to meeting at Microsoft, for access
to those developers. We may also be able to draw people from

IJ: We need to make sure to reach out to the people who don't attend
our teleconfs but who signed up for our ftf meeting.

MQ: What about NetMeeting?

DP: I think that we should schedule the meeting. I agree that face
time is very important.

JG: Proposed dates - 29-30 (Monday-Tuesday)
    Or maybe 22-23.

IJ: W3C AC meetings 4-10 November.
TL: The critical part is that I'll need to know how many people. The
tricky part is getting a conference room for two days.

JG: I suspect 10-12 people.

JA: We have videoconf opportunities at the school.

Action JG:

  - Propose dates on the list, and contact all people 
    who attended previous meeting. 

Action IJ:

  - Find out about availability of videoconf equipment.

Action JA:

  - Find out if JA and RS can meet in the same place in Austin.

3.Section 508 verses UAAG and developer implementation issues

Previous report by Ian:

JG: It would be useful to show how UAAG is more comprehensive than
508. We need people to work on a document that explains some of the

JG: We need to improve lobbying for UAAG. We need to be part of the
education process. Do people have organizations they can work with.

Action JA: Work on a comparison document.

JA: I have AFB and ACB connections.  (Curtis Chong)

JG: Can you talk to their leadership about advocacy?

JA: Yes.

/* TL leaves */

DP: My connections are with ACB. They have just reformed their
informationa access advisory council. I'm on that committee. We will
be taking up issues like this. Also, the Section 508 accessibility
forum meeting was canceled last week. We will be looking for standards
that support and enhance section 508. US Standards development needs
to take into account the international context. I will work with the
people at the ACB to try to develop something around UAAG.

DP: Problems I see with 508 is lack of specificity. UAAG is more

MQ: I'm working with the ACB. Also, the special interest group for
computers. I probably don't have the type of connections that you're
thinking of.

JG: But if you're a member, you can send messages to the leadership.

Closed Action Items

1. JG: Create new issues list for candidate recommendation 

2.IJ: Add statement "sounds started on load are implicitly
synchronized with other sounds that are played automatically later"
techniques document
Source: http://lists.w3.org/Archives/Public/w3c-wai-ua/2001JulSep/0220

3.JG: Contact Aaron Smith about a review
Source: http://lists.w3.org/Archives/Public/w3c-wai-ua/2001JulSep/0240

Open Action Items

1.JG: Talk about JG's tool and EARL integration at WAI CG.
  Source: http://lists.w3.org/Archives/Public/w3c-wai-ua/2001JulSep/0188 

2.JG: Review Netscape version 6.X 
  Source: http://lists.w3.org/Archives/Public/w3c-wai-ua/2001JulSep/0191 

3.GR: Contact Dolphin 
  Source: http://lists.w3.org/Archives/Public/w3c-wai-ua/2001JulSep/0188 
4.TL: Review initial implementation report for IE and comment
  Source: http://lists.w3.org/Archives/Public/w3c-wai-ua/2001JulSep/0191 

Ian Jacobs (ij@w3.org)   http://www.w3.org/People/Jacobs
Tel:                     +1 718 260-9447

Received on Thursday, 20 September 2001 15:47:36 UTC