- From: Ian B. Jacobs <ij@w3.org>
- Date: Thu, 20 Sep 2001 15:47:05 -0400
- To: w3c-wai-ua@w3.org
20 September 2001 UAWG teleconference
Agenda announcement:
http://lists.w3.org/Archives/Public/w3c-wai-ua/2001JulSep/0270
Participants: Jon Gunderson (Chair), Ian Jacobs (Scribe), David
Poehlman, Mickey Quenzer, Harvey Bingham, Karl DuBost, Jim Allan,
Tim Lacy
Regrets: Al Gilman, Eric Hansen, Loretta Reid
Absent: Denis Anson, Gregory Rosmaita, Rich Schwerdtfeger
Previous meeting: 6 Sep 2001
http://lists.w3.org/Archives/Public/w3c-wai-ua/2001JulSep/0240
Next meeting: 27 September
Reference document 12 September Candidate Recommendation:
http://www.w3.org/TR/2001/CR-UAAG10-20010912/
====================================================================
-------------
1.Test Suites
-------------
/* Karl Dubost joins the meeting. Karl is the W3C Conformance
Manager */
KD: W3C just started the Quality Assurance (QA) Activity. We are not
yet organized, and will be developing guidelines.
KD: First of all, let me say that your conformance section is very
good, and I appreciate the way that you have modularized conformance.
KD: I think it's very hard to do a full/complete test suite. If you
want to organize your work, you should first find the testable
assertions in the Recommendation. Build test suites from these.
For example, see the SVG test suite guidelines written by Lofton
Henderson.
http://www.w3.org/Graphics/SVG/Test/svgTest-manual.htm
KD: You need:
- A process for contributing to the test suite, modifying
a test, approving a test, comments on the test suite.
- Copyright for the test suite.
- Need a dated version of a test suite (not the individual
test cases).
- You need to explain the tests (how to use the test suite).
KD: It's usually good to start a test suite earlier in the process.
HB: I expect that the development of a serious test suite is a very
challenging job.
TL: It's extremely challenging to do test suites. The test of
testable scenarios is basically limitless.
IJ: I imagine that some of our checkpoints will be
format-independent, and those tests will be independent of tests
that are format-specific (e.g., HTML test files). See the
MathML test suite, for example:
http://www.w3.org/Math/testsuite
KD: Your tests should allow some independence (different software can
satisfy the checkpoints in different ways). If you show examples,
make sure that you indicate that there may be other ways to satisfy a
checkpoint.
JG: I think resources (within this WG) are an issue. I'd like to
focus on the format-specific tests first.
IJ Summarizing:
1) Process for building and managing the test suite
2) Tools for building a test suite
KD: We don't have any tools for test suites within W3C. MathML uses
an XML format, transformed by XSLT. DOM WG is using a mailing list
for input, and using CVS as a repository for the tests.
KD: First QA meeting is in Bruxelles in November.
Refer to QA home page: http://www.w3.org/QA/
KD: The life of each test is important. If you delete a test, need to
explain why (so that not discussed over and over). Tracking test
cases is very important.
KD: Need to distinguish conformance to the test suite from
conformance to the technical report. Test suites don't cover all
cases.
IJ: At some point, we might create format-specific bindings
that are formal documents to which one can conform.
KD: This is like the DOM specification / test suites. DOM is an API,
the test suites are bindings in a particular programming language.
That's why the DOM WG has a formal process for building the
test suite.
http://www.w3.org/DOM/DOMTS-Process
IJ: I sent mail to www-math about how they built their test suite.
TL: We have internal tools that we use to manage, say, 18,000
individual test cases. There is some software that distributes the
tests to the clients, and the results are compared
automatically. Much of what we do is to verify that changes don't
introduce problems. We'll run a scenario manually and then code it
up. When we verify that it's automated, then that case is run every
couple of days.
IJ: How much could you automate in our case (e.g., here's a NOFRAMES
element test file, and you should have access to the following
content...")? It may be possible to encode our evaluations in such as
way that N evaluations can be compared.
TL: I was hoping to have someone from IE who does test suites
come to the face-to-face meeting. That person should be able
to attend a future meeting.
KD: If you have any questions on this in the future, send mail
to the public mailing list www-qa@w3.org.
/* KD leaves */
JG: Here's a model:
1) Developers can use the test suites to determine what
they do or need to do.
2) Evaluations will tell developers what they don't do.
3) Techniques will tell them some ways to do things.
MQ: I have no idea on how to start an evaluation.
IJ: I agree - I think a "how to do an evaluation" manual would be
useful.
MQ: If we have different expectations when doing an evaluation, we
will have different results.
IJ: I agree with both points:
- "How to do an evaluation" is a good idea.
- Test suites will provide us with a common baseline.
IJ: I expect to be working on the test suites and tools. Who wants to
work on the manual?
Check out "Geting started: Making a Web site accessible;"
http://www.w3.org/WAI/gettingstarted
IJ: For instance, some checkpoints are hard to evaluate unless you
talk to the developer (or get information from their site). That
should be called out to reviewers. Other checkpoints are
format-specific and we should be able to provide a discrete list of
elements/attributes. A "how to evaluate" document should explain how
to deal with hard checkpoints, how to deal with documentation (e.g.,
12.1, which points off to WCAG), etc.
JA: How should we coordinate with developers? How do we get the right
developer on the phone, for example?
JG: We already know people in many organizations. I think that it's
key to get developer input in these evaluations.
IJ: I recommend that if you want to do an evaluation, to coordinate
on our mailing list - someone may know who to talk to, someone may
have already done an evaluation, etc.
Action JG: Draw up a "how to do an evaluation" draft.
IJ will be coming up to speeds on test suites in the background.
HB: Chris Ridpath and Sean Palmer could provide input about the AERT
test files.
--------------------------
2.Rescheduling FTF meeting
--------------------------
TL: Microsoft is willing to host the meeting.
JG: Thank you, Tim.
JG: In light of the events in New York, people may be less willing to
travel, and companies may be changing their travel policies.
MQ: Seattle is great for me (I don't have to fly).
JA: Unrelated to last week, the state has curtailed all out-of-state
travel. The school will give me time to go to the meeting, but won't
purchase the ticket for me. I can't get reimbursed for travel.
HB: I would prefer by telephone.
DP: I am still contemplating the impact of last week on my long
distance travel. I wouldn't rule this out, but want to give it some
thought.
TL: Personally, I don't love travel anyway....my Microsoft budget
would make it difficult for me to organize travel.
MQ: What about video conferencing?
IJ: That is something that W3C is looking into for this type of
meeting. Jon, you might want to raise this in the Coordination Group.
IJ: I think there are advantages to meeting face-to-face.
JG: I think there are advantages to meeting at Microsoft, for access
to those developers. We may also be able to draw people from
RealNetworks.
IJ: We need to make sure to reach out to the people who don't attend
our teleconfs but who signed up for our ftf meeting.
MQ: What about NetMeeting?
DP: I think that we should schedule the meeting. I agree that face
time is very important.
JG: Proposed dates - 29-30 (Monday-Tuesday)
Or maybe 22-23.
IJ: W3C AC meetings 4-10 November.
TL: The critical part is that I'll need to know how many people. The
tricky part is getting a conference room for two days.
JG: I suspect 10-12 people.
JA: We have videoconf opportunities at the school.
Action JG:
- Propose dates on the list, and contact all people
who attended previous meeting.
Action IJ:
- Find out about availability of videoconf equipment.
Action JA:
- Find out if JA and RS can meet in the same place in Austin.
--------------------------------------------------------------
3.Section 508 verses UAAG and developer implementation issues
--------------------------------------------------------------
Previous report by Ian:
http://lists.w3.org/Archives/Public/w3c-wai-ua/2001JanMar/0561
JG: It would be useful to show how UAAG is more comprehensive than
508. We need people to work on a document that explains some of the
differences.
JG: We need to improve lobbying for UAAG. We need to be part of the
education process. Do people have organizations they can work with.
Action JA: Work on a comparison document.
JA: I have AFB and ACB connections. (Curtis Chong)
JG: Can you talk to their leadership about advocacy?
JA: Yes.
/* TL leaves */
DP: My connections are with ACB. They have just reformed their
informationa access advisory council. I'm on that committee. We will
be taking up issues like this. Also, the Section 508 accessibility
forum meeting was canceled last week. We will be looking for standards
that support and enhance section 508. US Standards development needs
to take into account the international context. I will work with the
people at the ACB to try to develop something around UAAG.
DP: Problems I see with 508 is lack of specificity. UAAG is more
specific.
MQ: I'm working with the ACB. Also, the special interest group for
computers. I probably don't have the type of connections that you're
thinking of.
JG: But if you're a member, you can send messages to the leadership.
---------------------
Closed Action Items
---------------------
1. JG: Create new issues list for candidate recommendation
2.IJ: Add statement "sounds started on load are implicitly
synchronized with other sounds that are played automatically later"
techniques document
Source: http://lists.w3.org/Archives/Public/w3c-wai-ua/2001JulSep/0220
3.JG: Contact Aaron Smith about a review
Source: http://lists.w3.org/Archives/Public/w3c-wai-ua/2001JulSep/0240
-----------------
Open Action Items
-----------------
1.JG: Talk about JG's tool and EARL integration at WAI CG.
Source: http://lists.w3.org/Archives/Public/w3c-wai-ua/2001JulSep/0188
2.JG: Review Netscape version 6.X
Source: http://lists.w3.org/Archives/Public/w3c-wai-ua/2001JulSep/0191
3.GR: Contact Dolphin
Source: http://lists.w3.org/Archives/Public/w3c-wai-ua/2001JulSep/0188
4.TL: Review initial implementation report for IE and comment
Source: http://lists.w3.org/Archives/Public/w3c-wai-ua/2001JulSep/0191
--
Ian Jacobs (ij@w3.org) http://www.w3.org/People/Jacobs
Tel: +1 718 260-9447
Received on Thursday, 20 September 2001 15:47:36 UTC