Agenda EvalTF 26 January

Dear Eval TF,

The next teleconference is scheduled for Thursday 26 January 2012 at:
  * 15:00 to 16:00 UTC
  * 16:00 to 17:00 Central European Time (time we use as reference)
  * 10:00 to 11:00 North American Eastern Time (ET)
  * 07:00 to 08:00 North American Pacific Time (PT)
  * 23:00 to 24:00 Western Australia Time

Please check the World Clock Meeting Planner to find out the precise date for your own time zone:
  - <http://www.timeanddate.com/worldclock/meeting.html>

The teleconference information is: (Passcode 3825 - "EVAL")
  * +1.617.761.6200
  * SIP / VoIP -http://www.w3.org/2006/tools/wiki/Zakim-SIP

We also use IRC to support the meeting: (http://irc.w3.org)
  * IRC server: irc.w3.org
  * port: 6665
  * channel: #eval


AGENDA:

#1. Welcome, Scribe…

#2. Specific discussion on section 3.3, 3.4 and 3.8
We need the text to be clearer on what key functionality adds to complete processes. Complete processes are well defined, but not all functionality on a website falls within complete processes.
Some site owners might argue that parts of their website fall outside of the scope of the evaluation like the video’s they point to on their Youtube channel, the electronic banking section of their banksite etc. How can an evaluator decide if this is a valid exception.
Or as happens sometimes, a site owner places a part of the website on another website to avoid accessibility issues. They do however point directly to that website, or even link directly to it from the main or sub menu.

The link to subclause 3.8 is made because it could also be possible to divide the evaluation of a website into multiple evaluations. Each evaluation could have its own scope. The total of the evaluations would cover the complete website. A few questions would be: Does each evaluation lead to a separate report? Is there to be a clear relation between the reports? How do we show that an evaluation is only part of a collection (in the report probably?


#3. Specific discussion on section 5.1 Manual and machine evaluation
We are making a methodology for full evaluation of a website. This is not possible when only using machine evaluation. But it is also not a good idea to do a complete full evaluation manually. If people use tools, they can render very different results. Even one tool can change over time. Do we need to say things about the tools? Set general requirements for them to enhance replicability. This could just be by asking evaluators to state what tools they used, date, version, etc. But we could also ask them to check the tools with a specific control mechanism to be sure that the tools do what they expect.

#4. Evaluation section.
We will start this section next week. If there is time, we can do a short introduction.

Regards,

Eric

Received on Wednesday, 25 January 2012 09:55:53 UTC