Re: Agenda EvalTF 26 January

Dear Eric, All, 

Is there a new version of the methodology document available - or is the one from the 19th the latest?  It would be great to update it regularly, preferably a day before the telecon, so that we can make comments about it.  Especially necessary when agreement has been made in a telecon / email list on content which should appear in the document.

I only ask as I have noticed that concepts agreed in our emails / telecons have not as yet, at least to me, made there way into the methodology document... 

All the best 

Alistair

On 25 Jan 2012, at 10:55, Velleman, Eric wrote:

> Dear Eval TF,
> 
> The next teleconference is scheduled for Thursday 26 January 2012 at:
>  * 15:00 to 16:00 UTC
>  * 16:00 to 17:00 Central European Time (time we use as reference)
>  * 10:00 to 11:00 North American Eastern Time (ET)
>  * 07:00 to 08:00 North American Pacific Time (PT)
>  * 23:00 to 24:00 Western Australia Time
> 
> Please check the World Clock Meeting Planner to find out the precise date for your own time zone:
>  - <http://www.timeanddate.com/worldclock/meeting.html>
> 
> The teleconference information is: (Passcode 3825 - "EVAL")
>  * +1.617.761.6200
>  * SIP / VoIP -http://www.w3.org/2006/tools/wiki/Zakim-SIP
> 
> We also use IRC to support the meeting: (http://irc.w3.org)
>  * IRC server: irc.w3.org
>  * port: 6665
>  * channel: #eval
> 
> 
> AGENDA:
> 
> #1. Welcome, Scribe…
> 
> #2. Specific discussion on section 3.3, 3.4 and 3.8
> We need the text to be clearer on what key functionality adds to complete processes. Complete processes are well defined, but not all functionality on a website falls within complete processes.
> Some site owners might argue that parts of their website fall outside of the scope of the evaluation like the video’s they point to on their Youtube channel, the electronic banking section of their banksite etc. How can an evaluator decide if this is a valid exception.
> Or as happens sometimes, a site owner places a part of the website on another website to avoid accessibility issues. They do however point directly to that website, or even link directly to it from the main or sub menu.
> 
> The link to subclause 3.8 is made because it could also be possible to divide the evaluation of a website into multiple evaluations. Each evaluation could have its own scope. The total of the evaluations would cover the complete website. A few questions would be: Does each evaluation lead to a separate report? Is there to be a clear relation between the reports? How do we show that an evaluation is only part of a collection (in the report probably?
> 
> 
> #3. Specific discussion on section 5.1 Manual and machine evaluation
> We are making a methodology for full evaluation of a website. This is not possible when only using machine evaluation. But it is also not a good idea to do a complete full evaluation manually. If people use tools, they can render very different results. Even one tool can change over time. Do we need to say things about the tools? Set general requirements for them to enhance replicability. This could just be by asking evaluators to state what tools they used, date, version, etc. But we could also ask them to check the tools with a specific control mechanism to be sure that the tools do what they expect.
> 
> #4. Evaluation section.
> We will start this section next week. If there is time, we can do a short introduction.
> 
> Regards,
> 
> Eric
> 
>

Received on Thursday, 26 January 2012 08:33:26 UTC