Re: Draft ERT WG Charter for review and comment

Hello to everybody.
I'd like to contribute with my view of what AERT might be focussing
on. This is more an open discussion rather than specific points to be
made to the charter proposal. My apologies for this generality.

I think we need to establish some sort of consensus on a long-term
vision of what the group is about, and then define some of the steps
(and hence deliverables) that should be produced in the short-term to
achieve progress along those lines.

As for the long-term vision, what is the service that AERT WG is going
to offer and to whom?

EARL is not directly an answer, as it is a means to achieve something:
better interoperability between tools, so that end-users, tool
manufacturers, possible purchasers, consumer associations, etc could
be able to compare tools and could be able to compare/integrate
different evaluations on certain websites. To achieve this we
need tools that can incorporate EARL report, process them, and produce
interesting and useful results, not simply tools that can only generate 
EARL reports.

I think the AERT WG should keep constantly under the radar how EARL is
or might be used, and do what can be done to promote its usage. One
idea that was discussed sometime ago was to develop a service that
would be able to upload EARL reports produced by several tools and
offer to the user a sort of integration and comparison of results. The
service could then be used by the public just like the HTML or CSS 
validators are used now. I would consider this a great success of AERT WG.

Secondly, I would not like AERT WG to focus on technical aspects of
what techniques AER tools should/could employ to evaluate
websites. Let the developers do this.

Why not concentrating on understanding how to measure the
effectiveness, usability, quality of use, accessibility of tools?
Development of an appropriate methodology could lead to (i) valid and
useful comparisons between tools, (ii) and corresponding labeling (in
terms of strength and weaknesses) of evaluations performed by the
tools (perhaps evaluations could also be compared if available in
EARL).

The service AERT WG would provide in this case is again towards
end-users, purchasers, tools manufacturers, etc in terms of a
well-studied and robust tool evaluation methodology, that would be
able to appropiately classify tools and results produced by tools.

If appropriate, AERT WG could also run evaluations of tools according
to such a methodology, and "sell" these results.

A first draft of an example of methodology is described in
http://www.dimi.uniud.it/giorgio/publications.html#evalmeth

Best regards to all of you

         Giorgio Brajnik
______________________________________________________________________
Dip. di Matematica e Informatica   | voice: +39 (0432) 55.8445
Università di Udine                | fax:   +39 (0432) 55.8499
Via delle Scienze, 206             | email: giorgio(at)dimi.uniud.it
Loc. Rizzi -- 33100 Udine -- ITALY | http://www.dimi.uniud.it/giorgio

and UsableNet Inc. http://www.usablenet.com

Received on Thursday, 10 June 2004 06:06:24 UTC