RE: [for review] updated draft AERT

Dear all,

Below you may find my comments to the latest editor's draft of AERT:

In general, in my opinion, section 2 now starts looking quite mature, so I
moved to reflecting on section 3. Indeed, looking into tool examples of
section 3 proved quite useful (these profiles are our "personas" of tools),
as it made me realize other things that may be relevant for section 2, for
which I am providing comments here as well. 

Section 2
-----------

2.4.1 Workflow integration. Plug-ins for Integrated Development Environments
(IDEs) or for Content Management Systems (CMS). 
Suggested change: split into two list items, one for IDEs and the other for
CMS.
Rationale: integration mechanisms and steps in the workflow are different.
For instance, API tools integrated in CMSs evaluate hosted templates.

2.4.5 Tool accessibility
Suggested change: add a mention to the accessibility of reports (e.g.
compliance with WCAG).
Rationale: this point currently considers only accessibility of GUI.

2.2.2 Automatic, semiautomatic and manual testing
Suggested change: change title to "test mode" or "test modes: automatic,
semiautomatic and manual".
Rationale: identify the feature with an easier title, which can also be used
as a header in Table 1.

2.3 Reporting and monitoring
Suggested change: add new report types: 
- Superimposed (in-content). I think this was approved in the latest
conference. I have just checked, but no conclusion was recorded in the
minutes. 
Rationale: There are tools that provide these reports without being
integrated as browser plug-ins (e.g. HERA, TAW... just flying the flag high
:-) ), so that would be a different feature.
- Natural language reports. Not sure if this would be the correct title, but
I think it expresses the concept I want to convey.
Rationale: when considering tool profiles, there was no feature that matched
something like "the tool outputs an HTML  or a PDF report". 

Section 3
------------

3.4 Overview (Table 1)
Suggested change: Table contents would not merely consist of yes/no, but
provide more specific contents.  Further details can be seen in the modified
table appearing in the attachment (columns for Tools A and B).
Rationale: I would expect that a tool profile describes up to which extent
each feature is supported. For instance, which authentication mechanisms,
reporting formats, natural languages, etc. are supported (not just whether
some are supported). I would also expect that some features depend on the
support by external software the evaluation tool integrates within (e.g. a
browser in the case of a plug-in). 

3.3 Tool C: Accessibility evaluation tool for mobile applications
For me it is not clear what the tool does. I guess the tool emulates user
agents on different mobile platforms: it both mimics the user interface
(screen size, resolution, user controls, rendering engine), and it
implements device APIs (e.g. HTML Media Capture, Geolocation) although it
just provides "mock", hardwired implementations of these, which do not
return real values. Then it allows instrumenting the evaluation process
through WebDriver API. However, all that I needed to imagine from context
and experience. I understand the description is currently the shortest of
the three tools, so maybe it could be expanded.
In addition, I am still missing a more specific mention to the testing
functionalities provided.

Editorial changes
---------------------
2.4.1. Workflow integration. Plug-ins for Integrated Development
Environments (IDEs) or for Content Management Systems (CMS).
Suggested change: clarify the last sentence.
It is not clear enough whether we are talking about the tool API or the IDE
API, likewise, it is not clear whether the tool accesses or provides a REST
API.

3.3 Tool C: Accessibility evaluation tool for mobile applications
Suggested change: add a mention and link to table 1.
Rationale: consistency with the other tool profiles.

3.4 Overview
Suggested change: either change the title to "Tabular overview" or change
the corresponding entry in the table of contents.
Rationale: consistency between ToC and headings.


-----Mensaje original-----
De: Shadi Abou-Zahra [mailto:shadi@w3.org] 
Enviado el: martes, 01 de abril de 2014 22:29
Para: ERT WG
Asunto: [for review] updated draft AERT

Dear Group,

The latest draft of AERT (working title) for review is here:
  - http://www.w3.org/WAI/ER/WD-AERT/ED-AERT

Please send comments for discussion to the list!

Note: it's dated April 1st but it's not a joke... ;)

Regards,
   Shadi

--
Shadi Abou-Zahra - http://www.w3.org/People/shadi/ Activity Lead, W3C/WAI
International Program Office Evaluation and Repair Tools Working Group (ERT
WG) Research and Development Working Group (RDWG)