Issues and comments arising from UA evaluations

Dear UAWG,

A number of issues about the UAAG 1.0 Candidate Recommendation
[1] have come up during recent reviews with developers. Most
of these issues only require clarification. However, there are a
couple of proposals for more substantial changes below that
I'd like to discuss at an upcoming teleconference.

Jon, I don't suggest that we add these to the issues list until
the UAWG has had a chance to discuss them.

Comments welcome,

  - Ian

[1] http://www.w3.org/TR/2001/CR-UAAG10-20010912/

=============
 From the Mozilla review:

  1) Can checkpoints 6.1 and 6.2 be satisfied if the
     APIs are not available out-of-process?

     Proposed: Yes.

=============
 From the (incomplete) RealNetworks review:

  2) Add a section to the document explaining how other
     specifications can create profiles of UAAG 1.0
     conformance. For instance, if the FOO specification wants to
     require FOO user agents to conform to UAAG 1.0 in a
     particular manner, what should/must the FOO spec say?

     Proposed: I would like to write this up and send to the
     UAWG in the near future.

  3) UAAG 1.0 needs to be clearer about the fact that a user
     agent is not required to satisfy all requirements for
     all implemented formats. I believe the document already
     says this, but it's not easy to find.

  4) Make clearer up front (e.g., at the beginning of section
     2, not just the conformance section) that a user agent
     can conform to fewer than all the checkpoints.

=============
 From the Opera review:

  5) Checkpoint 2.10 needs clarification. Current text:

     "Once the user has viewed the original author-supplied
     content associated with a placeholder, allow the user to turn
     off the rendering of the author-supplied content."

     Proposed:

     "If the user agent has satisfied checkpoint 2.3 by using
      a placeholder, then allow the user to toggle rendering
      between the placeholder and the original author-supplied
      content."

  6) Checkpoint 3.1: Clarify that if the user agent allows the
     user to turn off all images, including background images,
     then this checkpoint is satisfied, but this is not
     the optimal solution (since better to be able to turn
     of background images separately).

  7) Clarify for 1.2, 9.5, 9.6 that checkpoint may be
     satisfied on a per event type basis; additional
     granularity (per handler) is not required. This
     clarification is consistent with our recent
     DOM discussions.

  8) Checkpoint 6.3: The title is "Programmatic
     access to non-HTML/XML content", but the first
     requirement refers to "markup language" (which would
     exclude, for example, images). Do we intend "all content"
     here, or just markup (non-HTML/XML) markup languages?

     I don't have a proposal for this one.

  9) [Editorial] Clarify in 3.6 why P2 and 3.5 is P1.
     This is already in the techniques doc, so not critical.

10) Checkpoints 6.2 and 6.3 (DOM write access and
     programmatic access to other types of content):
     there is some content where it's a security issue
     to provide *any* kind of programmatic access. [Tantek
     made similar comments during the Mac IE review.]

     Some ideas for dealing with this:

     a) Provide programmatic access, but when security
     is an issue, prompt the user for confirmation
     (on configuration).

     b) If the user agent has a mechanism for allowing trusted
     programmatic access, then ATs should use this mechanism,
     and the UA must make available all content to these
     trusted agents.

11) Checkpoints 6.1/6.2: Is it a P1 to implement some API
     and P2 to implement the DOM 2 Core? See my additional
     thoughts below.

12) Checkpoints 6.1-6.4: Perhaps it's better (though not
    necessarily simpler) to express these checkpoints as
    functional requirements. It may turn out that the
    functional requirements are met by implementing
    some piece of an API.

=============
 From the MAC IE review (not yet completed)

13) Checkpoint 2.2: "XML and SGML applications". According to Ian
   Hickson and Tantek Çelik (both there for the evaluation), one
   cannot reliably determine what's an xml and sgml application by
   sniffing content. We will need to be more precise about how one
   would determine that something is an xml or sgml application.

    I don't have a proposal for this one.

14) Checkpoint 3.4: Toggle scripts. Tantek argued that,
    since most pages include scripts, the following
    provision should not be P1:

       "In this configuration, provide an option to alert the
       user when executable content is available (but has not
       been executed)."

    The alert will be so frequent as to be ignored. One
    suggestion is to pull out this provision and make it a
    lower priority.

    Proposal: Make the alert a P3 requirement.

15) Checkpoint 6.2: Tantek stated that there are security
    issues for some types of programmatic access to content.
    (for instance, programmatic access to HTML's INPUT element,
    type="file", which might allow the author to upload a file
    from the user's computer without the user's permission).

    See my comments below on 6.1, 6.2, and 6.3.

16) Checkpoint 6.9: Timely access. The checkpoint reads:
     "Ensure that programmatic exchanges proceed in a timely
     manner." It's not clear which exchanges this checkpoint
     refers to - internal to the user agent? between the UA
     and ATs?

     Proposal: This refers to exchanges over the APIs required
     elsewhere in Guideline 6.

17) Checkpoint 7.4: Input configuration indications.

     Proposal: Clarify that the requirement is to "Follow
     operating environment *user interface* conventions to
     indicaxte the input configuration."

18) Checkpoints 7.3, 8.1:

     Proposal: Clarify that the expression "identified as such"
     means "identified as such in the specification". However,
     this does not account for W3C Notes such as "Accessibility
     features of CSS" which may be useful but not normative.

19) Checkpoint 9.3: Move content focus.

     Proposal: In point three, clarify that "each element" means
     each element in the set defined in point 1. Tantek
     recommended introducing the term "Focusable element" and
     using it in point 1.

20) Checkpoint 9.4: Restore history. The checkpoint reads:

      "For user agents that implement a viewport history
      mechanism, for each state in a viewport's browsing history,
      maintain information about the point of regard, content
      focus, and selection."

     However, even when a user agent implements a history
     mechanism, it may not keep all pages in its cache.

     Proposal: Do not require the user agent to restore state
     variables for pages removed from the history cache.

21) Checkpoint 9.7: The checkpoint title is "Move content focus
    optimally". The word "optimal" is a bit strong.

    Proposal: Change the title to "Move content focus reverse"
    since the primary (but not only) difference from 9.3 is the
    reverse requirement.

22) Checkpoints 10.2, 10.3, 10.4, 10.7: The expression "rely on
    color alone" is ambiguous. In the end, on a color display, all
    distinguishing marks rely on color (e.g., a solid line is a line
    of a different color than, say, the background color;
    three-dimensional effects are due to shading, etc.).

    Proposal: Given our checkpoint 4.3 (configure text colors), I
    propose that for the 10.x checkpoints, we modify the language
    to read "rely on text foreground and background color alone".

=============
Other observations and points for discussion:

Thought 1) I continue to wonder whether 6.1 and 6.2 should be
requiring DOM implementation at a P1 level.

  Pros:
   - Supposed to promote interoperability by requiring
     a known API.
   - An API is supposed to help developers by providing
     access to incremental changes in content; a simple
     text dump after any changes may decrease performance
     by requiring the AT to reparse each time (though in
     practice, apparently some do this anyway).

  Cons/Questions:
   - AT developers may not, in practice, be interested in
     implementing the DOM, even though in the past they have
     expressed interest.
   - Developers can't implement more suitable APIs for a given
     content type (e.g., SVG) since must implement DOM 2 Core,
     even though a specific SVG DOM might exist.
   - It is not guaranteed that the *entire* DOM 2 Core
     is required to to provide ATs with sufficient access.
     I don't look forward to requiring some "subset" of DOM 2
     Level Core, but it would be worthwhile to determine whether
     nearly all of DOM 2 is required for communication with
     ATs

  Some ideas for other changes based on discussions:

   - Change 6.1, 6.2, and 6.3 to resemble 6.4.
     For example, a new checkpoint replacing 6.1, 6.2, and 6.3
     might be:

    <NEW 6.x>
     Programmatic access to content (P1)
     1. Provide programmatic read access to content.
     2. If the user can modify content through the user
        interface, provide the same functionality
        programmatically.
     3. To satisfy these requirements, implement at least
        one API that is either
	* defined by a W3C Recommendation,
	* a publicly documented API designed to enable
	  interoperability with assistive technologies.
     4. If no such API is available, or if available APIs do
        not enable the user agent to satisfy the requirements,
        implement at least one publicly documented API that
        allows programmatic operation of all of the
        functionalities that are available through the user
        agent user interface, and follow operating environment
        conventions for the use of input and output APIs.
     5. An API is considered available if the specification of
        the API is published (e.g., as a W3C Recommendation) in
        time for integration into a user agent's development
        cycle.

      Note: We encourage developers to satisfy this checkpoint
      by using the DOM Level 2 Core specification for read
      and write access to XML and HTML content, or a more
      specific DOM (e.g., for HTML, SVG, or SMIL) when one exists.
    </NEW 6.x>


Thought 2) Change the priorities of some requirements related to
default values.  A long time ago, Greg Lowney suggested that we
should not have priority 1 checkpoints about user agent default
styles, but rather priority 1 checkpoints that the user be able
to change values, and perhaps priority 2 checkpoints about what
the defaults should be. I continue to think that this would be a
good change to make, in particular after the evaluations I have
done. The following checkpoints relate to default styles:

   7.2 Respect input configuration conventions. (P1)

    1. Ensure that default input configurations of the user agent
    do not interfere with operating environment accessibility
    conventions (e.g., for keyboard accessibility).

   10.3 Distinct default highlight styles. (P1)

    1. Ensure that all of the default highlight styles for the
    selection and content focus, as well as for enabled elements,
    recently visited links, and fee links in rendered content
    do not rely on color alone, and differ from each other, and
    not by color alone.

   10.7 Highlight current viewport. (P1)

    2. For graphical viewports, the default highlight mechanism
    must not rely on color alone.

Proposals:

  - Checkpoint 11.3 is a P2 ("Allow the user to override any
  binding that is part of the user agent default input
  configuration.") I propose making this checkpoint a P1 and
  making 7.2.1 a P2.

  - Since checkpoints 10.2 and 10.4 are P1, require the UA to
   allow the user to configure styles, and cover all the pieces
   mentioned in 10.3, make 10.3.1 a priority 2 checkpoint. I realize
   that without 10.3 at a P1 level, the user might not be able
   to distinguish some highlight styles "out of the box", but I
   think that should be relatively easy to fix thanks to 10.2 and
   10.4

  - I propose simply lowering the priority of 10.7.2 to P2. I note
   that we have no requirement that the user be able to configure
   the style of the viewport highlight mechanism. I don't propose
   that we add such a requirement.

Note: In the above, "10.3.1" refers to checkpoint 10.3,
requirement number 1.

Note: There are other checkpoints (7.2, about default keyboard
configurations. I don't propose that we change their priorities.

Thought 3) Clarify the relation between UAAG 1.0 and other
specifications. UAAG 1.0 includes requirements for specific
features, but also asks for conformance to other specifications
(e.g., HTML, CSS, etc.). I would like to add something to the
conformance section that clarifies that:

  a) Developers should follow specifications, especially when
     consistent with UAAG 1.0 requirements.

  b) UAAG 1.0 may ask developers to implement optional features,
     i.e., go beyond what is strictly required for conformance
     to another specification.

  c) UAAG 1.0 may ask developers to implement features that
     are not part of another specification.

  d) UAAG 1.0 does not make a definitive pronouncement about what
     to do when a UAAG 1.0 requirement is in direct conflict with
     the requirement of another specification (something we hope
     will be rare, and even rarer for W3C specs thanks to the
     good work of the WAI PF Working Group). I think we should
     state explicitly that in these cases, UAAG 1.0 would
     prefer that developers meet the needs of people with
     disabilities.

=============
Some comments and questions about evaluations (perhaps should
be discussed and added to "How to Evaluate a User Agent" [2])

  - I think that each requirement of each checkpoint should be
    rated separately, rather than, say, trying to average 4
    ratings for the 4 requirements of a single checkpoint.

  - Should known bugs that are expected to be fixed count
    against a given rating? I think that bugs should not
    count against the rating, provided there is a good faith
    commitment by the developer to fix them. Furthermore, they
    should be advertised in the claim, not hidden.

  - How can we calibrate evaluations, i.e., make them more
    comparable? How do we ensure even rating across evaluations?

[2] http://www.w3.org/WAI/UA/2001/10/eval

-- 
Ian Jacobs (ij@w3.org)   http://www.w3.org/People/Jacobs
Tel:                     +1 718 260-9447

Received on Tuesday, 12 March 2002 15:26:23 UTC