Fourth Draft of UAAG Exit Criteria (with WG edits)

Here is the Fourth Draft of the Exit Criteria as revised by UAWG during 
the meeting of 26 February 2015. Still more work to do.  We will talk to 
browser people at CSUN and see if we can figure out the technical 
details of separating out the rendering engine and the browser UI.

------------------------

Exit Criteria
The User Agent Accessibility Guidelines Working Group intends to submit
this document for consideration as a W3C Proposed Recommendation as soon
as the following conditions are met. We expect to complete testing and
show evidence of meeting all exit criteria, and change or remove the
at-risk items as needed, no earlier than @@[date]@@.

1. Define test cases: Update the set of tests needed to cover all UAAG
normative success criteria.

2. Test Implementations: Perform these tests and verify at least two 
independent results on user agents, and to the extent needed, 
combinations of user agents and extensions.

Note on Independent Implementations:
In order to be considered independent, each implementation (e.g. 
rendering engine, user agent, plugin, or extension) of a success 
criterion must be developed by a different party and cannot share, 
reuse, or derive from code used by another qualifying implementation. 
Sections of code that have no bearing on the implementation of the 
success criterion would be exempt from this requirement.

a) Success criteria that are implemented at the rendering engine level 
must be demonstrated by browsers that use different rendering engines.
b) For success criteria that are only implemented at the user interface 
level (e.g. something set by a user), independent results can be 
demonstrated by browsers that share a common rendering engine. Examples 
include:
1.8.3 Provide Viewport Scrollbars
1.8.13 Allow Same User Interface

c) Success criteria can be demonstrated by two independent extensions to 
the same or different user agents, by a combination of an extension and 
a native feature in the same or different user agent, or by native 
features in two different user agents.
Example: A browser has a built-in feature that allows direct navigation 
to links, thus complying with 2.3.1. If an extension provides a 
different, perhaps more feature-rich method of doing the same thing, 
that can count as a second qualifying implementation even if it is for 
the same browser.

NOTE: <Greg> I don't think we have consensus on whether Chrome and Opera 
(or IE and Tencen's) would count as independent implementations for an 
SC that is implemented in their shared rendering engine but adjusted 
through their different UIs.

On 2/19/2015 3:12 PM, Jeanne Spellman wrote:
> Here is the Third Draft of the Exit Criteria as revised by UAWG during
> the meeting of 19 February 2015. It is not complete, and will be
> continued at the 26 February meeting, but I didn't want to chance losing
> the edits we made.
>
> Exit Criteria
> The User Agent Accessibility Guidelines Working Group intends to submit
> this document for consideration as a W3C Proposed Recommendation as soon
> as the following conditions are met. We expect to complete testing and
> show evidence of meeting all exit criteria, and change or remove the
> at-risk items as needed, no earlier than @@[date]@@.
>
> 1. Define test cases: Update the set of tests needed to cover all UAAG
> normative success criteria.
>
> 2. Test Implementations: Perform these tests and verify at least two
> independent results on user agents, and to the extent needed,
> combinations of user agents and extensions.
>
> Note on Independent Implementations:
> The goal is to have independent implementations for each success
> criteria, while taking into account that other software components that
> are not connected to the success criteria being tested may be shared.
>
> a) Success criteria that are implemented at the rendering engine level
> must be demonstrated by browsers that use different rendering engines.
> Examples include:
> 1.1.2 Indicate Unrendered Alternative Content
> 1.8.6 Maintain Point of Regard
>
> b) For success criteria that are only implemented at the user interface
> level (e.g. something set by a user), independent results can be
> demonstrated by browsers that share a common rendering engine. Examples
> include:
> 1.8.3 Provide Viewport Scrollbars
> 1.8.13 Allow Same User Interface
>
> c) Success criteria that are implemented by extensions can be
> demonstrated by two independent extensions to the same user agent.
>
> -----------------------
>
> d) Implementations features that satisfy a specific success criteria
> (plugins, extensions or user agents) must be from  different code bases
> in order to be considered independent.
>
>
>
>
>
> On 2/6/2015 6:43 PM, Jeanne Spellman wrote:
>> UAWG WG,
>>
>> Since getting Exit Criteria approved is a long process, I'm getting it
>> started.
>>
>> Exit Criteria are used by the W3C Director (or designate) to show that
>> the spec deserves to advance to the next stage of the W3C process. It is
>> used to show that the spec works in the real world by showing that each
>> feature is implemented in 2 independent products. Good exit criteria
>> tell a convincing story that the spec is mature, complete and widely
>> implemented.
>>
>> I expect that a tricky issue for us will be determining what are
>> independent products when many products share the same rendering engine.
>>
>> Here are some sample Exit Criteria:
>>
>> ATAG: http://www.w3.org/TR/2013/CR-ATAG20-20131107/#exit
>> WCAG: http://www.w3.org/TR/2008/CR-WCAG20-20080430/#status_exit
>> ARIA: http://www.w3.org/TR/2011/CR-wai-aria-20110118/#sotd_exit
>>
>> Note that all of these include Features At Risk, so we need to finish up
>> identifying all the implementations we have of each success criteria.
>>
>> Here's a proposal for a first draft of the Exit Criteria for UAAG 2.0:
>>
>> ----------------------------------------
>> Exit Criteria
>> The User Agent Accessibility Guidelines Working Group intends to submit
>> this document for consideration as a W3C Proposed Recommendation as soon
>> as the following conditions are met. We expect to complete testing and
>> show evidence of meeting all exit criteria, and change or remove the
>> at-risk items as needed, no earlier than @@[date]@@.
>>
>> 1. Define test cases: Identify a set of tests needed to cover all UAAG
>> normative success criteria.
>>
>> 2. Test Implementations: Perform these tests on multiple separate
>> combinations of users agents and/or user agents and extensions.
>>
>> 3. Verify interoperable results: Find at least two independent
>> implementations of each normative success criterion.
>>
>> Note on Independent Implementations:
>> 1) Success criteria that can only be met using the rendering engine of
>> the user agent will need implementations of two different rendering
>> engines.
>>
>> 2) Success criteria that can only be met with the user agent user
>> interface may have implementations using the same rendering engine.
>>
>> 2) Success criteria that can be met with an extension to the user agent
>> may be met with two different extensions on the same base user agent.
>>
>> 3) Implementations (plugins, extensions or user agents) must be from
>> different code bases in order to be considered independent.
>>
>>
>

-- 
_______________________________
Jeanne Spellman
W3C Web Accessibility Initiative
jeanne@w3.org

Received on Thursday, 26 February 2015 19:33:08 UTC