Priorities, Impacts, Etc.

Date: 10 December 1998, 18:12 hrs
To: List Members, Page Authoring Working Group, W3C/WAI
From: Eric Hansen, ETS
Re: Priorities, Impacts, Etc.

Item-0 (EH: 10Dec98). Introduction.

This memo focuses (a) on "priorities," (b) on another ratings I call 
"impacts," and (c) on a few related matters.

In brief, I suggest two major points.

First, I suggest the development of a new rating called "impact" (or 
something similar) which estimates the negative impact of violations of 
individual guidelines upon groups, with special emphasis on disability 
groups (blind, low-vision, deaf, deaf-blind, etc.). These impacts are 
closely tied to the stable nature of the disabilities themselves and, as 
such, more stable than "priorities." Impact rating summaries (or even the 
main ratings) might be stable enough to be part of the normative material. 
(I think that these "impacts" are close to what was intended but not 
implemented in the priorities attached to guidelines.)

Second, I suggest a clarification of "priorities," which I think should be 
global estimates of importance of implementing specific techniques. These 
priorities should take into account many issues (impact of violations 
[above], effectiveness of techniques, practicality, and cost-effectiveness) 
for both disabled and nondisabled users operating in a wide variety of 
situations. Priorities, as at present, are tied to techniques, and are 
non-normative. (This is a clarification rather than a major restructuring 
of the current idea of priorities that are attached to techniques.)
 
Item-1 (EH: 10Dec98). Clarification of the Priority Ratings

I think that there needs to be clarification of the nature of the 
priorities that are attached to the guidelines and to the techniques. 

I believe that there is currently a mismatch between what the document says 
about these priorities and the way in which they are actually derived. I 
think that mismatch leads to some confusion about the meaning of priorities 
and whether they should be included in the normative part of the guideline 
document.

The 7 December version of the Page Authoring Guidelines states that:

"For the guidelines, the priority refers to the importance of addressing 
the issue identified by the guideline. For techniques, the priority refers 
to which technique best improves accessibility with respect to this 
guideline. For example, a Priority 1 guideline may have a Priority 1 
technique, which must be done to provide accessibility, and a Priority 3 
technique, which may also be done to help address the issue."

If I understand correctly, the foregoing distinction between technique 
priorities and guideline priorities is not accurate.  I believe that 
currently, the guideline priorities are derived directly from the technique 
priorities. 

I think that the first step in solving this problem is to explain what is 
actually done. For this, I suggest the following.

"Priorities are estimates of the importance of adhering to the techniques 
that implement the guidelines. Each technique has a priority rating, which 
indicates the level of importance of the technique for addressing the 
accessibility and usability problems caused by violations of the guideline. 

The working group that produced these priorities intend that they cover Web 
use by individuals with and without disabilities, for a variety of content 
purposes (informational, educational, commercial, entertainment), using a 
variety of technologies (speech output, regular browsers, braille, etc., 
etc.), in a variety of situations (hands-free, noise-free, etc.) and 
cultures (languages, etc.) [This list needs to be refined.]. Over time, it 
is hoped that research will help support or refine these estimates.

"The priority rating for each guideline is simply the highest priority 
level (where 1 is the highest and 3 is the lowest) found among the 
techniques that address that guideline."

Item-2 (EH: 10Dec98). What Is the Shelf-Life of Priorities and Other 
Document Components?

Is shelf-life of information our primary guide to what will be normative 
versus non-normative?

I am not well enough acquainted with the W3C recommendation process to know 
how hard it is to modify something that has become a recommendation.

My assumption is that the normative material for the page authoring 
guidelines should have a shelf-life of about 3 to 5 years, i.e., should not 
need to undergo anything more than "cosmetic" modifications for that 
period. (Is that unreasonable to expect?) I would hope that virtually all 
technique definitions (i.e., a one-sentence or less statement of the 
technique) as well as the priority ratings would not need to undergo 
modifications any more often than 2 to 3 years. Working group members seem 
agreed that techniques should be non-normative. I expect that the other 
technique material (some of which is in the current page authoring 
document) might undergo rather frequent updating, expansion, and revision.

Item-3 (EH: 10Dec98). Flag Techniques that May Be Modified in the Next 1 to 
2 Years

I suggest that any technique that we think will have its priority rating or 
other major feature revised within the next 1 to 2 years be flagged and an 
explanation given of why this might change. These flags and explanations 
would be non-normative.

Item-4 (EH: 10Dec98). Produce Impact Ratings

I think that it is important to generate "impact ratings." Each impact 
rating would indicates the severity of the problem (i.e., seriousness of 
the negative impact) caused by violations of each guideline for each major 
disability group. 

One reason that this may be important is because these impact or severity 
ratings are likely to have a long shelf-life and, thus, might be included 
in the normative part of the document. The impact ratings, as I will 
explain, should be tied to the nature of disabilities (which are 
essentially unchanging) and are more slowly affected by technology change.

Following is a suggested method for producing impact ratings.

#1. List all major affected groups.

This "standard" list of major affected groups should include major 
disability groups and all other significant groups adversely affected by 
violations of each guideline, plus a baseline group, the individuals of 
which have neither disability nor other special conditions. (Note. It is 
important that the set of disability groups on the list become fairly 
standard.) 

This list may include groups whose distinguishing characteristic is 
disability (blind, low-vision, deaf, hard of hearing, LD, physical 
disability, deaf-blind, nondisabled, etc.), culture (non-native speakers of 
the language, etc.), technology (new tech, old tech, voice, mobile, etc.), 
situation (hands-free, noise-free, etc.). For the sake of simplicity, 
include only the most major groups of multiply-constrained users, such as 
deaf-blind.

A substantial portion of this work has been done by the working group in 
the 7 Dec 1998 version of the page authoring guidelines document. Basic 
information about impacts are found in connection with each guideline in a 
section I have termed the "Impact-of-violation" section. This section 
indicates major groups that have been adversely affected. I would suggest 
taking a detailed inventory of the terms used to describe these groups of 
users in order to ensure that all important groups have been included and 
that the names of these groups are used consistently.
 
#2. Define the "Most Favorable" Operating Conditions for Each Group 

The most favorable conditions are those typically preferred by users in the 
group. One facet of this "most favorable conditions" definition is that 
they be using content that is highly accessible and usable (for example, 
that conforms to the WAI page authoring guidelines). For example, for the 
baseline group (whose members are nondisabled individuals), the most 
favorable conditions consist of a fully functional multimedia PC with an 
up-to-date browser in quiet, well-lighted, surroundings using content that 
adheres to the WAI page authoring guidelines. Groups with disabilities, 
particularly blind, low vision, or physically disabled would have 
additional or different hardware and software.

#3. Generate an Impact Rating for Each Group-Guideline Combination

Generate an impact rating that indicates the magnitude of the accessibility 
and usability problems caused by a violation of each guideline for each 
group. Note that when generating this rating, one considers each group 
operating in its own most favorable conditions EXCEPT for the violation to 
the one guideline. For example, ideally one would ask the group member: 
"How large is the negative accessibility and usability impact caused by a 
violations of this one guideline: Nonexistent, minor, moderate, major, or 
catastrophic? Please keep in mind that the size of this impact is 
essentially a difference between the accessibility/usability of Web content 
under two conditions: (1) your preferred conditions and (2) your preferred 
condition EXCEPT for consistent violation of the one guideline." (Perhaps 
the wording should be improved, but this is a rough cut.) As much as 
possible, people in the relevant disability groups should generate the 
impact ratings. As a quick draft, someone trusted by the working group 
could generate the ratings and have them confirmed by knowledgeable people 
with and without disabilities. 

#4. Generate an Adjusted Impact Rating for Each Group-Guideline Combination

Generate an adjusted impact rating. One very important piece of information 
for calculating the adjustments is the impact ratings for the baseline 
group. For example, violations which differentially disadvantage the 
disability-group more than the baseline group result in upward adjustments 
of the impacts for that disability-group-guideline combination. For example,
 a Web failure to provide an interpretation of a foreign language text to 
the group of blind users would cause a small or nonexistent adjustment to 
the impact rating since (let us suppose) the baseline group would be just 
as disadvantaged as the blind users. A contrasting example is failure to 
provide alternative text to blind users. This violation differentially 
disadvantages the blind group over the baseline group and would thus cause 
a significant upward adjustment of the impact in comparison to the baseline 
group. The same kinds of adjustments could be made for 
non-disability-related groups.
 
#5. Summarize Results for Each Guideline

The collection of impacts for the various group-guideline combinations 
could be summarized in a variety of ways, such as mean, median, range, 
minimum, maximum, etc.

While we are interested in differences in technology, culture, and 
situation (see above), we are primarily interested in the 
disability-related groups.

I strongly suggest that the key summary results focus primarily on the 
disability groups (along with the baseline group).

One way of generating the "summary" would be simply to use either the 
highest impact rating generated in #2 or the highest adjusted impact rating 
in #3. (Again, to emphasize the disability focus, this should be the 
highest among the "standard" list of disability groups [see #1] that the 
working group would establish for the paging authoring document.) 

Generally, I think that these impacts, are close to what may have been 
intended, but not implemented, for the guideline priorities in the 7 Dec 
1998 page authoring guidelines.)

Conclusion

As I indicated, because these impact ratings may have a long shelf-life, 
they could become part of the normative material. 

Arguably, the guideline statements without some sort of impact or severity 
rating have no "teeth" or "bones." In a sense, it would be a shame not to 
have some kind of rating within the normative part of the document.

I feel that the method that I have just presented may result in ratings 
that are more stable than the priorities derived from the techniques. 

Yet whether or not the impact ratings are included in the normative part of 
the document, going through this exercise additional advantages. 

First, it will facilitate the cross-disability review announced a few days 
ago by Judy Brewer. People with the disabilities will be better able to 
evaluate the working group's perceptions of how severely they are affected 
by violations of the guidelines. Second, it will help ensure that certain 
disability groups (e.g., deaf-blind) are not overlooked. Third, if my 
recommendation is followed regarding focusing on disability groups for 
summary statistics, then it will strengthen disability access as the 
primary purpose of the document. Fourth, it may increase the defensibility 
of the document as a whole.

I welcome comments and reactions to this (and other) suggestions.

Item-5 (EH: 10Dec98): The Importance of Techniques

The techniques are extremely important, especially since priorities are 
tied to them. For example, perhaps the simplest recommendation one could 
make to an organization that has very limited resources to devote to 
increasing accessibility of its Web sites is that it adhere to all the 
priority 1 techniques. To recommend adherence to all priority 1 guidelines 
would, in many cases, involve adhering to lower priority techniques (2 and 
3).

Item-6 (EH: 10Dec98): Technique Definitions Should Be Able to Stand as a 
Single Sentence

I suggest that each technique definition be able stand as a single sentence 
(even if it is ordinarily embedded within a longer sentence).

Item-7 (EH: 10Dec98): Use of a Single Technique by More Than One Guideline

I believe that it is important at this stage to ask ourselves: "Is it 
possible for the same technique to be used for more than one guideline?"

I think that currently the answer is "No." 

But it may be worthwhile to consider an alternative approach. If, for 
example, it is possible for the same technique to be used across more than 
one guideline, then it is appropriate to attach a unique identifier to each 
technique and provide an index which shows all the guidelines for which 
that each technique is used. This approach might have the benefit of 
reducing the complexity of the document since instead of, say, 60 
techniques, many of which vary only slightly, there might be perhaps 30 or 
40 different techniques. This smaller number of techniques might seem less 
overwhelming, and thereby help improve acceptance of the guidelines. 

Item-8 (EH: 10Dec98): Are Conditions Part of the Technique Definition?

The foregoing question regarding whether a technique can be attached to 
more than one guideline is directly related to the question of whether 
specific conditions (special cases, areas of special usefulness, exceptions,
 etc.) are part of the technique definitions.

I believe that the techniques section (or the techniques document itself) 
is generally the proper place to discuss conditions or constraints (special 
cases, caveats, exceptions, qualifiers, limitations of scope, time period 
during which the technique is applicable, etc.). If these "conditions" may 
be part of the technique definition, then virtually all 60 (or so) of the 
techniques are likely be unique. Otherwise, it is possible that the same 
technique might be used with several different guidelines.

I think that it is important to be consistent in the way that one refers to 
techniques and especially the technique definitions.

Examples of the Structure of Technique Definitions
 
Below are several examples of how technique definitions can be constructed.

First let us consider examples of the case in which a technique definition 
may include conditions:

Example 1. TECHNIQUE-DEFINITION = [TECHNIQUE-PHRASE]. (Like A.1.6, A.10.3)
Example 2. TECHNIQUE-DEFINITION = For [CONDITIONS], [TECHNIQUE-PHRASE]. 
(Like A.3.1)
Example 3. TECHNIQUE-DEFINITION = [TECHNIQUE-PHRASE], e.g., [CONDITIONS]. 
(Like A.1.1)
Example 4. TECHNIQUE-DEFINITION = [TECHNIQUE-PHRASE] when [CONDITIONS]. 
(Looking for examples)
Example 5. TECHNIQUE-DEFINITION = If [CONDITIONS], then [TECHNIQUE-PHRASE]. 
(Looking for examples)

In this case, technique priorities should rate the "importance" of the 
technique definition.

Now let us consider the case in which a technique definition does not 
include conditions. As indicated earlier, in this case, it is quite likely 
that the same technique definition could be found at many locations within 
the document. See examples as follows.

Example 6. [TECHNIQUE-DEFINITION]. (Like A.1.6, A.10.3)
Example 7. For [CONDITIONS], [TECHNIQUE- DEFINITION]. (Like A.3.1)
Example 8. [TECHNIQUE-DEFINITION], e.g., [CONDITIONS]. (Like A.1.1)
Example 9. [TECHNIQUE-DEFINITION] when [CONDITIONS]. (Looking for examples)
Example 10. If [CONDITIONS], then [TECHNIQUE-DEFINITION]. (Looking for 
examples)

In this case, technique priorities should rate the "importance" of the 
technique-conditions _combination_ (or perhaps technique-guideline 
_combination_).

Unless someone tells me differently I will assume that the technique 
definition consists of a technique-phrase and zero or more conditions and 
that the entire technique definition should encompass no more than one 
sentence. Obviously, this one-sentence restriction does not prevent the use 
of additional sentences for providing elaboration or detail. It does, 
however, challenge the working group to ensure that there exists a succinct,
 one-sentence statement of each technique. I suggest that this sentence be 
the first sentence in any technique section. 

By the way, the underlining [hyperlinking] of only phrases rather than 
whole sentences seems to suggest that the "phrase" _is_ the 
technique-definition. If the technique definition _is_ a whole sentence, as 
I assume now to be the case, this might necessitate underlining the whole 
sentence. An alternative to underlining the whole sentence might be to 
underline a technique identifier and a 2 or 3 word technique label, e.g., 
_A.8.1. Summaries for Tables_. Provide summaries for tables (e.g., via the 
"summary attribute on HTML TABLE elements). (By the way, I suggest 
attaching a full identifier to each technique, e.g., A.8.1).

Item-9 (EH: 10Dec98): A Few Other Issues

I cannot help noticing a few other issues in the document that affect its 
underlying structural complexity and thereby its amenability to automatic 
compilation. 

I realize that there currently appears to be no driving urgency for 
automatic generation of WAI guideline-related documents, but I urge the 
working group to avoid features that would make it hard or impossible to 
implement such a technology. (I suppose that one way to think about this is 
that we might want to develop a general WAI DTD for guideline documents, so 
we want our guideline document to be readily comformable to that DTD when 
it is developed.)

None of the issues below seem deadly at this moment, but I think that they 
are important issues to consider.

Item-10 (EH: 10Dec98): The Use of Sub-Techniques

I think that it is OK when necessary to have techniques within techniques. 

I suggest that each priority be addressable by a consistent outline format. 
Note that A.9.4.2.b actually lacks a the "b" term of the address. (Actually,
 I feel a little bit torn on this issue because while I know that each 
priority needs to be uniquely specifiable, I dislike turning unordered 
lists into unordered lists.)

One issue that comes up with sub-techniques is how one assigns priority 
levels at any given level of the technique hierarchy. For example, should 
the priority level at one level be defined simply as the value of the 
highest priority (lowest numbered) next-level-down sub-technique? Whatever 
method is decided should be made stated and followed consistently.

One of the major challenges in the technique section is how to best capture 
the relationships between techniques. The challenge of this can be 
demonstrated by the various relationships that may exist between 
techniques: logical sequence, alternatives for the same purpose, 
complementary techniques, etc. It is hard to capture that diversity in a 
single hierarchy, much less in a simple list.
 
Item-11 (EH: 10Dec98): Complex priority statements

Ideally, a priority statement would be simple, consisting of the word 
"priority" followed by a number. One would avoid complex priority 
statements. Technique A.11.1 is complex because it includes conditions with 
alternative priorities: "[Priority 1 if information or functionality is 
important, and not presented elsewhere, otherwise priority 2]". (See also 
B.1.3. and A.1.3.b.). At this point, I would much prefer breaking 
techniques like this into two sub-techniques.

Item-12 (EH: 10Dec98):  "See Also"

Noted in previous memo.

Item-13 (EH: 10Dec98):  Notes

Noted in previous memo.

Item-14 (EH: 10Dec98): Consistent phrasing of guidelines

Guidelines should be consistently phrased. Guideline A.11 stands out 
(except for A.14) by being phrased passively. It does not start with an 
active verb. Shouldn't it be phrased something like the technique, such as: 
"Make programmatic elements, such as scripts and applets, directly 
accessible." The technique statement properly adds the qualification "Where 
possible..."
 
Item-15 (EH: 10Dec98): Undefined or Vague Words

Examples of such words that "raise flags" with me include:

"Where possible..." (A.11.1)
"If possible..." (A.12.2)
"important" (A.2) - (By the way, it was good to provide a definition at the 
end of the document, though I find it less than wonderful to have so many 
links to that definition. I guess I am hard to pleasež)
"proper" (A.14), "properly" (A.8)

The legalistic side of me wants to work hard to eliminate such words and 
phrases.

A more reasonable side of me says eliminate them from guidelines and 
include them, if necessary, in techniques.

On the other extreme, part of me recognizes that these phrases can be 
genuinely useful in the conveying important subtlety of meaning. For 
example, they may be quiet tip-offs that the working group knows that 
applying these specific guidelines and techniques is likely to be difficult,
 requiring judgement on the part of the page author, and that therefore the 
working group will allow authors some wiggle room.
=============================
Eric G. Hansen, Ph.D.
Development Scientist
Educational Testing Service
ETS 12-R
Rosedale Road
Princeton, NJ 08541
(W) 609-734-5615
(Fax) 609-734-1090
E-mail: ehansen@ets.org

Received on Thursday, 10 December 1998 18:13:46 UTC