FW: Conformance Level Determination

Hi All, 

 

At the last GL Teleconference call we discussed the levels of
conformance and how to determine what goes in which.   I promised to
take a crack at capturing what has been discussed and done in this
regard.

 

 

Here is what it looks like so far

 

 

Preamble

1 - We already decided that success criteria had to all be things that
were testable.   This could either be machine testing  or Human testing
with a high inter-rater reliability (HIRR). (i.e. most raters who
understand issue and/or measure would all say the same thing.)

 

2 - We also decided that Levels 1 and 2 would have to be things that
applied to all (types of) sites.  Especially important for Level 1 -
since no type of conformance could be claimed if without all level one
items.  And if we made it so that some sites could not reasonably meet a
level 1, we would be making it so that some sites could make not claims
of accessibilty.  

 

 

 

 

In the notes below 

MACHINE - means machine testable

HIRR  - means  Human evaluate-able with High Inter Rater Reliability
(meaning that raters would agree)

  

 

POSSIBLE GOAL STATEMENTS FOR CREATING LEVEL 1,2,3 etc SUCCESS CRITERIA

 

Criteria for Level 1 Success Criteria

 

-          Things that are testable  (Machine or HIRR)

-          We want to recommend that these be done everywhere

-          Needs to be things that can be done on every site - no
exceptions unless in the success criteria

-          Absolute essentials.  If these aren't done - most everyone in
a major disability group can't access the information.  (e.g. no
captions)

 

 

Level 2 

 

-          Things that are testable  (Machine or HIRR)

-          Things that go beyond Level 1 but that can still be done on
all sites (if designed properly)

-          Important and readily achievable on all sites without great
burden

-          These are things that we hope and encourage all sites to do -
and want these to be doable enough that we can ask all managers to
require (i.e. require Level 2 conformance) 

 

 

Level 3 

 

-          Things that are testable (Machine or HIRR) 

-          Things that can be done on all sites but that would require
extra effort and commitment.  

-          Don't expect these to be done on most sites. But would be
done on sites that really want to go the extra mile.

-          First level that would include things that require a certain
level of control over the server or grasp of programming/technologies
fall here? (ex. a typical hosting service won't allow you to do X or run
Y script on their servers)

 

 

Additional / Advisory Items

-          things that are not testable but are good ideas to address
this checkpoint  (good advice)

-          things that can't be done on all sites

-          things that might be done to target or tune a site
specifically for a disability group who needs this checkpoint

 

 

SHOULD THERE BE A LEVEL 4????

-        only reason for a level 4 would be to put the Advisory Items
into a Level - instead of moving them to the non-normative block of
information (which would have to happen if they were not success
criteria).  Since some listing may ONLY list the success criteria (and
not the non-normative advisory section) making these level 4 would keep
them visible

-        to make the advisory items into level 4 items though, they
would have to be played with to make them testable.   And I'm not sure
we can do that without making them harder to read and or less effective.

 

 

NOTE:  As you go down the levels - you will pick up more and more people
who cannot use a site unless lower level items are implemented.  Thus
the lower levels make it easier for some people with disabilities, but
they will make it possible for others, particularly with multiple or
very severe disabilities.

 

 

 

Oh yes.

 

I also looked at the     MUST, SHOULD, MAY   and    CAN'T USE,    HARD
TO USE,     EASIER TO USE  formats of the WCAG 1.0 guidelines - and
couldn't make them work here.

 

One thing that we found in our work in the GL group - is that items that
make it easier to use for some people - are critical for others.   So
all the items end up being "MUST" or "CAN'T USE WITHOUT" for some
people.    

 

Thus we decided that all checkpoints need to have Level 1 items --  even
if they were hard to write and still make the Machine or HIRR
testability criteria.  So we end up with 

 

"the site  reviewed the content with the objectives below in mind"

or 

 "The site asserts that they reviewed the content with the objectives
below in mind" 

 

Or level 2 that are 

" the user asserts they considered this and did the best they felt was
possible and appropriate."

 

But we don't want to have checkpoints that don't have any testable
measures that can be applied across sites (or we make some sites that
cannot claim any conformance since L1 is minimum)

 

AND - we don't want to take some checkpoints off the table for L1 (or
they might not even show up on people's todo list).

 

 

Your thoughts?

 

 

Gregg

------------------------------------
Gregg Vanderheiden Ph.D.
Ind Engr - Biomed - Trace,  Univ of Wis
gv@trace.wisc.edu

 

--