Re: testing conformance

Another process that can be used as an example is the old ATRC 
Courseware Accessibility Study at: 
http://snow.utoronto.ca/initiatives/crseval/crseval.html. This is has 
been updated and usability tests are ongoing, once they are complete 
(end of this month) we can make the new methodology public.

Jutta

  At 1:41 AM -0400 6/27/00, Charles McCathieNevile wrote:
>I have attached my rough notes on testing procedure by checkpoint. I also
>include them below.
>
>charles
>
>
>   Guideline 1. Support accessible authoring practices.
>
>     Checkpoints:
>    
>    1.1 Ensure that the author can produce [1]accessible content in the
>           [2]markup language(s) supported by the tool. [Priority 1]
>           ([3]Checkpoint 1.1)
>           If there is a source editing mode then this can be done -
>           alternatively, if all elemments of the markup language can be
>           used it is met. If not, check against the W3C Notes access
>           features of HTML/CSS/SMIL, or specification for the markup used
>           (e.g. PDF, etc)
>          
>    1.2 Ensure that the tool preserves all [4]accessibility information
>           during authoring, [5]transformations, and [6]conversions.
>           [Priority 1] ([7]Checkpoint 1.2)
>           Take content with as much accessiblity information as possible
>           and run it through the possible transformations
>          
>    1.3 Ensure that when the tool automatically generates markup it
>           conforms to the W3C's Web Content Accessibility Guidelines 1.0
>           [8][WCAG10]. [Relative Priority] ([9]Checkpoint 1.3)
>           I have been testing this by contradiction. It would be nice if
>           there were test suites or test tasks available - I believe the
>           ER group is working on them. Note that this is a relative
>           priority test, so trying to do the things that are outlined in
>           the WCAG checklist by priorityt level is helpful - if it fails
>           at level A then it has failed at higher levels (although it is
>           useful to know more details)
>          
>    1.4 Ensure that templates provided by the tool conform to the Web
>           Content Accessibility Guidelines 1.0 [10][WCAG10].
>           [Relative Priority] ([11]Checkpoint 1.4)
>           Manual checking of the templates (assisted by various tools -
>           see WAI-ER' list of existing tools, the work by the EO group on
>           review, etc).
>          
>   Guideline 2. Generate standard markup.
>
>     Checkpoints:
>    
>    2.1 Use the latest versions of W3C Recommendations when they are
>           available and appropriate for a task. [Priority 2]
>           ([12]Checkpoint 2.1)
>           Check the W3C Technical Reports page for applicable
>           specifications. Typically XHTML, PNG, XML, XML namespaces,
>           Including Stylesheets in XML, MathML, CSS...
>          
>    2.2 Ensure that the tool automatically generates valid markup.
>           [Priority 1] ([13]Checkpoint 2.2)
>           validate output.
>
>    2.3 If markup produced by the tool does not conform to W3C
>           specifications, [14]inform the author. [Priority 3]
>           ([15]Checkpoint 2.3)
>           If the tool can be made to produce markup that is
>           non-conformant to a w3c specification, do it and see if it
>           informs the author
>          
>   Guideline 3. Support the creation of accessible content.
>
>     Checkpoints:
>    
>    3.1 [16]Prompt the author to provide [17]equivalent alternative
>           information (e.g., [18]captions, [19]auditory descriptions, and
>           [20]collated text transcripts for video). [Relative Priority]
>           ([21]Checkpoint 3.1)
>           Import a document containing images, movies, server-side image
>           maps, applets and scripts, and see if there is prompting
>           available to point out that some of these are missing
>           alternative content. Add the elements and see if there is
>           pormpting.
>          
>    3.2 Help the author create structured content and separate information
>           from its presentation. [Relative Priority] ([22]Checkpoint 3.2)
>           Is it obvious how to create structured markup or is it a
>           natural part of the process, or are you prompted to do it in
>           accessibility checks?
>          
>    3.3 Ensure that prepackaged content conforms to the Web Content
>           Accessibility Guidelines 1.0 [23][WCAG10]. [Relative Priority]
>           ([24]Checkpoint 3.3)
>           Same testing as 1.4.
>          
>    3.4 Do not automatically generate [25]equivalent alternatives. Do not
>           reuse previously authored alternatives without author
>           confirmation, except when the function is known with certainty.
>           [Priority 1] ([26]Checkpoint 3.4)
>           Insert images / movies / etc and inspect the source.
>          
>    3.5 Provide functionality for managing, editing, and reusing
>           [27]alternative equivalents for multimedia objects.
>           [Priority 3] ([28]Checkpoint 3.5)
>          
>   Guideline 4. Provide ways of checking and correcting inaccessible content.
>
>    Note: Validation of markup is an essential aspect of checking the
>    accessibility of content.
>   
>     Checkpoints:
>    
>    4.1 [29]Check for and [30]inform the author of [31]accessibility
>           problems. [Relative Priority] ([32]Checkpoint 4.1)
>           Is there some form of accessibility check? If so, does it cover
>           the WCAG checkllist (to what priority level - this is a
>           relative priority).
>          
>    4.2 Assist authors in correcting [33]accessibility problems.
>           [Relative Priority] ([34]Checkpoint 4.2)
>           Is there a correction wizard, or help documentation associated
>           with accessibilty checking?.
>          
>    4.3 Allow the author to preserve markup not recognized by the tool.
>           [Priority 2] ([35]Checkpoint 4.3)
>           Make up some arbitrary source (or modify something existing by
>           adding some garbage markup) and see if it can be imported and
>           preserved (it is legitimate for the tool to refuse to process
>           further...)
>          
>    4.4 Provide the author with a summary of the document's accessibility
>           status. [Priority 3] ([36]Checkpoint 4.4)
>           Yes/No test...
>          
>    4.5 Allow the author to transform [37]presentation markup that is
>           misused to convey structure into [38]structural markup, and to
>           transform presentation markup used for style into style sheets.
>           [Priority 3] ([39]Checkpoint 4.5)
>           Is there a way to transform markup that converts from styles to
>           markup and presentation markup to style + structure markup?
>          
>   Guideline 5. Integrate accessibility solutions into the overall "look and
>   feel".
>
>     Checkpoints:
>    
>    5.1 Ensure that functionality related to [40]accessible authoring
>           practices is naturally integrated into the overall look and
>           feel of the tool. [Priority 2] ([41]Checkpoint 5.1)
>           While testing other checkpoints, are the solutions part of the
>           standard interface or do they require some other technique
>           (source editing in a WYSIWYG tool, setting configuration
>           options, ...
>          
>    5.2 Ensure that [42]accessible authoring practices supporting Web
>           Content Accessibility Guidelines 1.0 [43][WCAG10] Priority 1
>           checkpoints are among the most obvious and easily initiated by
>           the author. [Priority 2] ([44]Checkpoint 5.2)
>           Are the functions needed to meet WCAG P1 requirements in front
>           of the author, or do you have
>          
>   Guideline 6. Promote accessibility in help and documentation.
>
>     Checkpoints:
>    
>    6.1 Document all features that promote the production of accessible
>           content. [Priority 1] ([45]Checkpoint 6.1)
>           Check. This requires extensive knowledge of the tool and the
>           documentation to do properly.
>          
>    6.2 Ensure that creating accessible content is a naturally integrated
>           part of the documentation, including examples. [Priority 2]
>           ([46]Checkpoint 6.2)
>           Check the help examples against WCAG
>          
>    6.3 In a dedicated section, document all features of the tool that
>           promote the production of accessible content. [Priority 3]
>           ([47]Checkpoint 6.3)
>           Is there an accessibility section in help? Is it complete?
>          
>   Guideline 7. Ensure that the authoring tool is accessible to authors with
>   disabilities.
>
>     Checkpoints:
>    
>    7.1 Use all applicable operating system and accessibility standards
>           and conventions (Priority 1 for standards and conventions that
>           are essential to accessibility; Priority 2 for those that are
>           important to accessibility; Priority 3 for those that are
>           beneficial to accessibility). ([48]Checkpoint 7.1)
>           Any hints people?
>          
>    7.2 Allow the author to change the presentation within [49]editing
>           views without affecting the document markup. [Priority 1]
>           ([50]Checkpoint 7.2)
>           Can the author add a user style sheet, or specify a
>           presentation format other than the published format (e.g.
>           disable the specified style sheets and inherit system values)?
>          
>    7.3 Allow the author to edit all [51]properties of each [52]element
>           and object in an accessible fashion. [Priority 1]
>           ([53]Checkpoint 7.3)
>           Is there souirce editing available? Is there a property editor
>           for elements which includes available attributes? (see also
>           guideline 2)
>          
>    7.4 Ensure that the [54]editing view allows navigation via the
>           structure of the document in an accessible fashion.
>           [Priority 1] ([55]Checkpoint 7.4)
>           Is it possible to move around the document element by element?
>          
>    7.5 Enable editing of the structure of the document in an accessible
>           fashion. [Priority 2] ([56]Checkpoint 7.5)
>           Can elements be cut/copied/pasted as elements?
>          
>    7.6 Allow the author to search within [57]editing views. [Priority 2]
>           ([58]Checkpoint 7.6)
>           Is there a search funstion?
>      _________________________________________________________________
>   
>    [[59]contents]
>
>References
>
>
>Content-Type: TEXT/html; name="forau.html"
>Content-ID: <Pine.LNX.4.20.0006270141090.3627@tux.w3.org>
>Content-Description:
>Content-Disposition: attachment; filename="forau.html"
>
>Attachment converted: Macintosh HD:forau.html (TEXT/MSIE) (000358B2)

Received on Tuesday, 27 June 2000 12:26:37 UTC