Evaluation Results In XML - SSB Technologies Thoughts

All-
	I have put together a short set of ideas for a what a possible XML
evaluation document could look like.  To begin the conversation here is a
quick proposal of what such a document could look like:

<?xml version="1.0" encoding="UTF-8"?>
<!-- PI if we want to define a DTD / schema for verification -->
<PAGE url="http://www.yahoo.com">
  <VIOLATIONS priority="1" total="optional">
    <VIOLATION key="foo" total="optional">
      <ELEMENT line="optional">
		 <!-- XHTML opening tag for element -->
	     <IMG src="here" alt="Wretched Alt" />
      </ELEMENT>
    </VIOLATION>
</VIOLATIONS>
</PAGE>

Primarily I would suggest initially limiting the scope of the evaluation
document to capturing data relating solely to the violations for a
particular page, rather than combining the page itself and violation data.
This will limit the scope of the document, solely to evaluating the
accessibility of a page.  The reason I would give for this is that the
response we [SSB Technologies] have been getting from the market is that
developers seem to want evaluation independent of repair.  For most sites
that have dynamically produced content evaluation and repair are very
distinct issues and data portability between the two has limited appeal.  If
we preserve the original document state we are assuming that doing so will
allow us to create separate evaluation and repair tools that use XML to
transport data between.

My experience, however, has been that the vast majority of web developers
consider evaluation and repair separately.  Thus a schema that combines the
two may well just add more unnecessary work.   When evaluating a document
the developer wants to know what the issues are.  They want summary reports
and information sorted by priority.  Evaluation data should answer the
questions:
	1.What must I do?
	2.In what order must I do it in?
To this end a document that orders violations linearly by way of the
original HTML v. by priority weakens our ability to address these questions
succinctly.  Concretely assuming we create an XML evaluation file [XML of
the form proposed by Chris] that maintains a version of the original HTML
document.  With that document one cannot produce priority ordered or summary
reports via XSLT.  This is due to the current constraints of XPath and XSLT.
(I won't get into the why of this here but would be happy to discuss it off
the list).  Whereas if the evaluation document is focused on report
production (which is what I think the demand is for) such a report can be
produced.

With that said I must admit that a solution that retains the original
document would be the most elegant.  While I think it may not be *totally*
necessary it does seem to be the best solution.  My only concern would be
that in preserving the original document we retain a way to extract
"reporting" information from the underlying XML doc via XSLT.

The second issue I would like to address is the DTD / schema for the
document.  I believe that it is important that all evaluation tool makers
should be able to extend the document to export custom data.  To this end it
is important that we maintain an unrestrictive document.  I am not well
versed in DTDs or XML schemas to know the technical language for this but my
proposal is simple.  Define a base set of required elements & attributes but
allow tool authors to add additional elements & attributes while maintaining
a valid document.

The third issue is that of having a particular unique identifier for each
possible violation.  I am open to this however there are some important
considerations.  Primarily we currently divide the WCAG based on the
underlying architecture of the program.   To give an example our program
flags "Images without alt attributes", "Images with null alt attributes" and
"Images with suspicious alt attributes" all as different violations.  We do
this because it gives the most detail to our clients as well as mapping to
our programmatic architecture.  Is this division dictated by the W3C? No.
Are we altering the WCAG in doing so? No, just being practical in our
approach to dividing it.  So what does this mean?  It means that the
division of violations can be subjective based on underlying programmatic
architecture.  Thus if we get into the business of assigning particular
violation identifiers we have to be careful to either make them:
	1. Map to the architecture of evaluation engines or
	2. Independent of the architecture of evaluation engines

Going the later route means that we will have to be less specific,
obfuscating the nature of the problem.  Going the former means that we are
writing practical specifications and we may have to make tradeoffs based on
current technical feasibility.

Pursuant to the above, another question arises from defining specific
divisions of the WCAG.  Who should maintain the text that describes the
violation?  I envision these descriptions in the vein of those included with
Bobby and A-prompt.  I would prefer that the W3C maintain descriptions of
violation problems.  The idea being that if the W3C maintains the
descriptions it will avoid re-writes of the guidelines that could "weaken"
the accessibility violation.  To give an example company eAccess (I have no
idea if this is a real company.  Hope note) builds an evaluation product
that tests all of the WCAG.  They profess to be compliant with the WAI but
include their own descriptions of accessibility.  For IMG alt violations
their descriptive text mentions that "page authors should include alt tags
only if they feel like it, or on every other Tuesday."   Obviously an
extreme example but I believe tinkering with the wording of the violations
should be out of the hands of companies trying to evaluate accessibility.

Finally (sorry this is so long) it makes sense to store the elements in the
evaluation document (however that ends up) in XHTML.  The arguments for this
our fairly simple:
	1. It is a more accurate representation than storing them as CDATA
	2. It should be fairly easy to do with an evaluation engine that builds a
DOM (Document Object Model)
	3. It will speed adoption of XHTML

Okay that's it!  I have attached to this e-mail copies of the proposed XML
file as well as a copy of the XML we currently produce with InSight (our
evaluation engine).  While by no means complete they should both be good
food for thought.

TimS

-----Original Message-----
From: w3c-wai-er-ig-request@w3.org
[mailto:w3c-wai-er-ig-request@w3.org]On Behalf Of Chris Ridpath
Sent: Monday, October 23, 2000 8:52 AM
To: WAI ER IG List
Subject: Evaluation Results In XML


We have been working on a means of storing the accessibility evaluation of
an HTML document. Our current approach is to store the evaluation in an XML
document. The XML doc contains the original HTML with any accessibility
problems marked with new XML elements. For example, the following snippet
contains the evaluation of an IMG element that is missing the 'alt'
attribute:

<problem problemName="MISSING_IMG_ALTTEXT" problemID="1234">
<![CDATA[ <img src="rex.jpg" longdesc="rex-desc.html">]]>
</problem>

The XML file that contains the above evaluation is attached to this message.

Each accessibility problem is given a code number so it may be referenced.

A report tool can take the XML document and prepare a report of
accessibility problems.

A repair tool can take the entire document, or pieces of the document, make
repairs then update the original XML document.

The original XML document can be easily converted back to HTML by XSLT or a
simple program.

If the group can agree on a set of specifications then all tool makers can
generate and use the same XML evaluation document.

Comments?

Chris