RE: Bulk use of accessibility checkers and other auditing tool

FYI.  We, at NIST, have been following the development of EARL and are going to 
try a couple of examples of the output of our WebSAT tool ( that does some 
usability checking) represented in EARL   The Common Industry Format for 
Reporting on User Tests (from the IUSR project, another project in my group) is 
a format for report on testing that a usability engineer does with users, not 
for automated checking, so the format contains information such as the user 
demographics, the tasks, the metrics (such as time on task, and number of 
errors), the data collected about these metrics,etc.  So, the CIF reports more 
on the process and data analysis from testing with users.  You can read more 
about it at http://www.nist.gov/iusr  and please feel free to email us for more 
info. on the CIF.  I'm not sure what the synergy would be except that something 
analogous could be build to report on testing for accessibility, but to report 
on the output of automated tools testing, say 508, would be very different that 
anything in the CIF.  Sharon

Quoting karl.hebenstreit@gsa.gov:

> 
> Regarding EARL, there's been an activity at the National Institute of
> Standards and Technology (NIST) on developing an Industry USability
> Reporting (IUSR).  There could be some interesting synergy between
> these
> activities.   Currently, IUSR has an open comment period until Sept. 10
> on
> a standards proposal through National Committee for Information
> Technology
> Standards (NCITS):
> http://zing.ncsl.nist.gov/iusr
> 
> _____
> 
> Karl Hebenstreit, Jr.
> US General Services Administration
> Office of Governmentwide Policy
> Office of the Chief Information Officer
> 1800 F Street NW, Room G-12
> Washington, DC  20405
> E-mail:  karl.hebenstreit@gsa.gov
> Website:  http://www.gsa.gov
> 
> 
> 
> 
> 
>                                                                         
>                                              
>                                                                         
>                                              
>                     "Brian Kelly"          To:     "'Al Gilman'"
> <asgilman@iamdigex.net>, "'Charles McCathieNevile'"  
>                     <b.kelly@ukoln.        <charles@w3.org>             
>                                              
>                     ac.uk>                 cc:     w3c-wai-ig@w3.org,
> chitchcock@cast.org, (bcc: Karl F. Hebenstreit  
>                     Sent by:               Jr./MJ/CO/GSA/GOV)           
>                                              
>                     w3c-wai-ig-requ        Subject:     RE: Bulk use of
> accessibility checkers and other auditing     
>                     est@w3.org             tool                         
>                                              
>                                                                         
>                                              
>                                                                         
>                                              
>                     08/03/01 01:10                                      
>                                              
>                     PM                                                  
>                                              
>                                                                         
>                                              
>                                                                         
>                                              
> 
> 
> 
> 
> Thanks for your message Al.  Excellent points as always!
> 
> I'm involved in some related work - statistics for measuring use of
> e-journals, in which you the publishers wish to agree on standards for
> filtering hits from robots, double clicks in a short period, etc.
> 
> I've been looking for bodies involved in standardisation in this area.
> I've found two main bodies - the Advertising Community and the
> Auditing
> Community.
> 
> Links to resources in this area are available at:
> http://groups.yahoo.com/group/usage-processing/links
> 
> The following seem to be the main standards bodies.  However it can be
> difficult to tell just be looking at Web sites how authorative and
> international they are.
> 
> ABCE's definitions
> http://www.abce.org.uk/cert/def.asp
> 
> IAB UK's definitions
> Definitions of advertising impressions and insertions
> http://131.103.219.14/index.php3?class=news&view=238
> 
> JICWEBS (The Joint Industry Committee for Web Standards in the UK and
> Ireland) home page
> http://www.jicwebs.org/
> 
> JICWEBS page on standards
> http://www.jicwebs.org/standards.html
> 
> Website Audit
> Work of the International Federation Audit Bureaux of Circulations
> (IFABC)
> http://www.ivc.com.ar/v_ing/audit_1.htm
> 
> 
> Brian
> ---------------------------------------
> Brian Kelly
> UK Web Focus
> UKOLN
> University of Bath
> BATH
> BA2 7AY
> Email: B.Kelly@ukoln.ac.uk
> Web: http://www.ukoln.ac.uk/
> Phone: 01225 323943
> 
> > -----Original Message-----
> > From: Al Gilman [mailto:asgilman@iamdigex.net]
> > Sent: 03 August 2001 16:09
> > To: Brian Kelly; 'Charles McCathieNevile'
> > Cc: w3c-wai-ig@w3.org; chitchcock@cast.org
> > Subject: RE: Bulk use of accessibility checkers and other
> > auditing tool
> >
> >
> > At 11:47 AM 2001-08-03 , Brian Kelly wrote:
> > >Thanks Charles.  I was aware of the EARL work, and it has some
> > >relevance. However although it will allow you to make
> > statements such
> > >as "The entry point is 50K which is within the x guidelines" in a
> > >machinereadable way, the meaning of the size of a page does not yet
> > >appear to be defined anywhere - i.e. does it mean the orgin of a
> > >redirect page, the destination, or the sum; what does it mean if
> > >user-agent content negotiation occurs; etc.
> >
> > AG::
> >
> > Brian,
> >
> > This is an excellent issue.  Just please put an arm's length
> > between "the size of the entry transaction" and the phrase
> > "the size of a page."
> >
> > What we need to define is "that to which the data volume
> > standard applies."
> > This is the size of the "enter the site transaction."  Not
> > any single management unit of retained data.  The page notion
> > is handy in the user appreciation of what they do and get,
> > but generates confusion when we think we can bind it
> > one-to-one onto something the service retains or does.
> >
> > Talking about "the size of a page" puts you unfortunately in
> > static "format" view whereas for measuring network load you
> > want to be in protocol or "transaction" view.
> >
> > The unit you want in the rule you mention is
> >
> >   The total data communication which is automatically
> > engendered when the user activates
> >   a hyperlink referencing "the home page" of "a site."
> >
> > That's almost right.  First variation is to exclude
> > time-delayed auto-refreshes.  Second variation is to include
> > back in time-delayed auto-refreshes of under [say, ten
> > seconds] delay so that automatically passing through a splash
> > page is counted as an automatic part of what you have to go
> > through on entering the site.
> >
> > But these details should be handed as definitions first and
> > standards second (later).
> >
> > What constitutes "a site" is up to the service offeror.  We
> > presume that there is a default starting URI for a given site
> > and that is "the home page" for that site.
> >
> > The best commercial practice here is that this is something
> > with an http: URL that is all DNS and no filepath.  But our
> > assessment vocuabulary should allow for the units of service,
> > and home URI spelling, of the offeror's choice.
> >
> > Your rule will constrain the volume of network data transport
> > service that the user has bought by one "explicit user
> > action" event, in the idea-set of the UAAG.
> >
> > All the variations in the actual network utilization episode
> > that you mention show that the actual data volume is a
> > distribution of different values under various contingencies.
> >  But if there is an automatic redirect, the traffic for both
> > GETs is clearly included.  For negotiated alternatives, you
> > have to pick your policy.  There is no policy-free technical
> > definition.  In terms of surveys, standards-setting, and
> > quality of service measurement, it is reasonable to use
> > averages across the experienced distribution of hit-subcases
> > as "what is constrained by the norm."  Perhaps max limits as
> > well.  What you set norms for is there a policy matter, not a
> > matter of common definitions that can serve a variety of
> > policy decisions.  The technical terms are the primitives
> > that you use in defining what the norm applies to, not the
> > bottom line you decide to constrain.
> >
> > >I'm not sure who should be involved in this work.  Is it
> > within scope
> > >of the WAI group?  If not, is there any work going on in,
> > say, the Web
> > >auditing communities - I know there is some standarisation
> > work going
> > >on in that community.
> > >
> >
> > Any standardisation work in Web auditing would be of interest
> > to at least EO and ER.  If you can provide pointers where we
> > can follow up that would be super.
> >
> > This topic intersects with the scopes of at least ER, GL, and
> > EO.  GL is endeavoring to test the testability of their
> > pronouncements before publishing them as guidance.  "Don't
> > make your home page too big" is a web content guideline, IMHO.
> >
> > Al
> >
> > >Thanks
> > >
> > >Brian
> > >---------------------------------------
> > >Brian Kelly
> > >UK Web Focus
> > >UKOLN
> > >University of Bath
> > >BATH
> > >BA2 7AY
> > >Email: B.Kelly@ukoln.ac.uk
> > >Web: <http://www.ukoln.ac.uk/>http://www.ukoln.ac.uk/
> > >Phone: 01225 323943
> > >
> > >> -----Original Message-----
> > >> From: Charles McCathieNevile
> > [<mailto:charles@w3.org%5D>mailto:charles@w3.org]
> > >> Sent: 03 August 2001 14:03
> > >> To: Brian Kelly
> > >> Cc: w3c-wai-ig@w3.org; chitchcock@cast.org
> > >> Subject: Re: Bulk use of accessibility checkers and other
> > >> auditing tool
> > >>
> > >>
> > >> Hi Brian,
> > >>
> > >> there are checkers that are starting to offer customisation
> > >> features. The ones I know are either ones that cst money, or
> > >> require other tools that cost money.
> > >>
> > >> But finally, yes, there are tools producing output in a
> > >> machine readable format, there is work in the ERT group on a
> > >> project called EARL (Evaluation and Report Language) which is
> > >> an RDF vocabulary specially designed for reporting results
> > >> like Bobby in machine readable format. And Josh Krieger and
> > >> CHris Ridpath (noted for Bobby and A-prompt respectively)
> > >> have just  produced a tool that outputs EARL.
> > >>
> > >> More information is probably most easily available from the
> > >> EARL homepage
> > <http://www.w3.org/2001/03/earl>http://www.w3.org/2001/03/earl
> > for people
> > >> intersted in the details, or from the ER group's mailing list
> > >> archives - look for example at the thread beginning at
> > >>
> > <http://lists.w3.org/Archives/Public/w3c-wai->http://lists.w3.
> org/Archives/P
> ublic/w3c-wai-> er-ig/2001Jul/0055
> >>
> >> cheers
> >>
> >> Charles
> >>
> >> On Fri, 3 Aug 2001, Brian Kelly wrote:
> >>
> >>   Has anyone provided standard definitions for what
> >> constitutes a page,
> >>   and what action user-agent should take when such strange
> >> things happen?
> >>   Are any auditing tools providing customisation over the actions
> they
> >>   will take?
> >>
> >>   Finally are any of the Web sites which provide such Web analysis
> >>   features looking at going down the "Web service" route, and
> >> providing
> >>   output in a machine understandable format - so that the
> >> results can be
> >>   more easily post-processed?
> >>
> >>
> >>
> >
> 
> 
> 
> 
> 
> 
>

Received on Friday, 3 August 2001 15:19:12 UTC