W3C home > Mailing lists > Public > public-wai-evaltf@w3.org > October 2011

Re: methodology and method

From: Detlev Fischer <fischer@dias.de>
Date: Wed, 26 Oct 2011 09:55:57 +0200
Message-ID: <4EA7BD0D.7000503@dias.de>
To: public-wai-evaltf@w3.org
Hi Shadi,

no, clearly there is no need to explain why accessibility an its 
evaluation is important, that's a given. In my view, the rationale (or 
methodology) explains anf justifies the logic or rationale of the 
hands-on method / approach specified, which in turn just covers the 
things that testers need to know to carry out an evaluation competently.

For example, the methodology would briefly state (with reference to WCAG 
conformance requirement 3) that complete processes must be accessible 
because the success of the completed interaction counts. It would then 
describe how a complete process is to be reflected in a page or page 
state selection, including, possibly, a statement that repetitive 
content (such as a navigation context that doesn't change across the 
process) can be ignored, or tested just once.
To give an example what this would mean on the level of implemented 
testing applicaton: the BITV-Test offers the option for the test of 
selected elements, like a lightbox. This avoids the possible distortion 
of having repeatedly postive results for basically the same page / template.

The rationale might also explain why processes after selection should be 
assigned a priority depending on their importance for the successful use 
of a site, and that this priority might play a part in the tolerance 
metrics and the final verdict (conformant / not conformant). For Amazon, 
comleting a purchase at checkout would be crucial while writing a 
produict comment might be considered ancillary and much less crucial. 
The methodology must argue all these things, for the tester, they are of 
little practical value in the process.

Felle free to forward to the list if you think it might help others 
understand my argument...

Best regards,
Detlev

Am 26.10.2011 09:24, schrieb Shadi Abou-Zahra:
> Hi Detlev,
>
> Just to understand what you mean by "rationale": are you suggesting that
> Eval TF develops a document explaining why it is important to evaluate
> websites for accessibility? This is not in the initial plan.
>
> Best,
> Shadi
>
>
> On 25.10.2011 22:08, Detlev Fischer wrote:
>> I think the rationale (methodology) needs to be well argued and
>> referenced and cannot really be very short. It is just a type of
>> document that has a different purpose and audience compared to a
>> succinct procedural instruction. The latter should be optimised for the
>> practitioner tackling a website and should have little or no theoretical
>> ballast.
>>
>> Quoting Shadi Abou-Zahra <shadi@w3.org>:
>>
>>> Hi Detlev,
>>>
>>> I'm not sure what the others think but I'm not convinced we need two
>>> documents. The "rationale, audience & scope, rationale of sampling"
>>> sections should probably be very short to set the scene for the main
>>> sections but maybe this is unclear from the table of contents alone.
>>>
>>> Best,
>>> Shadi
>>>
>>>
>>> On 25.10.2011 09:45, Detlev Fischer wrote:
>>>> Hi Shadi, hi all,
>>>>
>>>> I made a suggestion a while ago to decouple the methodology (rationale)
>>>> from the method.
>>>>
>>>> * Methodology: rationale, audience & scope, rationale of sampling
>>>> and sampling methods, measurement of score / tolerances,
>>>> references etc.
>>>>
>>>> * Method: a usable procedure for the actual testing of websites, e.g.:
>>>> - make entry check (site applicable for testing)
>>>> - select WCAG level applied (A, AA, AAA)
>>>> - define page sample, select states and processes
>>>> - run through procedure proper, with checkpoints based on WCAG
>>>> success criteria and all checks needed to test whether the sites
>>>> meets them - which in case of 1.1.1 and 1.3.1 probably means
>>>> several checkpoints
>>>> - generate report and, if applicable, conformance statement
>>>>
>>>> The reason is simply that the procedure gets lost in the draft Table of
>>>> Contents. We also need to think from a point of view of the users of
>>>> the
>>>> method.
>>>>
>>>> Any comments?
>>>>
>>>> Detlev
>>>>
>>>> Am 18.10.2011 11:59, schrieb Shadi Abou-Zahra:
>>>>> Hi Detlev,
>>>>>
>>>>> I believe we agreed that the Methodology would explain how to use WCAG
>>>>> 2.0 Techniques for evaluating web page samples rather than to
>>>>> duplicate
>>>>> them or to create synthesized versions of them.
>>>>>
>>>>> If we go down that route we risk re-interpreting WCAG or running into
>>>>> maintenance issues for chasing web technologies as they are released.
>>>>>
>>>>> I think we should, as far as possible, decouple the Methodology from
>>>>> the
>>>>> detailed "alt-text checking"-level; this is already covered by WCAG.
>>>>>
>>>>> Having said that, I can imagine improvements to the "How to Meet
>>>>> WCAG2"
>>>>> quick reference [1], to better address the need of evaluators rather
>>>>> than developers. For example by organizing the Techniques thematically
>>>>> rather than by Success Criteria. However, this would be separate work.
>>>>>
>>>>> [1] <http://www.w3.org/WAI/WCAG20/quickref/>
>>>>>
>>>>> Regards,
>>>>> Shadi
>>>>>
>>>>>
>>>>> On 18.10.2011 11:24, Detlev Fischer wrote:
>>>>>> Am 18.10.2011 09:14, schrieb Shadi Abou-Zahra:
>>>>>>> Hi Detlev,
>>>>>>>
>>>>>>> On 18.10.2011 08:39, Detlev Fischer wrote:
>>>>>>>> Hi Shady,
>>>>>>>>
>>>>>>>> I am still not sure how we can clearly delineate the difference
>>>>>>>> between
>>>>>>>> the wider term "web content" (as in WCAG) and "website" as
>>>>>>>> explained on
>>>>>>>> - <http://www.w3.org/WAI/ER/methodology-reqs/#website>
>>>>>>>
>>>>>>> Agree, this is a challenge that we need to address.
>>>>>>>
>>>>>>>
>>>>>>>> Maybe we should clearly state in the definition whether "website"
>>>>>>>> includes any content offered via plugin technologies like Flash?
>>>>>>>
>>>>>>> As I understand it, "website" includes any web content as defined by
>>>>>>> WCAG 2.0, which would include Flash and other web technologies.
>>>>>>>
>>>>>>>
>>>>>>>> Thisa has practical implications. For example, if Flash would be
>>>>>>>> included, the testing procedure would then either have to be *very*
>>>>>>>> general (actually more general than in many tests of techniques,
>>>>>>>> which
>>>>>>>> do reference tools like aDesigner2) or fork depending on HTML,
>>>>>>>> FLASH,
>>>>>>>> etc.
>>>>>>>
>>>>>>> I'm not sure I understand this rationale. There are already WCAG 2.0
>>>>>>> Techniques for Flash. Maybe not yet a complete set but enough to
>>>>>>> show
>>>>>>> how the Methodology would utilize existing WCAG 2.0 Techniques.
>>>>>>
>>>>>> I guess the procedure would either reference, as planned, tests in
>>>>>> individual techniques (HTML, CSS, Flash, etc) which means it would be
>>>>>> very bitty and carry a lot of redundancy, or it would offer a
>>>>>> practical
>>>>>> synthesis from a hands-on point of view and thereby, add practical
>>>>>> value
>>>>>> for testers. When checking for alt texts, for example, there will be
>>>>>> many applicable techniques which are often mutually exclusive. If I
>>>>>> use
>>>>>> "list images" in WAT to trundle through the images of a web page and
>>>>>> check whether alt texts descripe content (unlinked images), indicate
>>>>>> link target or purpose (linked images) or are empty (decorative
>>>>>> images),
>>>>>> the economic procedure is to synthesize the assessment as you go
>>>>>> through
>>>>>> the list. No one would *actually* step through the atomic tests again
>>>>>> and again. It would also be disruptive from an operational
>>>>>> perspective.
>>>>>> You want to have *one* page which tells you what to do and how to
>>>>>> assess
>>>>>> the instances you find (for some complex SC like 1.1.1 and 1.3.1,
>>>>>> it may
>>>>>> be several, of course).
>>>>>>
>>>>>> So back to my original rationale:
>>>>>>
>>>>>> Either the test procedure is generic and says "Check all images opn
>>>>>> the
>>>>>> page to verify that the alt attribute is present and appropriate"
>>>>>> which
>>>>>> hardly goes beyond the text of WCAG Guideline 1.1. and is
>>>>>> therefore of
>>>>>> questionable merit, or it would present some sort of decision tree
>>>>>> which
>>>>>> recommends a sequence or path through options and thereby synthesizes
>>>>>> the atomic tests necessary to check for conformance to a particular
>>>>>> SC.
>>>>>>
>>>>>> Now, if we deal with HTML/CSS/JS content but also FLASH content, this
>>>>>> might then apear as a choice at some point the decision tree: you
>>>>>> check
>>>>>> if FLASH is used, fork to applicable Flash tests (or a procedure
>>>>>> syntheizing tests), the return to check HTML (just as an example). It
>>>>>> just makes an already complex HTML-oriented test procedure still more
>>>>>> complex.
>>>>>>
>>>>>> Just as an aside: in BIT-Test we deal with this problem in a 'silo
>>>>>> approach': we (will) offer separate WCAG-based tests for different
>>>>>> media
>>>>>> (HTML/CSS/JS, PDF, maybe some day Flash).
>>>>>>
>>>>>> I personally think (and I have said this before) that a test
>>>>>> procedure
>>>>>> that integrates atomic tests and offers advice regarding the
>>>>>> success of
>>>>>> failure of instances checked is the actual added value in practical
>>>>>> terms.
>>>>>>
>>>>>> Detlev
>>>>>>>
>>>>>>> Best,
>>>>>>> Shadi
>>>>>>>
>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Detlev
>>>>>>>>
>>>>>>>> Am 17.10.2011 15:15, schrieb Shadi Abou-Zahra:
>>>>>>>>> Dear Detlev, All,
>>>>>>>>>
>>>>>>>>> On 17.10.2011 10:15, Detlev Fischer wrote:
>>>>>>>>>> I have now fully caught up with the discussion of the term
>>>>>>>>>> "website"
>>>>>>>>>> and
>>>>>>>>>> I realise it can be (or has been) defined to include web apps
>>>>>>>>>> etc.
>>>>>>>>>
>>>>>>>>> Correct:
>>>>>>>>> - <http://www.w3.org/WAI/ER/methodology-reqs/#website>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Still, since the term "website" has been avoided in the title of
>>>>>>>>>> WCAG
>>>>>>>>>> and also carries connotations of a traditional hierachical
>>>>>>>>>> site, ist
>>>>>>>>>> seems more congenial to me to use "Web content" instead (even
>>>>>>>>>> if it
>>>>>>>>>> makes the acronym still thornier).
>>>>>>>>>
>>>>>>>>> It is not about the acronym but rather about the scope of WCAG
>>>>>>>>> versus
>>>>>>>>> that of the Methodology. Web content is used to build websites.
>>>>>>>>> WCAG
>>>>>>>>> applies to any web content while our Methodology is limited to
>>>>>>>>> websites.
>>>>>>>>>
>>>>>>>>> I think we should either use the term "website" to denote static
>>>>>>>>> and
>>>>>>>>> dynamic web pages, web applications, intranets, mobile sites,
>>>>>>>>> etc, or
>>>>>>>>> come up with another term that applies to websites in their
>>>>>>>>> entirety.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Shadi
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Detlev
>>>>>>>>>>
>>>>>>>>>> Am 17.10.2011 08:40, schrieb kvotis@iti.gr:
>>>>>>>>>>> Hello everyone
>>>>>>>>>>>
>>>>>>>>>>> i aggree with the following comment of Detlev and this was
>>>>>>>>>>> also my
>>>>>>>>>>> comment
>>>>>>>>>>> to a previous mail. I cannot understand why we are refereeing
>>>>>>>>>>> only to
>>>>>>>>>>> Websites. I think that we need a more general term including
>>>>>>>>>>> also
>>>>>>>>>>> Web
>>>>>>>>>>> applications, Mobile Web applications,etc...
>>>>>>>>>>>
>>>>>>>>>>> regards
>>>>>>>>>>>
>>>>>>>>>>> kostas
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Hi everyone,
>>>>>>>>>>>>
>>>>>>>>>>>> back from holidays, I am trying to catch up with discussions
>>>>>>>>>>>> now.
>>>>>>>>>>>>
>>>>>>>>>>>> I do not mind WAEM even though I do not think it is
>>>>>>>>>>>> particularly
>>>>>>>>>>>> easy
>>>>>>>>>>>> to pronounce (other then spelling out double-U - E - A - M).
>>>>>>>>>>>>
>>>>>>>>>>>> I have some concern regarding the W for "website", as in the
>>>>>>>>>>>> current
>>>>>>>>>>>> full title
>>>>>>>>>>>> "Website Accessibility Evaluation Methodology". The "WC" of
>>>>>>>>>>>> WCAG
>>>>>>>>>>>> sounds a lot more general than the term "website". This also
>>>>>>>>>>>> relates
>>>>>>>>>>>> to the important comments by Aaron Leventhal regarding the
>>>>>>>>>>>> scope of
>>>>>>>>>>>> the methodology, tool independence, and the possible need to
>>>>>>>>>>>> cover
>>>>>>>>>>>> different web content and UA scenarios.
>>>>>>>>>>>>
>>>>>>>>>>>> Is what we develop here focused on websites and not on things
>>>>>>>>>>>> like
>>>>>>>>>>>> web
>>>>>>>>>>>> apps for mobile UAs? If we want to be as general as WCAG we
>>>>>>>>>>>> might
>>>>>>>>>>>> need
>>>>>>>>>>>> to change the full name of the methodology and accordingly, the
>>>>>>>>>>>> acronym.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Detlev
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Zitat von "Velleman, Eric"<evelleman@bartimeus.nl>:
>>>>>>>>>>>>
>>>>>>>>>>>>> Dear all,
>>>>>>>>>>>>>
>>>>>>>>>>>>> We decided in the call to discuss the short name in the list.
>>>>>>>>>>>>> Below
>>>>>>>>>>>>> are some short names from the discussions we had sofar and a
>>>>>>>>>>>>> proposed shortlist of requirements:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Requirements for a short name:
>>>>>>>>>>>>> - It should be short
>>>>>>>>>>>>> - Easy to pronounce
>>>>>>>>>>>>> - Clear to all what it means (selfexplanatory)
>>>>>>>>>>>>> - Be about the methodology
>>>>>>>>>>>>>
>>>>>>>>>>>>> Shortnames (Make your choice or add new ones):
>>>>>>>>>>>>> - SiteAccess
>>>>>>>>>>>>> - WCAG-Method
>>>>>>>>>>>>> - WCAG-EM
>>>>>>>>>>>>> - WAEM (really using the title)
>>>>>>>>>>>>> - WCAG-Check
>>>>>>>>>>>>> - AccessSite
>>>>>>>>>>>>> - WCAG-Site
>>>>>>>>>>>>> - AccessCheck
>>>>>>>>>>>>> - SiteCheck
>>>>>>>>>>>>> - CheckSite
>>>>>>>>>>>>> - WAMBAM
>>>>>>>>>>>>> - WAME
>>>>>>>>>>>>> - MCEWA
>>>>>>>>>>>>> - CEWA
>>>>>>>>>>>>> - CEW2WCAG2
>>>>>>>>>>>>> - UWEM
>>>>>>>>>>>>> - SITE
>>>>>>>>>>>>> - MAWA
>>>>>>>>>>>>> - MDWC
>>>>>>>>>>>>> - EWAMAC
>>>>>>>>>>>>> - EMAWC
>>>>>>>>>>>>> - WEM
>>>>>>>>>>>>>
>>>>>>>>>>>>> Kindest regards,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Eric
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> ---------------------------------------------------------------
>>>>>>>>>>>> Detlev Fischer PhD
>>>>>>>>>>>> DIAS GmbH - Daten, Informationssysteme und Analysen im Sozialen
>>>>>>>>>>>> Gesch?ftsf?hrung: Thomas Lilienthal, Michael Zapp
>>>>>>>>>>>>
>>>>>>>>>>>> Telefon: +49-40-43 18 75-25
>>>>>>>>>>>> Mobile: +49-157 7-170 73 84
>>>>>>>>>>>> Fax: +49-40-43 18 75-19
>>>>>>>>>>>> E-Mail: fischer@dias.de
>>>>>>>>>>>>
>>>>>>>>>>>> Anschrift: Schulterblatt 36, D-20357 Hamburg
>>>>>>>>>>>> Amtsgericht Hamburg HRB 58 167
>>>>>>>>>>>> Gesch?ftsf?hrer: Thomas Lilienthal, Michael Zapp
>>>>>>>>>>>> ---------------------------------------------------------------
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>> --
>>> Shadi Abou-Zahra - http://www.w3.org/People/shadi/
>>> Activity Lead, W3C/WAI International Program Office
>>> Evaluation and Repair Tools Working Group (ERT WG)
>>> Research and Development Working Group (RDWG)
>>>
>>>
>>
>>
>>
>> --
>> ---------------------------------------------------------------
>> Detlev Fischer PhD
>> DIAS GmbH - Daten, Informationssysteme und Analysen im Sozialen
>> Geschäftsführung: Thomas Lilienthal, Michael Zapp
>>
>> Telefon: +49-40-43 18 75-25
>> Mobile: +49-157 7-170 73 84
>> Fax: +49-40-43 18 75-19
>> E-Mail: fischer@dias.de
>>
>> Anschrift: Schulterblatt 36, D-20357 Hamburg
>> Amtsgericht Hamburg HRB 58 167
>> Geschäftsführer: Thomas Lilienthal, Michael Zapp
>> ---------------------------------------------------------------
>>
>>
>


-- 
---------------------------------------------------------------
Detlev Fischer PhD
DIAS GmbH - Daten, Informationssysteme und Analysen im Sozialen
Geschäftsführung: Thomas Lilienthal, Michael Zapp

Telefon: +49-40-43 18 75-25
Mobile: +49-157 7-170 73 84
Fax: +49-40-43 18 75-19
E-Mail: fischer@dias.de

Anschrift: Schulterblatt 36, D-20357 Hamburg
Amtsgericht Hamburg HRB 58 167
Geschäftsführer: Thomas Lilienthal, Michael Zapp
---------------------------------------------------------------
Received on Wednesday, 26 October 2011 07:56:23 GMT

This archive was generated by hypermail 2.3.1 : Friday, 8 March 2013 15:52:12 GMT