Re: AW: methodology and method

Hi Kerstin,

The term "rationale" is ambiguous to me too. If I understand Detlev 
correctly then I think we are now talking more about explanatory 
background material, similarly to what is in Understanding WCAG 2.0.

This may have merit but I suggest we decide to split the document into 
"main" and "background" at a later stage, if it turns out we need to.

Best,
   Shadi


On 26.10.2011 10:56, Kerstin Probiesch wrote:
> Hi Shadi, all,
>
> can please someone from the stuff or the facilitator finally clarify if the goal of this TF is to develop just a rationale (justification) or a methodology? Methodology is defined as: "a guideline for solving a problem, with specific components such as phases, tasks, _methods_, techniques and tools". As written on http://www.w3.org/WAI/ER/2011/eval/#methodology (WCAG 2.0 Evaluation Methodology): "The purpose of this work is to develop an internationally harmonized methodology for evaluating the conformance of websites to WCAG 2.0."
>
> Best
>
> Kerstin
>
>
>> -----Ursprüngliche Nachricht-----
>> Von: public-wai-evaltf-request@w3.org [mailto:public-wai-evaltf-
>> request@w3.org] Im Auftrag von Detlev Fischer
>> Gesendet: Mittwoch, 26. Oktober 2011 09:56
>> An: public-wai-evaltf@w3.org
>> Betreff: Re: methodology and method
>>
>> Hi Shadi,
>>
>> no, clearly there is no need to explain why accessibility an its
>> evaluation is important, that's a given. In my view, the rationale (or
>> methodology) explains anf justifies the logic or rationale of the
>> hands-on method / approach specified, which in turn just covers the
>> things that testers need to know to carry out an evaluation
>> competently.
>>
>> For example, the methodology would briefly state (with reference to
>> WCAG
>> conformance requirement 3) that complete processes must be accessible
>> because the success of the completed interaction counts. It would then
>> describe how a complete process is to be reflected in a page or page
>> state selection, including, possibly, a statement that repetitive
>> content (such as a navigation context that doesn't change across the
>> process) can be ignored, or tested just once.
>> To give an example what this would mean on the level of implemented
>> testing applicaton: the BITV-Test offers the option for the test of
>> selected elements, like a lightbox. This avoids the possible distortion
>> of having repeatedly postive results for basically the same page /
>> template.
>>
>> The rationale might also explain why processes after selection should
>> be
>> assigned a priority depending on their importance for the successful
>> use
>> of a site, and that this priority might play a part in the tolerance
>> metrics and the final verdict (conformant / not conformant). For
>> Amazon,
>> comleting a purchase at checkout would be crucial while writing a
>> produict comment might be considered ancillary and much less crucial.
>> The methodology must argue all these things, for the tester, they are
>> of
>> little practical value in the process.
>>
>> Felle free to forward to the list if you think it might help others
>> understand my argument...
>>
>> Best regards,
>> Detlev
>>
>> Am 26.10.2011 09:24, schrieb Shadi Abou-Zahra:
>>> Hi Detlev,
>>>
>>> Just to understand what you mean by "rationale": are you suggesting
>> that
>>> Eval TF develops a document explaining why it is important to
>> evaluate
>>> websites for accessibility? This is not in the initial plan.
>>>
>>> Best,
>>> Shadi
>>>
>>>
>>> On 25.10.2011 22:08, Detlev Fischer wrote:
>>>> I think the rationale (methodology) needs to be well argued and
>>>> referenced and cannot really be very short. It is just a type of
>>>> document that has a different purpose and audience compared to a
>>>> succinct procedural instruction. The latter should be optimised for
>> the
>>>> practitioner tackling a website and should have little or no
>> theoretical
>>>> ballast.
>>>>
>>>> Quoting Shadi Abou-Zahra<shadi@w3.org>:
>>>>
>>>>> Hi Detlev,
>>>>>
>>>>> I'm not sure what the others think but I'm not convinced we need
>> two
>>>>> documents. The "rationale, audience&  scope, rationale of sampling"
>>>>> sections should probably be very short to set the scene for the
>> main
>>>>> sections but maybe this is unclear from the table of contents
>> alone.
>>>>>
>>>>> Best,
>>>>> Shadi
>>>>>
>>>>>
>>>>> On 25.10.2011 09:45, Detlev Fischer wrote:
>>>>>> Hi Shadi, hi all,
>>>>>>
>>>>>> I made a suggestion a while ago to decouple the methodology
>> (rationale)
>>>>>> from the method.
>>>>>>
>>>>>> * Methodology: rationale, audience&  scope, rationale of sampling
>>>>>> and sampling methods, measurement of score / tolerances,
>>>>>> references etc.
>>>>>>
>>>>>> * Method: a usable procedure for the actual testing of websites,
>> e.g.:
>>>>>> - make entry check (site applicable for testing)
>>>>>> - select WCAG level applied (A, AA, AAA)
>>>>>> - define page sample, select states and processes
>>>>>> - run through procedure proper, with checkpoints based on WCAG
>>>>>> success criteria and all checks needed to test whether the sites
>>>>>> meets them - which in case of 1.1.1 and 1.3.1 probably means
>>>>>> several checkpoints
>>>>>> - generate report and, if applicable, conformance statement
>>>>>>
>>>>>> The reason is simply that the procedure gets lost in the draft
>> Table of
>>>>>> Contents. We also need to think from a point of view of the users
>> of
>>>>>> the
>>>>>> method.
>>>>>>
>>>>>> Any comments?
>>>>>>
>>>>>> Detlev
>>>>>>
>>>>>> Am 18.10.2011 11:59, schrieb Shadi Abou-Zahra:
>>>>>>> Hi Detlev,
>>>>>>>
>>>>>>> I believe we agreed that the Methodology would explain how to use
>> WCAG
>>>>>>> 2.0 Techniques for evaluating web page samples rather than to
>>>>>>> duplicate
>>>>>>> them or to create synthesized versions of them.
>>>>>>>
>>>>>>> If we go down that route we risk re-interpreting WCAG or running
>> into
>>>>>>> maintenance issues for chasing web technologies as they are
>> released.
>>>>>>>
>>>>>>> I think we should, as far as possible, decouple the Methodology
>> from
>>>>>>> the
>>>>>>> detailed "alt-text checking"-level; this is already covered by
>> WCAG.
>>>>>>>
>>>>>>> Having said that, I can imagine improvements to the "How to Meet
>>>>>>> WCAG2"
>>>>>>> quick reference [1], to better address the need of evaluators
>> rather
>>>>>>> than developers. For example by organizing the Techniques
>> thematically
>>>>>>> rather than by Success Criteria. However, this would be separate
>> work.
>>>>>>>
>>>>>>> [1]<http://www.w3.org/WAI/WCAG20/quickref/>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Shadi
>>>>>>>
>>>>>>>
>>>>>>> On 18.10.2011 11:24, Detlev Fischer wrote:
>>>>>>>> Am 18.10.2011 09:14, schrieb Shadi Abou-Zahra:
>>>>>>>>> Hi Detlev,
>>>>>>>>>
>>>>>>>>> On 18.10.2011 08:39, Detlev Fischer wrote:
>>>>>>>>>> Hi Shady,
>>>>>>>>>>
>>>>>>>>>> I am still not sure how we can clearly delineate the
>> difference
>>>>>>>>>> between
>>>>>>>>>> the wider term "web content" (as in WCAG) and "website" as
>>>>>>>>>> explained on
>>>>>>>>>> -<http://www.w3.org/WAI/ER/methodology-reqs/#website>
>>>>>>>>>
>>>>>>>>> Agree, this is a challenge that we need to address.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Maybe we should clearly state in the definition whether
>> "website"
>>>>>>>>>> includes any content offered via plugin technologies like
>> Flash?
>>>>>>>>>
>>>>>>>>> As I understand it, "website" includes any web content as
>> defined by
>>>>>>>>> WCAG 2.0, which would include Flash and other web technologies.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Thisa has practical implications. For example, if Flash would
>> be
>>>>>>>>>> included, the testing procedure would then either have to be
>> *very*
>>>>>>>>>> general (actually more general than in many tests of
>> techniques,
>>>>>>>>>> which
>>>>>>>>>> do reference tools like aDesigner2) or fork depending on HTML,
>>>>>>>>>> FLASH,
>>>>>>>>>> etc.
>>>>>>>>>
>>>>>>>>> I'm not sure I understand this rationale. There are already
>> WCAG 2.0
>>>>>>>>> Techniques for Flash. Maybe not yet a complete set but enough
>> to
>>>>>>>>> show
>>>>>>>>> how the Methodology would utilize existing WCAG 2.0 Techniques.
>>>>>>>>
>>>>>>>> I guess the procedure would either reference, as planned, tests
>> in
>>>>>>>> individual techniques (HTML, CSS, Flash, etc) which means it
>> would be
>>>>>>>> very bitty and carry a lot of redundancy, or it would offer a
>>>>>>>> practical
>>>>>>>> synthesis from a hands-on point of view and thereby, add
>> practical
>>>>>>>> value
>>>>>>>> for testers. When checking for alt texts, for example, there
>> will be
>>>>>>>> many applicable techniques which are often mutually exclusive.
>> If I
>>>>>>>> use
>>>>>>>> "list images" in WAT to trundle through the images of a web page
>> and
>>>>>>>> check whether alt texts descripe content (unlinked images),
>> indicate
>>>>>>>> link target or purpose (linked images) or are empty (decorative
>>>>>>>> images),
>>>>>>>> the economic procedure is to synthesize the assessment as you go
>>>>>>>> through
>>>>>>>> the list. No one would *actually* step through the atomic tests
>> again
>>>>>>>> and again. It would also be disruptive from an operational
>>>>>>>> perspective.
>>>>>>>> You want to have *one* page which tells you what to do and how
>> to
>>>>>>>> assess
>>>>>>>> the instances you find (for some complex SC like 1.1.1 and
>> 1.3.1,
>>>>>>>> it may
>>>>>>>> be several, of course).
>>>>>>>>
>>>>>>>> So back to my original rationale:
>>>>>>>>
>>>>>>>> Either the test procedure is generic and says "Check all images
>> opn
>>>>>>>> the
>>>>>>>> page to verify that the alt attribute is present and
>> appropriate"
>>>>>>>> which
>>>>>>>> hardly goes beyond the text of WCAG Guideline 1.1. and is
>>>>>>>> therefore of
>>>>>>>> questionable merit, or it would present some sort of decision
>> tree
>>>>>>>> which
>>>>>>>> recommends a sequence or path through options and thereby
>> synthesizes
>>>>>>>> the atomic tests necessary to check for conformance to a
>> particular
>>>>>>>> SC.
>>>>>>>>
>>>>>>>> Now, if we deal with HTML/CSS/JS content but also FLASH content,
>> this
>>>>>>>> might then apear as a choice at some point the decision tree:
>> you
>>>>>>>> check
>>>>>>>> if FLASH is used, fork to applicable Flash tests (or a procedure
>>>>>>>> syntheizing tests), the return to check HTML (just as an
>> example). It
>>>>>>>> just makes an already complex HTML-oriented test procedure still
>> more
>>>>>>>> complex.
>>>>>>>>
>>>>>>>> Just as an aside: in BIT-Test we deal with this problem in a
>> 'silo
>>>>>>>> approach': we (will) offer separate WCAG-based tests for
>> different
>>>>>>>> media
>>>>>>>> (HTML/CSS/JS, PDF, maybe some day Flash).
>>>>>>>>
>>>>>>>> I personally think (and I have said this before) that a test
>>>>>>>> procedure
>>>>>>>> that integrates atomic tests and offers advice regarding the
>>>>>>>> success of
>>>>>>>> failure of instances checked is the actual added value in
>> practical
>>>>>>>> terms.
>>>>>>>>
>>>>>>>> Detlev
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Shadi
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Detlev
>>>>>>>>>>
>>>>>>>>>> Am 17.10.2011 15:15, schrieb Shadi Abou-Zahra:
>>>>>>>>>>> Dear Detlev, All,
>>>>>>>>>>>
>>>>>>>>>>> On 17.10.2011 10:15, Detlev Fischer wrote:
>>>>>>>>>>>> I have now fully caught up with the discussion of the term
>>>>>>>>>>>> "website"
>>>>>>>>>>>> and
>>>>>>>>>>>> I realise it can be (or has been) defined to include web
>> apps
>>>>>>>>>>>> etc.
>>>>>>>>>>>
>>>>>>>>>>> Correct:
>>>>>>>>>>> -<http://www.w3.org/WAI/ER/methodology-reqs/#website>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Still, since the term "website" has been avoided in the
>> title of
>>>>>>>>>>>> WCAG
>>>>>>>>>>>> and also carries connotations of a traditional hierachical
>>>>>>>>>>>> site, ist
>>>>>>>>>>>> seems more congenial to me to use "Web content" instead
>> (even
>>>>>>>>>>>> if it
>>>>>>>>>>>> makes the acronym still thornier).
>>>>>>>>>>>
>>>>>>>>>>> It is not about the acronym but rather about the scope of
>> WCAG
>>>>>>>>>>> versus
>>>>>>>>>>> that of the Methodology. Web content is used to build
>> websites.
>>>>>>>>>>> WCAG
>>>>>>>>>>> applies to any web content while our Methodology is limited
>> to
>>>>>>>>>>> websites.
>>>>>>>>>>>
>>>>>>>>>>> I think we should either use the term "website" to denote
>> static
>>>>>>>>>>> and
>>>>>>>>>>> dynamic web pages, web applications, intranets, mobile sites,
>>>>>>>>>>> etc, or
>>>>>>>>>>> come up with another term that applies to websites in their
>>>>>>>>>>> entirety.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Shadi
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Detlev
>>>>>>>>>>>>
>>>>>>>>>>>> Am 17.10.2011 08:40, schrieb kvotis@iti.gr:
>>>>>>>>>>>>> Hello everyone
>>>>>>>>>>>>>
>>>>>>>>>>>>> i aggree with the following comment of Detlev and this was
>>>>>>>>>>>>> also my
>>>>>>>>>>>>> comment
>>>>>>>>>>>>> to a previous mail. I cannot understand why we are
>> refereeing
>>>>>>>>>>>>> only to
>>>>>>>>>>>>> Websites. I think that we need a more general term
>> including
>>>>>>>>>>>>> also
>>>>>>>>>>>>> Web
>>>>>>>>>>>>> applications, Mobile Web applications,etc...
>>>>>>>>>>>>>
>>>>>>>>>>>>> regards
>>>>>>>>>>>>>
>>>>>>>>>>>>> kostas
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi everyone,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> back from holidays, I am trying to catch up with
>> discussions
>>>>>>>>>>>>>> now.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I do not mind WAEM even though I do not think it is
>>>>>>>>>>>>>> particularly
>>>>>>>>>>>>>> easy
>>>>>>>>>>>>>> to pronounce (other then spelling out double-U - E - A -
>> M).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have some concern regarding the W for "website", as in
>> the
>>>>>>>>>>>>>> current
>>>>>>>>>>>>>> full title
>>>>>>>>>>>>>> "Website Accessibility Evaluation Methodology". The "WC"
>> of
>>>>>>>>>>>>>> WCAG
>>>>>>>>>>>>>> sounds a lot more general than the term "website". This
>> also
>>>>>>>>>>>>>> relates
>>>>>>>>>>>>>> to the important comments by Aaron Leventhal regarding the
>>>>>>>>>>>>>> scope of
>>>>>>>>>>>>>> the methodology, tool independence, and the possible need
>> to
>>>>>>>>>>>>>> cover
>>>>>>>>>>>>>> different web content and UA scenarios.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Is what we develop here focused on websites and not on
>> things
>>>>>>>>>>>>>> like
>>>>>>>>>>>>>> web
>>>>>>>>>>>>>> apps for mobile UAs? If we want to be as general as WCAG
>> we
>>>>>>>>>>>>>> might
>>>>>>>>>>>>>> need
>>>>>>>>>>>>>> to change the full name of the methodology and
>> accordingly, the
>>>>>>>>>>>>>> acronym.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Detlev
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Zitat von "Velleman, Eric"<evelleman@bartimeus.nl>:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Dear all,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We decided in the call to discuss the short name in the
>> list.
>>>>>>>>>>>>>>> Below
>>>>>>>>>>>>>>> are some short names from the discussions we had sofar
>> and a
>>>>>>>>>>>>>>> proposed shortlist of requirements:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Requirements for a short name:
>>>>>>>>>>>>>>> - It should be short
>>>>>>>>>>>>>>> - Easy to pronounce
>>>>>>>>>>>>>>> - Clear to all what it means (selfexplanatory)
>>>>>>>>>>>>>>> - Be about the methodology
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Shortnames (Make your choice or add new ones):
>>>>>>>>>>>>>>> - SiteAccess
>>>>>>>>>>>>>>> - WCAG-Method
>>>>>>>>>>>>>>> - WCAG-EM
>>>>>>>>>>>>>>> - WAEM (really using the title)
>>>>>>>>>>>>>>> - WCAG-Check
>>>>>>>>>>>>>>> - AccessSite
>>>>>>>>>>>>>>> - WCAG-Site
>>>>>>>>>>>>>>> - AccessCheck
>>>>>>>>>>>>>>> - SiteCheck
>>>>>>>>>>>>>>> - CheckSite
>>>>>>>>>>>>>>> - WAMBAM
>>>>>>>>>>>>>>> - WAME
>>>>>>>>>>>>>>> - MCEWA
>>>>>>>>>>>>>>> - CEWA
>>>>>>>>>>>>>>> - CEW2WCAG2
>>>>>>>>>>>>>>> - UWEM
>>>>>>>>>>>>>>> - SITE
>>>>>>>>>>>>>>> - MAWA
>>>>>>>>>>>>>>> - MDWC
>>>>>>>>>>>>>>> - EWAMAC
>>>>>>>>>>>>>>> - EMAWC
>>>>>>>>>>>>>>> - WEM
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Kindest regards,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Eric
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> ----------------------------------------------------------
>> -----
>>>>>>>>>>>>>> Detlev Fischer PhD
>>>>>>>>>>>>>> DIAS GmbH - Daten, Informationssysteme und Analysen im
>> Sozialen
>>>>>>>>>>>>>> Gesch?ftsf?hrung: Thomas Lilienthal, Michael Zapp
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Telefon: +49-40-43 18 75-25
>>>>>>>>>>>>>> Mobile: +49-157 7-170 73 84
>>>>>>>>>>>>>> Fax: +49-40-43 18 75-19
>>>>>>>>>>>>>> E-Mail: fischer@dias.de
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Anschrift: Schulterblatt 36, D-20357 Hamburg
>>>>>>>>>>>>>> Amtsgericht Hamburg HRB 58 167
>>>>>>>>>>>>>> Gesch?ftsf?hrer: Thomas Lilienthal, Michael Zapp
>>>>>>>>>>>>>> ----------------------------------------------------------
>> -----
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Shadi Abou-Zahra - http://www.w3.org/People/shadi/
>>>>> Activity Lead, W3C/WAI International Program Office
>>>>> Evaluation and Repair Tools Working Group (ERT WG)
>>>>> Research and Development Working Group (RDWG)
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> ---------------------------------------------------------------
>>>> Detlev Fischer PhD
>>>> DIAS GmbH - Daten, Informationssysteme und Analysen im Sozialen
>>>> Geschäftsführung: Thomas Lilienthal, Michael Zapp
>>>>
>>>> Telefon: +49-40-43 18 75-25
>>>> Mobile: +49-157 7-170 73 84
>>>> Fax: +49-40-43 18 75-19
>>>> E-Mail: fischer@dias.de
>>>>
>>>> Anschrift: Schulterblatt 36, D-20357 Hamburg
>>>> Amtsgericht Hamburg HRB 58 167
>>>> Geschäftsführer: Thomas Lilienthal, Michael Zapp
>>>> ---------------------------------------------------------------
>>>>
>>>>
>>>
>>
>>
>> --
>> ---------------------------------------------------------------
>> Detlev Fischer PhD
>> DIAS GmbH - Daten, Informationssysteme und Analysen im Sozialen
>> Geschäftsführung: Thomas Lilienthal, Michael Zapp
>>
>> Telefon: +49-40-43 18 75-25
>> Mobile: +49-157 7-170 73 84
>> Fax: +49-40-43 18 75-19
>> E-Mail: fischer@dias.de
>>
>> Anschrift: Schulterblatt 36, D-20357 Hamburg
>> Amtsgericht Hamburg HRB 58 167
>> Geschäftsführer: Thomas Lilienthal, Michael Zapp
>> ---------------------------------------------------------------
>
>
>

-- 
Shadi Abou-Zahra - http://www.w3.org/People/shadi/
Activity Lead, W3C/WAI International Program Office
Evaluation and Repair Tools Working Group (ERT WG)
Research and Development Working Group (RDWG)

Received on Wednesday, 26 October 2011 09:24:46 UTC