AW: methodology and method

Hi Shadi, all,

can please someone from the stuff or the facilitator finally clarify if the goal of this TF is to develop just a rationale (justification) or a methodology? Methodology is defined as: "a guideline for solving a problem, with specific components such as phases, tasks, _methods_, techniques and tools". As written on http://www.w3.org/WAI/ER/2011/eval/#methodology (WCAG 2.0 Evaluation Methodology): "The purpose of this work is to develop an internationally harmonized methodology for evaluating the conformance of websites to WCAG 2.0." 

Best

Kerstin


> -----Ursprüngliche Nachricht-----
> Von: public-wai-evaltf-request@w3.org [mailto:public-wai-evaltf-
> request@w3.org] Im Auftrag von Detlev Fischer
> Gesendet: Mittwoch, 26. Oktober 2011 09:56
> An: public-wai-evaltf@w3.org
> Betreff: Re: methodology and method
> 
> Hi Shadi,
> 
> no, clearly there is no need to explain why accessibility an its
> evaluation is important, that's a given. In my view, the rationale (or
> methodology) explains anf justifies the logic or rationale of the
> hands-on method / approach specified, which in turn just covers the
> things that testers need to know to carry out an evaluation
> competently.
> 
> For example, the methodology would briefly state (with reference to
> WCAG
> conformance requirement 3) that complete processes must be accessible
> because the success of the completed interaction counts. It would then
> describe how a complete process is to be reflected in a page or page
> state selection, including, possibly, a statement that repetitive
> content (such as a navigation context that doesn't change across the
> process) can be ignored, or tested just once.
> To give an example what this would mean on the level of implemented
> testing applicaton: the BITV-Test offers the option for the test of
> selected elements, like a lightbox. This avoids the possible distortion
> of having repeatedly postive results for basically the same page /
> template.
> 
> The rationale might also explain why processes after selection should
> be
> assigned a priority depending on their importance for the successful
> use
> of a site, and that this priority might play a part in the tolerance
> metrics and the final verdict (conformant / not conformant). For
> Amazon,
> comleting a purchase at checkout would be crucial while writing a
> produict comment might be considered ancillary and much less crucial.
> The methodology must argue all these things, for the tester, they are
> of
> little practical value in the process.
> 
> Felle free to forward to the list if you think it might help others
> understand my argument...
> 
> Best regards,
> Detlev
> 
> Am 26.10.2011 09:24, schrieb Shadi Abou-Zahra:
> > Hi Detlev,
> >
> > Just to understand what you mean by "rationale": are you suggesting
> that
> > Eval TF develops a document explaining why it is important to
> evaluate
> > websites for accessibility? This is not in the initial plan.
> >
> > Best,
> > Shadi
> >
> >
> > On 25.10.2011 22:08, Detlev Fischer wrote:
> >> I think the rationale (methodology) needs to be well argued and
> >> referenced and cannot really be very short. It is just a type of
> >> document that has a different purpose and audience compared to a
> >> succinct procedural instruction. The latter should be optimised for
> the
> >> practitioner tackling a website and should have little or no
> theoretical
> >> ballast.
> >>
> >> Quoting Shadi Abou-Zahra <shadi@w3.org>:
> >>
> >>> Hi Detlev,
> >>>
> >>> I'm not sure what the others think but I'm not convinced we need
> two
> >>> documents. The "rationale, audience & scope, rationale of sampling"
> >>> sections should probably be very short to set the scene for the
> main
> >>> sections but maybe this is unclear from the table of contents
> alone.
> >>>
> >>> Best,
> >>> Shadi
> >>>
> >>>
> >>> On 25.10.2011 09:45, Detlev Fischer wrote:
> >>>> Hi Shadi, hi all,
> >>>>
> >>>> I made a suggestion a while ago to decouple the methodology
> (rationale)
> >>>> from the method.
> >>>>
> >>>> * Methodology: rationale, audience & scope, rationale of sampling
> >>>> and sampling methods, measurement of score / tolerances,
> >>>> references etc.
> >>>>
> >>>> * Method: a usable procedure for the actual testing of websites,
> e.g.:
> >>>> - make entry check (site applicable for testing)
> >>>> - select WCAG level applied (A, AA, AAA)
> >>>> - define page sample, select states and processes
> >>>> - run through procedure proper, with checkpoints based on WCAG
> >>>> success criteria and all checks needed to test whether the sites
> >>>> meets them - which in case of 1.1.1 and 1.3.1 probably means
> >>>> several checkpoints
> >>>> - generate report and, if applicable, conformance statement
> >>>>
> >>>> The reason is simply that the procedure gets lost in the draft
> Table of
> >>>> Contents. We also need to think from a point of view of the users
> of
> >>>> the
> >>>> method.
> >>>>
> >>>> Any comments?
> >>>>
> >>>> Detlev
> >>>>
> >>>> Am 18.10.2011 11:59, schrieb Shadi Abou-Zahra:
> >>>>> Hi Detlev,
> >>>>>
> >>>>> I believe we agreed that the Methodology would explain how to use
> WCAG
> >>>>> 2.0 Techniques for evaluating web page samples rather than to
> >>>>> duplicate
> >>>>> them or to create synthesized versions of them.
> >>>>>
> >>>>> If we go down that route we risk re-interpreting WCAG or running
> into
> >>>>> maintenance issues for chasing web technologies as they are
> released.
> >>>>>
> >>>>> I think we should, as far as possible, decouple the Methodology
> from
> >>>>> the
> >>>>> detailed "alt-text checking"-level; this is already covered by
> WCAG.
> >>>>>
> >>>>> Having said that, I can imagine improvements to the "How to Meet
> >>>>> WCAG2"
> >>>>> quick reference [1], to better address the need of evaluators
> rather
> >>>>> than developers. For example by organizing the Techniques
> thematically
> >>>>> rather than by Success Criteria. However, this would be separate
> work.
> >>>>>
> >>>>> [1] <http://www.w3.org/WAI/WCAG20/quickref/>
> >>>>>
> >>>>> Regards,
> >>>>> Shadi
> >>>>>
> >>>>>
> >>>>> On 18.10.2011 11:24, Detlev Fischer wrote:
> >>>>>> Am 18.10.2011 09:14, schrieb Shadi Abou-Zahra:
> >>>>>>> Hi Detlev,
> >>>>>>>
> >>>>>>> On 18.10.2011 08:39, Detlev Fischer wrote:
> >>>>>>>> Hi Shady,
> >>>>>>>>
> >>>>>>>> I am still not sure how we can clearly delineate the
> difference
> >>>>>>>> between
> >>>>>>>> the wider term "web content" (as in WCAG) and "website" as
> >>>>>>>> explained on
> >>>>>>>> - <http://www.w3.org/WAI/ER/methodology-reqs/#website>
> >>>>>>>
> >>>>>>> Agree, this is a challenge that we need to address.
> >>>>>>>
> >>>>>>>
> >>>>>>>> Maybe we should clearly state in the definition whether
> "website"
> >>>>>>>> includes any content offered via plugin technologies like
> Flash?
> >>>>>>>
> >>>>>>> As I understand it, "website" includes any web content as
> defined by
> >>>>>>> WCAG 2.0, which would include Flash and other web technologies.
> >>>>>>>
> >>>>>>>
> >>>>>>>> Thisa has practical implications. For example, if Flash would
> be
> >>>>>>>> included, the testing procedure would then either have to be
> *very*
> >>>>>>>> general (actually more general than in many tests of
> techniques,
> >>>>>>>> which
> >>>>>>>> do reference tools like aDesigner2) or fork depending on HTML,
> >>>>>>>> FLASH,
> >>>>>>>> etc.
> >>>>>>>
> >>>>>>> I'm not sure I understand this rationale. There are already
> WCAG 2.0
> >>>>>>> Techniques for Flash. Maybe not yet a complete set but enough
> to
> >>>>>>> show
> >>>>>>> how the Methodology would utilize existing WCAG 2.0 Techniques.
> >>>>>>
> >>>>>> I guess the procedure would either reference, as planned, tests
> in
> >>>>>> individual techniques (HTML, CSS, Flash, etc) which means it
> would be
> >>>>>> very bitty and carry a lot of redundancy, or it would offer a
> >>>>>> practical
> >>>>>> synthesis from a hands-on point of view and thereby, add
> practical
> >>>>>> value
> >>>>>> for testers. When checking for alt texts, for example, there
> will be
> >>>>>> many applicable techniques which are often mutually exclusive.
> If I
> >>>>>> use
> >>>>>> "list images" in WAT to trundle through the images of a web page
> and
> >>>>>> check whether alt texts descripe content (unlinked images),
> indicate
> >>>>>> link target or purpose (linked images) or are empty (decorative
> >>>>>> images),
> >>>>>> the economic procedure is to synthesize the assessment as you go
> >>>>>> through
> >>>>>> the list. No one would *actually* step through the atomic tests
> again
> >>>>>> and again. It would also be disruptive from an operational
> >>>>>> perspective.
> >>>>>> You want to have *one* page which tells you what to do and how
> to
> >>>>>> assess
> >>>>>> the instances you find (for some complex SC like 1.1.1 and
> 1.3.1,
> >>>>>> it may
> >>>>>> be several, of course).
> >>>>>>
> >>>>>> So back to my original rationale:
> >>>>>>
> >>>>>> Either the test procedure is generic and says "Check all images
> opn
> >>>>>> the
> >>>>>> page to verify that the alt attribute is present and
> appropriate"
> >>>>>> which
> >>>>>> hardly goes beyond the text of WCAG Guideline 1.1. and is
> >>>>>> therefore of
> >>>>>> questionable merit, or it would present some sort of decision
> tree
> >>>>>> which
> >>>>>> recommends a sequence or path through options and thereby
> synthesizes
> >>>>>> the atomic tests necessary to check for conformance to a
> particular
> >>>>>> SC.
> >>>>>>
> >>>>>> Now, if we deal with HTML/CSS/JS content but also FLASH content,
> this
> >>>>>> might then apear as a choice at some point the decision tree:
> you
> >>>>>> check
> >>>>>> if FLASH is used, fork to applicable Flash tests (or a procedure
> >>>>>> syntheizing tests), the return to check HTML (just as an
> example). It
> >>>>>> just makes an already complex HTML-oriented test procedure still
> more
> >>>>>> complex.
> >>>>>>
> >>>>>> Just as an aside: in BIT-Test we deal with this problem in a
> 'silo
> >>>>>> approach': we (will) offer separate WCAG-based tests for
> different
> >>>>>> media
> >>>>>> (HTML/CSS/JS, PDF, maybe some day Flash).
> >>>>>>
> >>>>>> I personally think (and I have said this before) that a test
> >>>>>> procedure
> >>>>>> that integrates atomic tests and offers advice regarding the
> >>>>>> success of
> >>>>>> failure of instances checked is the actual added value in
> practical
> >>>>>> terms.
> >>>>>>
> >>>>>> Detlev
> >>>>>>>
> >>>>>>> Best,
> >>>>>>> Shadi
> >>>>>>>
> >>>>>>>
> >>>>>>>> Regards,
> >>>>>>>> Detlev
> >>>>>>>>
> >>>>>>>> Am 17.10.2011 15:15, schrieb Shadi Abou-Zahra:
> >>>>>>>>> Dear Detlev, All,
> >>>>>>>>>
> >>>>>>>>> On 17.10.2011 10:15, Detlev Fischer wrote:
> >>>>>>>>>> I have now fully caught up with the discussion of the term
> >>>>>>>>>> "website"
> >>>>>>>>>> and
> >>>>>>>>>> I realise it can be (or has been) defined to include web
> apps
> >>>>>>>>>> etc.
> >>>>>>>>>
> >>>>>>>>> Correct:
> >>>>>>>>> - <http://www.w3.org/WAI/ER/methodology-reqs/#website>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> Still, since the term "website" has been avoided in the
> title of
> >>>>>>>>>> WCAG
> >>>>>>>>>> and also carries connotations of a traditional hierachical
> >>>>>>>>>> site, ist
> >>>>>>>>>> seems more congenial to me to use "Web content" instead
> (even
> >>>>>>>>>> if it
> >>>>>>>>>> makes the acronym still thornier).
> >>>>>>>>>
> >>>>>>>>> It is not about the acronym but rather about the scope of
> WCAG
> >>>>>>>>> versus
> >>>>>>>>> that of the Methodology. Web content is used to build
> websites.
> >>>>>>>>> WCAG
> >>>>>>>>> applies to any web content while our Methodology is limited
> to
> >>>>>>>>> websites.
> >>>>>>>>>
> >>>>>>>>> I think we should either use the term "website" to denote
> static
> >>>>>>>>> and
> >>>>>>>>> dynamic web pages, web applications, intranets, mobile sites,
> >>>>>>>>> etc, or
> >>>>>>>>> come up with another term that applies to websites in their
> >>>>>>>>> entirety.
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>> Shadi
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> Detlev
> >>>>>>>>>>
> >>>>>>>>>> Am 17.10.2011 08:40, schrieb kvotis@iti.gr:
> >>>>>>>>>>> Hello everyone
> >>>>>>>>>>>
> >>>>>>>>>>> i aggree with the following comment of Detlev and this was
> >>>>>>>>>>> also my
> >>>>>>>>>>> comment
> >>>>>>>>>>> to a previous mail. I cannot understand why we are
> refereeing
> >>>>>>>>>>> only to
> >>>>>>>>>>> Websites. I think that we need a more general term
> including
> >>>>>>>>>>> also
> >>>>>>>>>>> Web
> >>>>>>>>>>> applications, Mobile Web applications,etc...
> >>>>>>>>>>>
> >>>>>>>>>>> regards
> >>>>>>>>>>>
> >>>>>>>>>>> kostas
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi everyone,
> >>>>>>>>>>>>
> >>>>>>>>>>>> back from holidays, I am trying to catch up with
> discussions
> >>>>>>>>>>>> now.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I do not mind WAEM even though I do not think it is
> >>>>>>>>>>>> particularly
> >>>>>>>>>>>> easy
> >>>>>>>>>>>> to pronounce (other then spelling out double-U - E - A -
> M).
> >>>>>>>>>>>>
> >>>>>>>>>>>> I have some concern regarding the W for "website", as in
> the
> >>>>>>>>>>>> current
> >>>>>>>>>>>> full title
> >>>>>>>>>>>> "Website Accessibility Evaluation Methodology". The "WC"
> of
> >>>>>>>>>>>> WCAG
> >>>>>>>>>>>> sounds a lot more general than the term "website". This
> also
> >>>>>>>>>>>> relates
> >>>>>>>>>>>> to the important comments by Aaron Leventhal regarding the
> >>>>>>>>>>>> scope of
> >>>>>>>>>>>> the methodology, tool independence, and the possible need
> to
> >>>>>>>>>>>> cover
> >>>>>>>>>>>> different web content and UA scenarios.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Is what we develop here focused on websites and not on
> things
> >>>>>>>>>>>> like
> >>>>>>>>>>>> web
> >>>>>>>>>>>> apps for mobile UAs? If we want to be as general as WCAG
> we
> >>>>>>>>>>>> might
> >>>>>>>>>>>> need
> >>>>>>>>>>>> to change the full name of the methodology and
> accordingly, the
> >>>>>>>>>>>> acronym.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Regards,
> >>>>>>>>>>>> Detlev
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Zitat von "Velleman, Eric"<evelleman@bartimeus.nl>:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Dear all,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> We decided in the call to discuss the short name in the
> list.
> >>>>>>>>>>>>> Below
> >>>>>>>>>>>>> are some short names from the discussions we had sofar
> and a
> >>>>>>>>>>>>> proposed shortlist of requirements:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Requirements for a short name:
> >>>>>>>>>>>>> - It should be short
> >>>>>>>>>>>>> - Easy to pronounce
> >>>>>>>>>>>>> - Clear to all what it means (selfexplanatory)
> >>>>>>>>>>>>> - Be about the methodology
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Shortnames (Make your choice or add new ones):
> >>>>>>>>>>>>> - SiteAccess
> >>>>>>>>>>>>> - WCAG-Method
> >>>>>>>>>>>>> - WCAG-EM
> >>>>>>>>>>>>> - WAEM (really using the title)
> >>>>>>>>>>>>> - WCAG-Check
> >>>>>>>>>>>>> - AccessSite
> >>>>>>>>>>>>> - WCAG-Site
> >>>>>>>>>>>>> - AccessCheck
> >>>>>>>>>>>>> - SiteCheck
> >>>>>>>>>>>>> - CheckSite
> >>>>>>>>>>>>> - WAMBAM
> >>>>>>>>>>>>> - WAME
> >>>>>>>>>>>>> - MCEWA
> >>>>>>>>>>>>> - CEWA
> >>>>>>>>>>>>> - CEW2WCAG2
> >>>>>>>>>>>>> - UWEM
> >>>>>>>>>>>>> - SITE
> >>>>>>>>>>>>> - MAWA
> >>>>>>>>>>>>> - MDWC
> >>>>>>>>>>>>> - EWAMAC
> >>>>>>>>>>>>> - EMAWC
> >>>>>>>>>>>>> - WEM
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Kindest regards,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Eric
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> ----------------------------------------------------------
> -----
> >>>>>>>>>>>> Detlev Fischer PhD
> >>>>>>>>>>>> DIAS GmbH - Daten, Informationssysteme und Analysen im
> Sozialen
> >>>>>>>>>>>> Gesch?ftsf?hrung: Thomas Lilienthal, Michael Zapp
> >>>>>>>>>>>>
> >>>>>>>>>>>> Telefon: +49-40-43 18 75-25
> >>>>>>>>>>>> Mobile: +49-157 7-170 73 84
> >>>>>>>>>>>> Fax: +49-40-43 18 75-19
> >>>>>>>>>>>> E-Mail: fischer@dias.de
> >>>>>>>>>>>>
> >>>>>>>>>>>> Anschrift: Schulterblatt 36, D-20357 Hamburg
> >>>>>>>>>>>> Amtsgericht Hamburg HRB 58 167
> >>>>>>>>>>>> Gesch?ftsf?hrer: Thomas Lilienthal, Michael Zapp
> >>>>>>>>>>>> ----------------------------------------------------------
> -----
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>> --
> >>> Shadi Abou-Zahra - http://www.w3.org/People/shadi/
> >>> Activity Lead, W3C/WAI International Program Office
> >>> Evaluation and Repair Tools Working Group (ERT WG)
> >>> Research and Development Working Group (RDWG)
> >>>
> >>>
> >>
> >>
> >>
> >> --
> >> ---------------------------------------------------------------
> >> Detlev Fischer PhD
> >> DIAS GmbH - Daten, Informationssysteme und Analysen im Sozialen
> >> Geschäftsführung: Thomas Lilienthal, Michael Zapp
> >>
> >> Telefon: +49-40-43 18 75-25
> >> Mobile: +49-157 7-170 73 84
> >> Fax: +49-40-43 18 75-19
> >> E-Mail: fischer@dias.de
> >>
> >> Anschrift: Schulterblatt 36, D-20357 Hamburg
> >> Amtsgericht Hamburg HRB 58 167
> >> Geschäftsführer: Thomas Lilienthal, Michael Zapp
> >> ---------------------------------------------------------------
> >>
> >>
> >
> 
> 
> --
> ---------------------------------------------------------------
> Detlev Fischer PhD
> DIAS GmbH - Daten, Informationssysteme und Analysen im Sozialen
> Geschäftsführung: Thomas Lilienthal, Michael Zapp
> 
> Telefon: +49-40-43 18 75-25
> Mobile: +49-157 7-170 73 84
> Fax: +49-40-43 18 75-19
> E-Mail: fischer@dias.de
> 
> Anschrift: Schulterblatt 36, D-20357 Hamburg
> Amtsgericht Hamburg HRB 58 167
> Geschäftsführer: Thomas Lilienthal, Michael Zapp
> ---------------------------------------------------------------

Received on Wednesday, 26 October 2011 08:56:53 UTC