Re: AW: Additional Point/Question: problem centered / page centered evaluation

Hi,
I agree with Vivienne and Kerstin that a problem-centred approach is best. 
Using problems means that you look at the whole website (if appropriate) and 
are able to identify particular groups that are disadvantaged by the issues 
found. This is much more useful if you want to compare sites. Explaining 
that 20% of pages are not accessible has less impact than saying that a 
particular group (keyboard users,  blind users etc.) cannot use the site.

You can still apply the problem-centred approach to single pages, templates 
or components.

Richard

-----Original Message----- 
From: Kerstin Probiesch
Sent: Saturday, September 03, 2011 12:08 PM
To: 'Eval TF'
Cc: 'Vivienne CONWAY'
Subject: AW: Additional Point/Question: problem centered / page centered 
evaluation

Hi Vivienne, hi TF,

> The question of problem-centered or page-centered approach is going to
> be tricky.

Yes. It is.

> Personally I favour the problem-centered approach as I am
> seldom assessing a single page.  Violations in my opinion, usually are
> duplicated across pages.  For example if there is a lack of keyboard
> access on a single page, that is likely to occur on most if not all
> pages.  Some websites are so huge, that a page-centered approach is
> simply not feasible.

Same opinion. When following a page-centered approach you have to document
the same problem on every page again and again. Keyboard access is a good
example for that, another one is SC 3.1.1. The more single pages you have,
the more the page-centered approach might be imo insufficient. Not only for
the accessibility statement/report/result but also for the steps a website
owner has to do or order to do. An example: you have a huge website with
hundreds or more single pages and - let me say - 10 or more categories. In
this case an evaluation guided by a problem-centered approach will find more
violations and according to my experiences you can e.g. find out if all/not
all/just some/or one online editor/s know how to organize the content (Hx,
lists,...) or how to deal with SC 1.1.1. In addition you can give specific
instructions.

Another problem of the page-centered approach is imo the following: You
_have_ to define a set of "typical" pages to test on every website as part
of the methodology. A website owner would complain if you test different
types. What happens in following situation: on one homepage (main page) you
have a video which violates the SC, on another website you also have a video
which violates the SC but the video is not on the main page and probably
also not on one of the defined set of pages according to the followed
page-centered approach? One can argue: in this case we are testing also the
video. But, what about 1.3.3, 1.4.1 and 1.3.1 and how to deal with
processes? Important questions I think are

- are accessible statements which are reliable and valid based upon X
(3,4,5,10?) pages possible and how many pages one have to test? This
corresponds with "R04: Replicability: different Web accessibility evaluators
who perform the same tests on the same site should get the same results
within a given tolerance."
- how avoid human errors during the process of page selection before
starting a particular test and during the test itself?

Nevertheless and even if I prefer a problem-centered approach, a
page-centered one is probably the only acceptible and pragmatical method
when you have to test a great amount of websites in a comparative study -
but here again I think we will have limitations with consequences for the
accessibility statement(s).

I suggest that we also discuss, if we as TF should give redommendations/
suggestions for different scenarios:

- testing one single page (test should include every SC on this single
page)?!
- testing a website (walkthrough for every single SC _and_ every single step
of a process, if existing)?!
- comparative studies (testing every SC on a defined set of "typical" pages)
?!
- (...)?!

And we also have to consider the conformity levels.

I'm thinking about if it might be helpful to collect possible scenarios to
see which approach might be the best for a given scenario resp. discuss if
we recommend one approach for all scenarios?

Last not least: Also the problem-centered approach might lead to problems.
In the page-centered approach a tester has finished the test when checked
all SCs for the collected single pages. In a problem-centered approach one
have to find other criterias otherwise the tester is "lost in violations"
depending on the quality of the website.


> On the other hand, a developer who is adding a single page and wanting
> to know if it meets accessibility criteria would be testing only that
> page.  Thoughts?

This is an interesting question. It can be that the added single page
belongs to an already tested website or not. If the added single belongs to
an already tested site, maybe the test is long ago. I think in case of an
added single page a test should include all SCs which indicates the
problem-centered approach.

I hope my points are clear. Sadly, I'm not very familiar in writing English
:-( practice will do, hopefully.

Regards

Kerstin


>
>
> Regards
>
> Vivienne L. Conway
> ________________________________________
> From: public-wai-evaltf-request@w3.org [public-wai-evaltf-
> request@w3.org] On Behalf Of Shadi Abou-Zahra [shadi@w3.org]
> Sent: Tuesday, 30 August 2011 5:05 PM
> To: Eval TF
> Subject: Fwd: Additional Point/Question: problem centered / page
> centered  evaluation
>
> -------- Original Message --------
> Subject: Additional Point/Question: problem centered / page centered
> evaluation
> Date: Tue, 30 Aug 2011 08:21:34 +0000
> From: Kerstin Probiesch <mail@barrierefreie-informationskultur.de>
> To: <public-wai-evaltf@w3.org>
>
> Dear Eval TF,
>
> I want to add the following question: should we recommend a problem
> centered
> or a page centered approach? In my opinion this question is not only
> essential for further discussions about single questions and topics
> (true/false, rankings, involve people with disabilities) but essential
> for
> the whole methodology.
>
> Regs
>
> Kerstin
>
>
> -------------------------------------
> Kerstin Probiesch - Freie Beraterin
> Barrierefreiheit, Social Media, Webkompetenz
> Kantstraße 10/19 | 35039 Marburg
> Tel.: 06421 167002
> E-Mail: mail@barrierefreie-informationskultur.de
> Web: http://www.barrierefreie-informationskultur.de
>
> XING: http://www.xing.com/profile/Kerstin_Probiesch
> Twitter: http://twitter.com/kprobiesch
> ------------------------------------
>
> This e-mail is confidential. If you are not the intended recipient you
> must not disclose or use the information contained within. If you have
> received it in error please return it to the sender via reply e-mail
> and delete any record of it from your system. The information contained
> within is not the opinion of Edith Cowan University in general and the
> University accepts no liability for the accuracy of the information
> provided.
>
> CRICOS IPC 00279B

Received on Saturday, 3 September 2011 11:43:45 UTC