RE: 100% conformance for the pages sampled...

Hi Alistair,

The 100% conformance you refer to in your mail made it into the document in clause 5.5 Error Margin as we decided it: "The Methodology assumes a strict conformance requirement for all resources in the sample. I could change strict into 100%."

Also in the same section I added a disclaimer following the outcome of our discussion. We said that it is necessary to say that we cannot guarantee that there will not be another error somewhere on the site. This is caused by the use of a sample: "It is important to note that due to the use of a sample there is never a 100% guarantee that the full website is conformant."

I thought it might be good to explain that further and I propose to do that in the Random Resources clause (4.1.3). I already made a short link: "This specific error margin for a given confidence level is described in the section about Random Resources." We will discuss clause 4.1 next week (as agreed today), so we can then discuss if we want to keep that in the document or take it out again.

Kindest regards,

Eric


________________________________
Van: Alistair Garrison [alistair.j.garrison@gmail.com]
Verzonden: donderdag 26 januari 2012 15:50
Aan: Eval TF
Onderwerp: Fwd: 100% conformance for the pages sampled...

Dear All,

The concept of "100% conformance for the pages sampled" gained a reasonable level of consensus amongst EVAL TF members (6 or 7 people).  Several other ideas have seemingly gained the same group approval, but then fallen by the wayside as we've move ahead to debate something else...

I'd like to know, what mechanism are we following to capture such ideas, and ensure that they are appear in 'black & white' in our methodology updates?

All the best

Alistair

Begin forwarded message:

From: Alistair Garrison <alistair.j.garrison@gmail.com<mailto:alistair.j.garrison@gmail.com>>
Date: 20 January 2012 14:10:20 CET
To: Léonie Watson <lwatson@nomensa.com<mailto:lwatson@nomensa.com>>, Eval TF <public-wai-evaltf@w3.org<mailto:public-wai-evaltf@w3.org>>
Subject: Re: 100% conformance for the pages sampled...

Hi Léonie,

>From http://www.w3.org/TR/WCAG/#conformance-claims - "Conformance is not possible at a particular level if any page in the process does not conform at that level or better".

And, as an additional component of the conformance claim "A list of success criteria beyond the level of conformance claimed that have been met. This information should be provided in a form that users can use, preferably machine-readable metadata."

So, I would suggest that for our purposes (those of evaluating the whole site) this would mean - making a level AA conformance claim for all pages in the sample (website).  If required by the website owner (or other) it could be provided along with a list of urls for the web pages on which the additional AAA Success Criteria have been achieved.

If we were not looking at the whole site separate AA and AAA claims for the relevant pages would probably be the way to go.  I should add, that a AAA conformance claim could only be made for the pages which were AAA conformant - ruling out, in my opinion, a AAA with exceptions type claim.

Let me know what you think.

All the best

Alistair

On 20 Jan 2012, at 12:59, Léonie Watson wrote:

    Thanks Alastair, it does. Although it's started me thinking more about this. Given that WCAG acknowledges that Level AAA is not possible with all technologies, we could end up with a sample that is partly Level AA compliant and partly Level AAA compliant.

    I'm thinking out loud here as much as anything, but would we therefore consider the sample to be 100% Level AA compliant with additional achievements, Level AAA compliant with exceptions, or part Level AA and part Level AAA as applicable?


Léonie.

--
Nomensa - humanising technology

Léonie Watson, Director of Accessibility & Web Development
tel: +44 (0)117 929 7333
mob: +44 (0)792 116 8551
twitter: @we_are_Nomensa @LeonieWatson

Nomensa Email Disclaimer: http://www.nomensa.com/email-disclaimer



________________________________
From: Alistair Garrison [mailto:alistair.j.garrison@gmail.com]
Sent: 20 January 2012 09:38
To: Eval TF
Subject: Re: 100% conformance for the pages sampled...

Hi Leonie,

>From http://www.w3.org/TR/WCAG/#conformance-claims - "Conformance is defined only for Web pages. However, a conformance claim may be made to cover one page, a series of pages, or multiple related Web pages."

Where they say "for Web Pages" I think they meant to say "for full Web Pages" - referencing "Conformance (and conformance level) is for full Web page(s) only, and cannot be achieved if part of a Web page is excluded." from the same page.

Hope this clarifies things...

All the best

Alistair

On 20 Jan 2012, at 10:14, Léonie Watson wrote:

If I understand correctly, WCAG conformance relates to a single page, not to an entire website. I'd be interested in people's thoughts on how this might (or might not) influence our thinking about an error margin?


Léonie.

--
Nomensa - humanising technology

Léonie Watson, Director of Accessibility & Web Development
tel: +44 (0)117 929 7333
mob: +44 (0)792 116 8551
twitter: @we_are_Nomensa @LeonieWatson

Nomensa Email Disclaimer: http://www.nomensa.com/email-disclaimer

-----Original Message-----
From: Detlev Fischer [mailto:fischer@dias.de]
Sent: 19 January 2012 21:58
To: public-wai-evaltf@w3.org<mailto:public-wai-evaltf@w3.org>
Subject: Re: 100% conformance for the pages sampled...

Let's stop here and consider the implications.

Here and then, people in the EVAL TF have agreed that the 100% conformant site does not really exist 'out there'. Aren't we holding the bone a wee bit too high? I wonder what that will mean for the practical acceptance of the methodology. Will it come to be derided as academic, as impossibly demanding? Who then is the customer of a (sorry, chap) refused seal of conformance who bows to gracefully accept the list of flaws to rectify? Just wondering...it just strikes me as slightly surreal...

Detlev



Quoting RichardWarren <richard.warren@userite.com<mailto:richard.warren@userite.com>>:

Dear Alistair and All,

Having just spent a fortune getting my son's car through its MOT I
have to agree with Alistair 100%. Our task is to establish a
methodology for evaluating website accessibility. If the evaluation
identifies that the site fully meets the guidelines then a conformance
claim can be made to that effect. Everyone will know exactly what that
means.

If the site "almost" meets the guidelines then perhaps some other form
of "compliance statement" can be made - BUT that is not our current
problem. Maybe, once we have finished our methodology, we can
recommend a new task force to look at variance in conformance claims
<grin>.


Regards
Richard



-----Original Message----- From: Alistair Garrison
Sent: Thursday, January 19, 2012 9:02 PM
To: Eval TF
Subject: 100% conformance for the pages sampled...

Dear All,

If I understood correctly from this afternoon's EVAL TF telecon -
there was a suggestion that we should (at a minimum) require the
representative sample pages to be in 100% conformance with WCAG 2.0
(at the chosen level) in order to say the site conforms (at that
level).  If this was the case, I strongly agree with it (meant to
write it in the IRC at the time).

In addition, I noted from some a worry about telling a website owner
(a client, etc) that their website doesn't conform - especially when
they might have tried hard to do so.  To my mind, worries of this kind
should not deter us from asking for nothing less than 100% conformance
(on any given sample). The person that does the MOT on my car has
absolutely no worries about telling me about any failures, but
possibly that's because everyone doing MOTs requires 100% conformance
from a car for a pass.

Surely, we want people to try their absolute best to conform 100%.
We must encourage them to shoot for the stars (100% conformance) -
some, of course, will initially only hit the moon, but they will at
least know what is expected from them... Let's not, however, start to
congratulate people for simply getting off the ground - that time must
have passed long, long, long ago.

Anyway, look forward to seeing you all on the list.

Alistair



--
---------------------------------------------------------------
Detlev Fischer PhD
DIAS GmbH - Daten, Informationssysteme und Analysen im Sozialen
Geschäftsführung: Thomas Lilienthal, Michael Zapp

Telefon: +49-40-43 18 75-25
Mobile: +49-157 7-170 73 84
Fax: +49-40-43 18 75-19
E-Mail: fischer@dias.de<mailto:fischer@dias.de>

Anschrift: Schulterblatt 36, D-20357 Hamburg Amtsgericht Hamburg HRB 58 167
Geschäftsführer: Thomas Lilienthal, Michael Zapp
---------------------------------------------------------------

Received on Friday, 27 January 2012 00:41:14 UTC