Re: 100% conformance for the pages sampled...

Hi Eric, all

I still don't agree with the section "Error margin". As pointed out: error margin a specific term. We should not use this term, when we speak about failures on pages.

To make clear what we mean, we should use something like "failures" or "violations..." as heading for this section. Nevertheless we also need a section "error margin", where we describe possible systematic errors like sampling errors. I remember that we discussed also the term "systematic error" just in the context of failures on pages. As this an evaluation methodology I believe we should be careful with terms and shouldn't use specific terms of testing theories and methodologies in general for anything else to avoid confusion.

Best

Kerstin

Am 27.01.2012 um 01:18 schrieb "Velleman, Eric" <evelleman@bartimeus.nl>:

> Hi Alistair,
> 
> The 100% conformance you refer to in your mail made it into the document in clause 5.5 Error Margin as we decided it: "The Methodology assumes a strict conformance requirement for all resources in the sample. I could change strict into 100%."
> 
> Also in the same section I added a disclaimer following the outcome of our discussion. We said that it is necessary to say that we cannot guarantee that there will not be another error somewhere on the site. This is caused by the use of a sample: "It is important to note that due to the use of a sample there is never a 100% guarantee that the full website is conformant."
> 
> I thought it might be good to explain that further and I propose to do that in the Random Resources clause (4.1.3). I already made a short link: "This specific error margin for a given confidence level is described in the section about Random Resources." We will discuss clause 4.1 next week (as agreed today), so we can then discuss if we want to keep that in the document or take it out again.
> 
> Kindest regards,
> 
> Eric
> 
> 
> ________________________________
> Van: Alistair Garrison [alistair.j.garrison@gmail.com]
> Verzonden: donderdag 26 januari 2012 15:50
> Aan: Eval TF
> Onderwerp: Fwd: 100% conformance for the pages sampled...
> 
> Dear All,
> 
> The concept of "100% conformance for the pages sampled" gained a reasonable level of consensus amongst EVAL TF members (6 or 7 people).  Several other ideas have seemingly gained the same group approval, but then fallen by the wayside as we've move ahead to debate something else...
> 
> I'd like to know, what mechanism are we following to capture such ideas, and ensure that they are appear in 'black & white' in our methodology updates?
> 
> All the best
> 
> Alistair
> 
> Begin forwarded message:
> 
> From: Alistair Garrison <alistair.j.garrison@gmail.com<mailto:alistair.j.garrison@gmail.com>>
> Date: 20 January 2012 14:10:20 CET
> To: Léonie Watson <lwatson@nomensa.com<mailto:lwatson@nomensa.com>>, Eval TF <public-wai-evaltf@w3.org<mailto:public-wai-evaltf@w3.org>>
> Subject: Re: 100% conformance for the pages sampled...
> 
> Hi Léonie,
> 
>> From http://www.w3.org/TR/WCAG/#conformance-claims - "Conformance is not possible at a particular level if any page in the process does not conform at that level or better".
> 
> And, as an additional component of the conformance claim "A list of success criteria beyond the level of conformance claimed that have been met. This information should be provided in a form that users can use, preferably machine-readable metadata."
> 
> So, I would suggest that for our purposes (those of evaluating the whole site) this would mean - making a level AA conformance claim for all pages in the sample (website).  If required by the website owner (or other) it could be provided along with a list of urls for the web pages on which the additional AAA Success Criteria have been achieved.
> 
> If we were not looking at the whole site separate AA and AAA claims for the relevant pages would probably be the way to go.  I should add, that a AAA conformance claim could only be made for the pages which were AAA conformant - ruling out, in my opinion, a AAA with exceptions type claim.
> 
> Let me know what you think.
> 
> All the best
> 
> Alistair
> 
> On 20 Jan 2012, at 12:59, Léonie Watson wrote:
> 
>    Thanks Alastair, it does. Although it's started me thinking more about this. Given that WCAG acknowledges that Level AAA is not possible with all technologies, we could end up with a sample that is partly Level AA compliant and partly Level AAA compliant.
> 
>    I'm thinking out loud here as much as anything, but would we therefore consider the sample to be 100% Level AA compliant with additional achievements, Level AAA compliant with exceptions, or part Level AA and part Level AAA as applicable?
> 
> 
> Léonie.
> 
> --
> Nomensa - humanising technology
> 
> Léonie Watson, Director of Accessibility & Web Development
> tel: +44 (0)117 929 7333
> mob: +44 (0)792 116 8551
> twitter: @we_are_Nomensa @LeonieWatson
> 
> Nomensa Email Disclaimer: http://www.nomensa.com/email-disclaimer
> 
> 
> 
> ________________________________
> From: Alistair Garrison [mailto:alistair.j.garrison@gmail.com]
> Sent: 20 January 2012 09:38
> To: Eval TF
> Subject: Re: 100% conformance for the pages sampled...
> 
> Hi Leonie,
> 
>> From http://www.w3.org/TR/WCAG/#conformance-claims - "Conformance is defined only for Web pages. However, a conformance claim may be made to cover one page, a series of pages, or multiple related Web pages."
> 
> Where they say "for Web Pages" I think they meant to say "for full Web Pages" - referencing "Conformance (and conformance level) is for full Web page(s) only, and cannot be achieved if part of a Web page is excluded." from the same page.
> 
> Hope this clarifies things...
> 
> All the best
> 
> Alistair
> 
> On 20 Jan 2012, at 10:14, Léonie Watson wrote:
> 
> If I understand correctly, WCAG conformance relates to a single page, not to an entire website. I'd be interested in people's thoughts on how this might (or might not) influence our thinking about an error margin?
> 
> 
> Léonie.
> 
> --
> Nomensa - humanising technology
> 
> Léonie Watson, Director of Accessibility & Web Development
> tel: +44 (0)117 929 7333
> mob: +44 (0)792 116 8551
> twitter: @we_are_Nomensa @LeonieWatson
> 
> Nomensa Email Disclaimer: http://www.nomensa.com/email-disclaimer
> 
> -----Original Message-----
> From: Detlev Fischer [mailto:fischer@dias.de]
> Sent: 19 January 2012 21:58
> To: public-wai-evaltf@w3.org<mailto:public-wai-evaltf@w3.org>
> Subject: Re: 100% conformance for the pages sampled...
> 
> Let's stop here and consider the implications.
> 
> Here and then, people in the EVAL TF have agreed that the 100% conformant site does not really exist 'out there'. Aren't we holding the bone a wee bit too high? I wonder what that will mean for the practical acceptance of the methodology. Will it come to be derided as academic, as impossibly demanding? Who then is the customer of a (sorry, chap) refused seal of conformance who bows to gracefully accept the list of flaws to rectify? Just wondering...it just strikes me as slightly surreal...
> 
> Detlev
> 
> 
> 
> Quoting RichardWarren <richard.warren@userite.com<mailto:richard.warren@userite.com>>:
> 
> Dear Alistair and All,
> 
> Having just spent a fortune getting my son's car through its MOT I
> have to agree with Alistair 100%. Our task is to establish a
> methodology for evaluating website accessibility. If the evaluation
> identifies that the site fully meets the guidelines then a conformance
> claim can be made to that effect. Everyone will know exactly what that
> means.
> 
> If the site "almost" meets the guidelines then perhaps some other form
> of "compliance statement" can be made - BUT that is not our current
> problem. Maybe, once we have finished our methodology, we can
> recommend a new task force to look at variance in conformance claims
> <grin>.
> 
> 
> Regards
> Richard
> 
> 
> 
> -----Original Message----- From: Alistair Garrison
> Sent: Thursday, January 19, 2012 9:02 PM
> To: Eval TF
> Subject: 100% conformance for the pages sampled...
> 
> Dear All,
> 
> If I understood correctly from this afternoon's EVAL TF telecon -
> there was a suggestion that we should (at a minimum) require the
> representative sample pages to be in 100% conformance with WCAG 2.0
> (at the chosen level) in order to say the site conforms (at that
> level).  If this was the case, I strongly agree with it (meant to
> write it in the IRC at the time).
> 
> In addition, I noted from some a worry about telling a website owner
> (a client, etc) that their website doesn't conform - especially when
> they might have tried hard to do so.  To my mind, worries of this kind
> should not deter us from asking for nothing less than 100% conformance
> (on any given sample). The person that does the MOT on my car has
> absolutely no worries about telling me about any failures, but
> possibly that's because everyone doing MOTs requires 100% conformance
> from a car for a pass.
> 
> Surely, we want people to try their absolute best to conform 100%.
> We must encourage them to shoot for the stars (100% conformance) -
> some, of course, will initially only hit the moon, but they will at
> least know what is expected from them... Let's not, however, start to
> congratulate people for simply getting off the ground - that time must
> have passed long, long, long ago.
> 
> Anyway, look forward to seeing you all on the list.
> 
> Alistair
> 
> 
> 
> --
> ---------------------------------------------------------------
> Detlev Fischer PhD
> DIAS GmbH - Daten, Informationssysteme und Analysen im Sozialen
> Geschäftsführung: Thomas Lilienthal, Michael Zapp
> 
> Telefon: +49-40-43 18 75-25
> Mobile: +49-157 7-170 73 84
> Fax: +49-40-43 18 75-19
> E-Mail: fischer@dias.de<mailto:fischer@dias.de>
> 
> Anschrift: Schulterblatt 36, D-20357 Hamburg Amtsgericht Hamburg HRB 58 167
> Geschäftsführer: Thomas Lilienthal, Michael Zapp
> ---------------------------------------------------------------
> 
> 
> 
> 
> 
>

Received on Friday, 27 January 2012 07:45:27 UTC