Re: Following properly documented existing procedures - A hypothetical question?

Hi Detlev, 

You have to remember that WCAG-EM "is primarily designed for evaluating existing websites, for example to learn about and monitor their level of accessibility…"

My question is hypothetical, but I think it also represents a realistic case.  Let's pretend I'm the owner of a Government website in some country (lets say some European State).  Is it not conceivable that a third-party (for example a European QA department) could use WCAG-EM to check my web pages without my consent?   

What I was driving at is - if I say I conform when I check using my procedure, what happens if they say I do not conform using their third-party one - as they simple check different things?

Would it not be better to say that they have to follow my procedure, if I have documented it in line with WCAG-EM?

Hope this clarifies.

All the best 

Alistair

On 18 Nov 2013, at 20:23, Detlev Fischer wrote:

> Hi Alistair,
> 
> Is the scene you sketch here (below) a realistic one? I.e., will you ever be asked to poke around a site that has published conformance claims for individual pages to prove the site owner wrong? I would imagine interest groups and organisations would much rather test and then shame or sue site owners who clearly violate accessibility best practices, not those sites that actually made the effort to test and document conformance claims?
> 
> In any case (as I keep repeating) there is enough wiggle room on the level of operationalized testing for any two testers to arrive at different results when testing content against particular WCAG techniques. 
> 
> Take the headings techniques G141 and H42 (http://www.w3.org/TR/WCAG-TECHS/G141.html and http://www.w3.org/TR/WCAG-TECHS/H42.html). While you may think it is clearly implied here that headings should be used to reflect the hierarchy, checks in both techniques refrain from requing correct nesting, and F2 has a somewhat ambiguous check ("Check that the proper semantic structure (e.g., HTML headings) is used with the text to convey the information") which can be interpreted both ways (A) just check headings are marked up with h1-h6 regardless of flawless hierarchy, or (B) checking any heading mark-up in the context of the overall structure (including content headed).
> 
> As we know, WCAG-EM refrains from specifying *how* to test, so your WCAG-EM evaluator will have to make up his or her mind how to operationalize on the practical level (not necessarily following the same procedure as the one used by the organisation making the conformance claim). By the same token, checks according to WCAG-EM will never be strictly repeatable UNLESS you follow a highly operationalized and well documented procedure which is clearly not part of WCAG-EM. (And even then, there remains ample wiggle room in many SCs, as I know from the experience of tandem-testing sites according to the same operationalized procedure).
> 
> Finally, even if your own different testing procedure reveals failures of Sufficient Techniques, this will be no more than an *indication* of a SC failure since the disclaimer under each and every test in a sufficient technique makes it clear that conformance to the SC *may* have been reached in some other way - which you'd need to sniff out if you wanted to dispute the claim. It seems that using Failure Techniques would be the only way to safely dispute a claim (but see F2 which did nit help in the example above). But then again -- is that really a realistic scenario?
> 
> Best,
> Detlev
> 
> On 18 Nov 2013, at 17:54, Alistair Garrison wrote:
> 
>> Dear All, 
>> 
>> A hypothetical question.
>> 
>> Scene-setting:
>> 
>> A qualified person has evaluated a single web page and has made a report - properly documenting all things asked for in the WCAG-EM.  They tested the sufficient techniques used by the developers, and all relevant failure criteria. They have found no issues in the web content in the web page.  Based on their report (but as a separate additional activity to the WCAG-EM) they have gone on to make a proper WCAG 2.0 conformance claim for the single web page.
>> 
>> I have been asked to evaluate the web page above.  I find the claim, and the supporting evidence. 
>> 
>> My question is this - "Am I honour bound to follow the procedure they have documented?"  
>> 
>> The thought in my head is yes - that I should follow their procedure if it is properly documented.  I would of course check all relevant failure conditions, but if I didn't follow their procedure and started to test the page using tests from sufficient techniques I've chosen (which have not been used to develop the web content) I might find a failure or two - just because they have done things differently.
>> 
>> Any thoughts on the above would be good, as we might have to mention the necessity to follow properly documented existing procedures when re-evaluating web pages somewhere in our document.
>> 
>> All the best 
>> 
>> Alistair   
> 
> -- 
> Detlev Fischer
> testkreis - das Accessibility-Team von feld.wald.wiese
> c/o feld.wald.wiese
> Thedestraße 2
> 22767 Hamburg
> 
> Tel   +49 (0)40 439 10 68-3
> Mobil +49 (0)1577 170 73 84
> Fax   +49 (0)40 439 10 68-5
> 
> http://www.testkreis.de
> Beratung, Tests und Schulungen für barrierefreie Websites
> 
> 
> 
>

Received on Monday, 18 November 2013 21:05:43 UTC