- From: Alistair Garrison <alistair.j.garrison@gmail.com>
- Date: Thu, 10 May 2012 12:53:41 +0200
- To: Detlev Fischer <detlev.fischer@testkreis.de>, Eval TF <public-wai-evaltf@w3.org>
Hi Detlev, I asked the question for a good reason… If I develop web content according to a set of techniques which I believe produces accessible content - but, which does not relate to your checks… what happens? My web content might not do so well in your evaluation - it may even fail... In reality, however, it might have only failed because I have not implemented the techniques you test for… On the flip side, we could say that only people who adopt the techniques you have in mind when testing will do well in your evaluation… To my mind, through the use of pre-defined checklists (such as BITV-TEST) we are accidentally imposing a set of techniques on web developers - and, by proxy defeating the W3C object of allowing people to use different techniques to create accessible content… In fact, if we all stick to our pre-defined checklists (in which we probably all have different things we check for) we will never be able to have a properly reproducible evaluation. The point of my previous request was to ensure that we test the actual techniques used to develop the web content - to try and remove the issue described above and increase the chance of reproducible results… P.s. It would of course be much simpler for everyone if the web developers would stick to using the W3C Sufficient Techniques and Failure Conditions. All the best Alistair On 10 May 2012, at 12:14, Detlev Fischer wrote: > Hi Alistair, > > Your answer does not address the problems I listed. > > Regarding your question: We use a web-based application, BITV-Test, which is based on 50 publicly documented checkpoints which reference techniques, but consolidate the often similar tests in techniques to achieve an efficient testing workflow. > > The application itself then has a page per checkpoint where you can record your ratings and any comments you have for all the pages in the sample. > > Unfortunately the application is in German but this may give you an idea of one checkpoint description (one of the longest): > http://testen.bitvtest.de/index.php?a=di&iid=12&s=n > (This is the checkpoint for checking whether linked images have adequate alternative text.) > > Regards, > Detlev > > > > > On 10 May 2012, at 11:45, Alistair Garrison wrote: > >> Hi Detlev, >> >> If you don't know what techniques have been followed, or are not interested to know, what are you actually evaluating against? >> >> Can I just check - do you evaluate against a checklist for WCAG 2.0 which you have developed? or is it something else? >> >> All the best >> >> Alistair >> >> On 10 May 2012, at 11:42, Detlev Fischer wrote: >> >>> Hi Alistir, >>> >>> If a commissioner says: I have used this new technique for, say, skipping blocks, or displaying lightboxes, it certainly makes sense to report back on the success of that particular technique. >>> >>> However, I see several problems making this step mandatory: >>> >>> * In some cases, evaluators will have no access to the authors of the site under test >>> >>> * Where do you stop? There are hundreds of techniques. Which ones should be defined? >>> >>> * Many (most) implementations are similar to the bare-bones WCAG techniques, but rarely exactly the same. Mapping adapted techniques to WCAG Techniques reliably will be tricky. >>> >>> * Advanced script-based techniques are very difficult to check. We can look at the page and check whether, say, a dynamically inserted element receives keyboard focus or is hidden automatically once the kb focus leaves it. But do we really need to dive into the script to see how this has been implemented? (maybe this is not what you meant) >>> >>> I think it may be useful to tick off techniques if it is obvious that they have been used (successfully or unsuccessfully), and especially, tick off failures when they clearly apply (because this proves that a SC has not been met in all cases, without disclaimer that some other technique might have been used). HOWEVER, identifying ALL techniques used during an evaluation seems a high burden. I can"t quite see the benefit. >>> >>> Regards, >>> Detlev >>> >>> On 10 May 2012, at 10:48, Alistair Garrison wrote: >>> >>>> Dear All, >>>> >>>> "Step 1.e: Define the Techniques to be used" - could we consider making this step non-optional? >>>> >>>> The first reason being that we really need to check their implementation of the techniques (W3C, their own code of best practice or whatever) they say they use. >>>> >>>> For example: >>>> >>>> - Case 1) If they have done something by using technique A, and we evaluate using technique B there could be an issue (they might fail B); >>>> - Case 2) If they have done something by using technique A, and we evaluate using technique A and B there still could be an issue (they might fail B); >>>> - Case 3) If they have done something by using technique A, and we evaluate using technique A - it seems to work. >>>> >>>> The second reason being that testing seems only to be really replicable if we know what the techniques were they said they implemented - otherwise, two different teams could easily get two different results based on the cases above. >>>> >>>> I would be interested to hear your thoughts. >>>> >>>> Very best regards >>>> >>>> Alistair >>>> >>> >>> -- >>> Detlev Fischer >>> testkreis - das Accessibility-Team von feld.wald.wiese >>> c/o feld.wald.wiese >>> Borselstraße 3-7 (im Hof) >>> 22765 Hamburg >>> >>> Tel +49 (0)40 439 10 68-3 >>> Mobil +49 (0)1577 170 73 84 >>> Fax +49 (0)40 439 10 68-5 >>> >>> http://www.testkreis.de >>> Beratung, Tests und Schulungen für barrierefreie Websites >>> >>> >>> >> > > -- > Detlev Fischer > testkreis - das Accessibility-Team von feld.wald.wiese > c/o feld.wald.wiese > Borselstraße 3-7 (im Hof) > 22765 Hamburg > > Tel +49 (0)40 439 10 68-3 > Mobil +49 (0)1577 170 73 84 > Fax +49 (0)40 439 10 68-5 > > http://www.testkreis.de > Beratung, Tests und Schulungen für barrierefreie Websites > > >
Received on Thursday, 10 May 2012 10:54:19 UTC