Fwd: Step 1.e: Define the Techniques to be used

Dear All, 

Would it be possible to add my comments about Step 1.e to the comments document - http://www.w3.org/WAI/ER/conformance/comments

Begin forwarded message:

> From: Alistair Garrison <alistair.j.garrison@gmail.com>
> Subject: Step 1.e: Define the Techniques to be used
> Date: 10 May 2012 10:48:41 CEST
> To: Eval TF <public-wai-evaltf@w3.org>
> 
> Dear All, 
> 
> "Step 1.e: Define the Techniques to be used" - could we consider making this step non-optional?
> 
> The first reason being that we really need to check their implementation of the techniques (W3C, their own code of best practice or whatever) they say they use.
> 
> For example: 
> 
> - Case 1) If they have done something by using technique A, and we evaluate using technique B there could be an issue (they might fail B);
> - Case 2) If they have done something by using technique A, and we evaluate using technique A and B there still could be an issue (they might fail B); 
> - Case 3) If they have done something by using technique A, and we evaluate using technique A - it seems to work.
> 
> The second reason being that testing seems only to be really replicable if we know what the techniques were they said they implemented - otherwise, two different teams could easily get two different results based on the cases above.
> 
> I would be interested to hear your thoughts.
> 
> Very best regards
> 
> Alistair
>