Comments on responses to test of website #1

To help discussions I give below my personal view of the results from testing steps 1 – 3 on website#1

For step 1 – Define the scope – the four completed results where reasonably similar which should give us confidence that the guidelines for this step are quite possibly adequate to produce consistent results. However in the real world we would expect to have some communication with the client during this step – if only to confirm the scope before proceeding. For future tests I suggest that the question includes more information, perhaps including things such as a statement about the purpose of the site and explanations as to what to include/exclude .

For step 2 – Explore the target site – The sample is probably too small to be meaningful (only one tester reported all four steps). We need to find out why people did not complete the report for this step (was it the question or the methodology that caused the lack of response?)

For step 3 – Select a representative sample – two testers select the whole site (each by a different route) and one selected a sample. The implication, I believe, is that we need to provide better guidance about the advantages/disadvantages of sampling and how to decide when it is probably more viable to test the whole site.


Regards
Richard


Richard Warren
Technical Manager
Website Auditing Limited (Userite)
http://www.website-accessibility.com