Re: Input for survey about random sampling

Hi, Eric and all,

Some more comments about the questionnaire about sampling:


General:

I don't understand very well why the questionnaire should be filled out 
more than once depending on question #01. The proposed activities could 
share the same sampling approach, so I would keep Q01 with checkboxes 
(for example, we perform both in-house and external evaluations, but the 
sampling method does not vary significantly. It depends much more on the 
goal or purpose of the evaluation, not the type).

Now, the introductory note seems to indicate that you should fill the 
questionnaire once per each different activity, which could be really 
tedious without offering new information. I would add soem clarification 
text to this note (if it is really needed):

"Note: *If your sampling approach varies for different activities, goals 
or purposes (or combinations of them)*, you can fill in the survey more 
than once using different choices in the first, second *and third* 
questions."

Or:

"Note: Some combinations of activities, goals and purposes defined in 
questions #01 to #03 might share the same sampling approach. If you use 
different sampling approaches depending on these, please consider 
filling out the survey more than once for each approach separately".

Doing this would also eliminate the need of some questions that are 
highly dependant on the sampling approach (see below).

For example, the sampling approach is very different if the 
activities/goals/purposes are: "In-house/external - Detailed/in-depth - 
In-depth analysis/Final check/Certification/Monitoring" or they are 
"In-house/External - Basic - Monitoring/Large-scale/Pre-sales/Training".

Note that wording of #01 to #03 would also require some slight changes.


Individual questions:


#01: I see that the "re-evaluate" option has been deleted. Is this 
correct? The sampling approach might vary depending on this. Anyway, if 
this option is kept, I also agree that "1 month" should be deleted. 
Depending on the site 1 month implies many changes (news, blogs, 
events...), while others would probably have no changes at all.


#02: Although I agree with the wording "Basic Evaluation", "Detalied 
Evaluation" (I will send a comment about it later), now the methodology 
uses a different one: "Basic Report" and "Detailed Report".


#03: I agree with Richard that some ranking is needed. We at Technosite 
perform almost all of the activities of this question in a day-to-day 
basis. It is really difficult to select only three of them.


#07: I'm not sure that "functionality" is the same as "processes". 
According to WCAG 2.0 a "process" is a sequence of pages that must be 
completed in a specific order, but some "functionalities" can be 
achieved or reached from different menus or using different methods. Are 
these "processes"?


#08: The parenthesis "(mobile)" at the beginning should be deleted, 
since the clarification text covers it and the parenthesis could be more 
confusing.


#09: Maybe add an option "Depending on the goal of the evaluation". This 
option could be ommited if the survey must be filled out separately 
depending on the goal. Or maybe an open option like in #10.


#11: Add an option "depends on...", since this can vary depending on 
many factors (in-house/external, goal, purpose...). This option could be 
ommited if the survey must be filled out separately.


#12: This is also highly dependant on the goal and purpose of the 
evaluation. I would add a "depends on..." or "varies" option (or ask to 
fill out separate surveys).

I would also change the order of #12 and #13, or add a note in #12 that 
refers to #13 for variable sample size.


#13: In addition to "technical" reasons, the goal and purpose of the 
evaluation may also influence.


#14: I would supress the parenthesis "(that has recently been evaluated 
and had failures)". Sometimes the evaluation is not "recent" (we need to 
define "recent", too) or the we re-evaluate a certified website to 
monitor if everything is still ok.


#15: I'm seeing a different between "evaluation of full websites" and 
"full evaluation of websites". Until I've read this question, I assumed 
that "Basic" evaluation only seeks to have an overall view of the 
accessibility of a website, so we don't need (in most cases) to do a 
complete sampling. However, a "full evaluation" seems to convey that the 
evaluation is always considering every possible aspect of the website. 
Is this correct? It is very important to clarify this.


#16: Also dependant on the purpose of the evaluation. It is different if 
we only need to do a large-scale, multi-site analysis than evaluating a 
large website to obtain a label.


#18: This can be quite subjective. I would change the wording to: "Do 
you consider that the results..."


#22: Depends also if it is a "fresh" evaluation or a re-evaluation, and 
on other factors (goal, purpose).


Hope this helps :)

Kind regards,
Ramón.
---
Ramón Corominas
Accessibility specialist
Technosite - Fundación ONCE
+34 91 121 0330

Received on Monday, 15 October 2012 13:15:43 UTC