Re: Input for survey about random sampling

Hi Eric,

Here is an attempted re-write your question for your consideration:

# For which purposes do you evaluate websites:
  - (checkbox) Provide in-depth analyses to guide website owners
  - (checkbox) Monitor the accessibility of a website over time
  - (checkbox) Final check before releasing or purchasing a website
  - (checkbox) Provide label or other certificate of conformance
  - (checkbox) Do large-scale evaluation of many websites
  - (checkbox) Other - please specify below (text)

# Do you select sample pages for evaluation based on:
  - (radio) Structured sampling only
  - (radio) Random sampling only
  - (radio) Both structured and random sampling

# Please specify any tools you use to help you select the sample, and 
how you use these tools for sampling. (text)

# If you use structured sampling, how do you select the individual web 
pages? (text)

# If you use random sampling, how do you select the individual web 
pages? (text)

# If you do evaluation for different purposes, how does the purpose of 
the evaluation influence your selection of sample pages? (text)

# How does the size of the website influence your selection? (text)

# How do you adjust your sampling approach for websites that are heavily 
template-driven? (text)

# How do you adjust your sampling approach for web applications? (text)

# Do you look out for widgets, snippets, and other web page components 
to avoid re-evaluating them every time they appear on a website? If so, 
how do you include them and the pages that they are integrated into the 
website? (text)

# How do you identify and sample the functionality (processes) provided 
on a website? (text)

# Do you select further sample pages based upon results from an initial 
selection (eg. to balance web page types or success criteria)? (text)

# Please feel free to provide further thoughts or comments about how you 
approach sampling. (text)


I wonder if we need to break some of these down even more so that we are 
asking ourselves very specific questions? Or maybe that will be a future 
iteration where we want to find out more details? Not sure.

Best,
   Shadi


On 2.10.2012 15:02, Velleman, Eric wrote:
> Dear EvalTF,
>
> In our last telco we discussed opening a group only survey about random sampling. This would then provide input to draft a first text.
>
> Please add questions to the list below that may be relevant to draft a first text about random sampling. Here are a few from the last telco and my own additions. Please mark them with YES or NO to delete or keep and add questions as you like:
>
> - How do you collect pages?
> - How do you collect pages in web-applications?
> - Is there a difference between large and small websites in the manner you collect the pages?
> - Is there a difference between large and small websites in the size of the sample?
> - Do you collect a random sample?
> - Do you think random sampling is necessary? and why?
> - What portion of your sample is randomly selected?
> - How do you collect randomly (do you count pages..)?
> - If you use tools for sampling?
> - Are tools sufficient to collect a sample?
> - Should the random sample be other pages than the ones already included in Step 3?
> - ...
> - ...
> - ...
>
> Sampling is now in our methodology in section 3.3 Step 3 Select a Representative Sample. The current version of the methodology lets you select a representative sample including Common Web Pages of the Website, Exemplar Instances of Web Pages, Other Relevant Web Pages and Complete Processes. Random sampling is not included.
>
> The endresult for this survey would be about 10 to 15 questions.
> We could set the max for answers to 100 words per question.
> Kindest regards,
>
> Eric
>
>
>

-- 
Shadi Abou-Zahra - http://www.w3.org/People/shadi/
Activity Lead, W3C/WAI International Program Office
Evaluation and Repair Tools Working Group (ERT WG)
Research and Development Working Group (RDWG)

Received on Tuesday, 2 October 2012 14:24:43 UTC