various items relating to evaluation methodology

I think we need a well-defined vocabulary to express important concepts relating to a web accessibility evaluation (for example, time of evaluation - an evaluation on the a website done at one point in time by the same individual  in the same "environment" may be different from an evaluation at a different point in time, because websites change - also terms specifying the particular "environment" of an evaluation need to be clarified.

Also perhaps considering principles of experimental design may help in our discussions (thinking of evaluations as possible "experiments" testing hypotheses.  A possible reference (given for information only - plenty of other references) in this regard is:

http://www.emathzone.com/tutorials/basic-statistics/basic-principles-of-experimental-designs.html

For example, one must consider the influence of any "side effects" or "omitted variable bias".

Not sure how much of this is applicable in our current deliberations, but just putting it out there..

Thanks and best wishes
Tim Boland NIST