- From: Shelley Powers <shelley.just@gmail.com>
- Date: Mon, 6 Jul 2009 10:08:11 -0500
- To: Lachlan Hunt <lachlan.hunt@lachy.id.au>
- Cc: Julian Reschke <julian.reschke@gmx.de>, Sam Ruby <rubys@intertwingly.net>, public-html@w3.org, www-archive <www-archive@w3.org>
On Mon, Jul 6, 2009 at 9:41 AM, Lachlan Hunt<lachlan.hunt@lachy.id.au> wrote: > Shelley Powers wrote: >> >> There is no empirical data to support either side in this issue. >> >> Reviewing randomly accessed web pages is not a proper study, and >> conclusions reached based on this review are subjective. > > I only referred to that data as observational data; I realise it isn't a > proper scientific study. But even so, in the absense of any other data, we > have to work with we've got and it's really difficult to reach any other > conclusion from that, other than almost a total lack of correct usage amidst > widespread misuse. > >> In addition, there is a looming deadline related to last call, so >> there is no time to do proper studies. > > About a week ago, I tried to start a discussion on how to obtain proper > scientific data [1]; sent in response to your repeated attempts to discredit > the data we have. But it's surprising to me that rather than even attempt > to contribute to producing a proper scientific study in any way at all, > you'd rather continue you're attempts to discredit what we have, say we > can't get anything else. But those suggestions weren't proper scientific studies -- not if the end goal is determining which is the best approach from an accessibility and usability point of view. To study the latter, we would need to ask for a group of volunteers from the community, both those who need AT and those who don't. We would need to derive appropriate test data, providing tests with summary, and tests without, or tests trying out other variations. The tests would need to be conducted in controlled circumstances, and observations made, as well as questionnaires created to evaluate the effectiveness of any one approach. To ensure an objective approach, those making the observations should probably do so unaware of which testing set the current subject is taking. When you try to analyze data related to existing web pages, it's virtually impossible to filter out any number of factors, including the fact that HTML tables were misused, the explanation of summary in HTML 4 may have been faulty, lack of promotion for summary until recently, and so on--even up to the point of testing out exactly how random the web page data is, and how representative is the data. > >> Frankly, neither side likely has the resources necessary to do proper >> studies. > > I don't think the responsibility for performing the study should fall onto > just one side of the debate. Rather, we should be working together to > figure out how to best perform the study and find volunteers within the > group capable of contributing resources. We've already seen several members > demonstrate the ability to collect data, from Ben Millard's table > collections and Philip's various collections of data, to some informal > survey results from Google and Opera's MAMA system. So I think it is naiive > to say that we don't have resources available to do proper studies. We just > need to sort out the details of what needs to be done and do them. > Again, you're relying almost exclusively on data derived from existing web pages (without any ability to control how the data is collected), but the issue really is based on human usability and overall accessibility. I realize this is a discussion we've had in the past. I copied this email to www-archive. I believe we should continue this particular discussion in that location. > [1] http://lists.w3.org/Archives/Public/public-html/2009Jun/0851.html > > -- > Lachlan Hunt - Opera Software > http://lachy.id.au/ > http://www.opera.com/ > Thanks Shelley
Received on Monday, 6 July 2009 15:08:55 UTC