Re: How to make complex data tables more accessible to screen-reader users

> The end goal is to determine the best solution that balances usability and
> accessibility from an end user point of view, with the the ability of
> authors to get it sufficiently right in practice.
>
> A theoretically perfect solution from an end user usability/accessibility
> POV is no good in practice if the demands on authors to get it right are too
> great and impractical.  Likewise, a solution that focuses too much on author
> ability while ending up with a result that isn't at all usable by end users
> isn't either.

I don't know that any suggestion put forward addresses the author and
what they can do. There's been a hypothesis proposed that authors are
not able to do the text right if it isn't displayed in the web page in
any browser. But there's been nothing that tests this hypothesis out
-- no solid, dependable study that demonstrates succinctly that
displaying the text in the page somehow helps authors get it.

So my own hypothesis is that when it comes to the authors, focusing
purely on the markup is ineffective because any one of the solutions
doesn't address the real problems: confusion about what needs to be
provided, good, clear cut examples, and encouragement from the web
community at large, and increased encouragement from the accessibility
community.

And again, the only way to really test this, objectively, is to get
volunteers from the web community at large, develop tests for each of
the approaches, have the people perform the tasks in controlled
environments, observe the behavior, the results, and also provide
questionnaires to test web authors preferences, understanding, etc.

Accessing existing web pages is good for developing hypothesis, makes
good anecdotal information, and can help determine where problems
might arise. But it can't usefully be used to provide a definitive
conclusion about what exactly is the root cause of the problems. Why?
Because the data arises in the wild, not within a carefully controlled
environment, where external factors can be filtered.


>
> Besides, I don't claim that my suggestions were perfect.  I'm sure there are
> many improvements that can be made.  But, as I wrote, my purpose was not to
> sort out all the details myself, but rather to get the group involved in
> sorting out the details together.

Sure, but I can't see this group being able to develop truly effective
usability studies within the short time we have, and neither do I see
any indication that we have the resources to conduct the proper tests.

I may be wrong: chairs, does the W3C have a usability lab?

>
>> To study the latter, we would need to ask for a group of volunteers
>> from the community, both those who need AT and those who don't. We
>> would need to derive appropriate test data, providing tests with
>> summary, and tests without, or tests trying out other variations...
>
> Indeed, the results of a study like that would also be useful.  But that's
> no reason discount the results of a study focussing on authoring practices.
>

But the same issues of working with data in the wild et al apply
equally to authoring practices.

>> When you try to analyze data related to existing web pages, it's
>> virtually impossible to filter out any number of factors, including
>> the fact that HTML tables were misused, the explanation of summary in
>> HTML 4 may have been faulty, lack of promotion for summary until
>> recently, and so on--even up to the point of testing out exactly how
>> random the web page data is, and how representative is the data.
>
> It's impossible to get a perfect dataset in any study.  What we can do,
> however, is filter based on a few selected major criteria
>

But you can't filter out all of the extraneous environmental factors
when you're working with data in the wild.

Even if we create a few web pages and ask people to try them out or
try editing something and look at the results, we aren't conducting
the study in a controlled environment, nor are we including safeguards
to ensure bias doesn't creep into the result.

Normally, I don't think we would need this level of rigor, but when
you have such a strong disagreement in the group, you have to go the
extra distance to ensure truly comprehensive and inclusive results.


>>> So I think it is naiive to say that we don't have resources
>>> available to do proper studies. We just need to sort out the
>>> details of what needs to be done and do them.
>>
>> Again, you're relying almost exclusively on data derived from existing
>> web pages
>
> Those who ignore history are doomed to repeat it.  I'm not saying we should
> relying exlusively on analysis of existing web sites; certainly, usability
> studies would also be valuable.  But we certainly must not ignore existing
> practice, especially if it indicates problems with the design that affects
> how authors make use of it.
>

But again--and I'm not sure I'm saying this clearly, so inviting
others to rephrase or add to what I'm saying--we don't fully
understand the root causes of the data that we do see.

Here's different scenarios and associated logic with each:

We have web scrapings from 100,000 web pages
We have queried on HTML tables
We have printed out the summaries for each table
We find...

We find that the vast majority of summary attributes are blank, and
therefore the summary attribute isn't useful and should be eliminated

We find that the vast majority of summary attributes are blank, but
the tables are navigational and layout tables, where summary isn't
useful, and therefore we can't come to any conclusion about the
effectiveness of the summary attribute, because of the contamination
to the sample data (tables used incorrectly)

We find several uses of summary but it doesn't provide what we
determine summary should have. We believe the reason is that the
summary isn't printed out, and the author can't check their work.
Therefore the summary attribute is effectively harmful, and should be
eliminated, and the author encouraged to put the data into the caption

We find several uses of summary but it doesn't provide what we
determine summary should have. We believe the reason is that
accessibility has not been promoted, and summary isn't completely
understood. Therefore the summary attribute should be maintained, at
lest for now, and additional education and encouragement provided web
authors.

I could go on with variations, all based on statements made in the
past. See the problem? We have no way of definitively defining what is
the root cause for the problems we're seeing with summary. We then
have to fall back on subjective analysis, leading to subjective
conclusions. Subjective conclusions carry all sorts of baggage,
including biases about non-printing attributes and so on.

Again, I may not be expressing myself clearly. I encourage others to
provide additional clarification.

>> (without any ability to control how the data is collected),
>
> There are various ways we can control how the data is collected.  As I
> mentioned in my previous e-mail, we can, for example, restrict the survey to
> specific genres of web sites that are most likely to have complex data
> tables where summaries are most useful.  There may be other, perhaps even
> more effective, ways to filter data too.  But it's short sighted to simply
> say there is no way and give up without even trying.
>

Rather than repeat my overly verbose discussion above I'll just point
you to it, again.

>> but the issue really is based on human usability and overall
>> accessibility.
>
> There is more than one aspect of this issue that needs to be looked at,
> ranging from existing authoring practices and how to improve those, to the
> usability of various technical solutions by end users.
>

ditto

>> I realize this is a discussion we've had in the past. I copied this
>> email to www-archive. I believe we should continue this particular
>> discussion in that location.
>
> I don't see why we should exclude the rest of the group from discussions
> about how to go about obtaining more useful results.
>

I'm cool with this, but I'm just a little wary -- I no longer can tell
when a discussion should take place here, or in www-archives. I'm
becoming uncomfortable participating in either place because of the
seemingly (to me) arbitrary nature of when a discussion is allowed
here, and when pushed to www-archives.

But if folks are cool with this discussion happening here, I'm happy
to continue here.


> --
> Lachlan Hunt - Opera Software
> http://lachy.id.au/
> http://www.opera.com/
>

Shelley

Received on Monday, 6 July 2009 17:26:34 UTC