Re: Measuring usability

On 16 Jan 2007, at 14:46, Phillip Lord wrote:

>>>>>> "BP" == Bijan Parsia <bparsia@cs.man.ac.uk> writes:
[snip]
>   BP> Usability is more *difficult* to measure than worst case
>   BP> complexity because it is highly multidimensional, and typically
>   BP> requires experiments. Usability is sensitive to, among many
>   BP> other things: task, user, and support environment (tools, books,
>   BP> accessible gurus).  Even when one has good data it can be
>   BP> difficult to interpret and even more difficult to generalize.
>
> Well, I tend to think of "measurability" as being an boolean value. My
> point was that you can measure usability, as you can anything
> else. Might be hard, but it is doable.

I don't really want to get into a huge debate about measurability  
theory, but "doable" in this case doesn't seem to mean "feasible".

I think it's fair to interpret "we don't have metrics" as "we don't  
know how to measure these things in a way at a cost that is clearly  
helpful". Part of measuring is deciding *what* to measure, and in  
usability there's a lot of *different* things you can measure. Some,  
I warrent, are incommenserable, thus I would be rather skeptical  
about a single usability metric.

>   BP> BTW, I don't think any "theorists", at least, any one involved
>   BP> in these debates, conflate worse case complexity with
>   BP> effectively implementability.
>
> I wasn't trying to imply that they did. My point was that measuring
> usability is often ad hoc, non-generalisable.

We agree.

> But, then, so does
> tractability,

Formal worse case tractability is neither, I think, ad hoc nor non- 
generalizable.

> or effective implementability, or how fast it goes.
[snip]

These are definitely subject to ad hoc-ness and non-generalizability.  
And I agree that this is what we should care about. Formal complexity  
analysis can give us clues there. The TF document does not *only*  
inlcude formal complexity analysis, but presents languages for which  
there is reasonable evidence of other virtues including useful  
expressivity and reasonable implementability.

I think we are on the same page.

Cheers,
Bijan.

Received on Tuesday, 16 January 2007 15:04:14 UTC