W3C home > Mailing lists > Public > public-semweb-ui@w3.org > June 2006

Re: seedling topic list for SWUI06

From: Alia Amin <A.K.Amin@cwi.nl>
Date: Tue, 27 Jun 2006 21:43:32 +0200
Message-ID: <44A18A64.6060909@cwi.nl>
To: public-semweb-ui@w3.org

Lloyd.Rutledge@cwi.nl wrote:

>Duane Degler wrote:
>
>  
>
>>As you know, I am interested in your last point, to identify if some of the
>>traditional usability testing methods need to change in light of a more
>>semantic web. Even the simple technique of a developer showing something to
>>friends reveals feedback, and formal usability testing often produces
>>valuable insights. Interaction with vocabulary and controlled vocabulary
>>    
>>
>has
>  
>
>>been studied, but needs more attention, I expect. User testing and
>>longitudinal data from pilots help identify what is useful, but how we
>>establish the protocols and interpret the results may need to be revisited
>>as semantic applications become more personalized, adaptive, and possibly
>>agent-driven.
>>    
>>
>
>Duane,
>
>Do you have any quick answers for these?  In particular, do you have
>tips for those preparing papers over the coming month for SWUI06 about
>how what usability tests they could effectively and efficiently apply
>in that timeframe?  What worked well in last year's workshop's papers
>(and similar venues), and what would you like to see more of?
>
>Thanks,
>Lloyd
>  
>


Correct me if I am wrong, before deciding on any usability test,
somethings should be clear such as:
1. what is your goal, what exactly do you want to find out
2. making sure your system/prototype is prepared to test the things you want
3. who would be your user
4. then designing the appropriate experiment such as: deciding the
metodology to use, how you  would carry out your experiment eg. no of
subjects, questions, etc., then how you would analyze the data you get.

As far as I know, this is as general as it gets.
I have run an evaluation for small subject sample in 2 weeks, but of
course it really depends on how many people you take, the usability test
you do and how well you prepare the experiment.

Unfortunately, just as there is no one-test-fits-all usability test for
HCI-UI evaluations, probably there no generic fast & usability test for 
SWUI that you can apply for any type of research question one might have.

Having said that, there are things you can do to save time.
Some pointers:
1. take enough sample, but not excessive if it does not add value to 
answering your questions. On the other hand, be aware that sample which 
is too low gives doubtful results.
2. plan your evaluation well, fail experiment means you might need to 
redo it and that takes time.
3. be aware that the more material you take (video, tape, interview, 
questionnaire) the more time you need to analyze them.


Alia
Received on Tuesday, 27 June 2006 20:18:55 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Tuesday, 8 January 2008 14:16:46 GMT