Re: 90 minutes with HCI researchers - what would you discuss?

>>> But can you describe what unique contributions the Semantic Web
>>> field has to offer to concerns currently held by the HCI community?

in terms of this question specifically. the issues we are facing,  
while trying to design user friendly (not data oriented systems as  
Lloyd mentions below) information systems is suddenly the wealth of  
knowledge that can be accessed.

suddenly the interconnected machine readable web could be used to get  
whatever information the user wants. how do you build that kind of  
interaction into an informatino system without having a sparql box in  
the ui? before, our information systems were designed around  
particular information stores, and interfaces were designed to convey  
useful information in a predictable way. things like the tabulator  
have started producing interactions that allow users to go endlessly  
through the semantic web, but i think its just the tip of the iceberg  
for producing free form interactions with such a broad network of  
unpredicatble data.

so now HCI design has to convey useful information in an  
unpredictable way - we dont know necessarily what sources are  
available or what users want to ask of them.  we want to design  
decent interactions that let people go on tangents, ask complex  
questions, etc. these are opportunities that are being created by  
semantic web technologies that will help bring the benefits of the  
semantic web to real users.

its the openness and endlessness that is sourcing many new and  
interesting hci (and IR) issues. it challenges many of our basic hci  
assumptions (you might want to read [Wilson & schraefel, swui 2006]),  
which are based on a closed world assumption.

max


On 7 May 2007, at 13:01, Lloyd Rutledge wrote:

>
> Hyowon Lee wrote:
>>> What is in it for HCI?  The need for HCI methods in Semantic Web
>>> research and practice is obvious: we now have all this great machine
>>> processing of knowledge, now we need to have users make the most of
>>> it.  But can you describe what unique contributions the Semantic Web
>>> field has to offer to concerns currently held by the HCI community?
>>
>> I have been working in the area of HCI for Multimedia Information
>> Retrieval area for a few years now, and I found the above question
>> very similar to what I have always been asking myself.  Here's my
>> explanation, hopefully it can throw some light by comparison to
>> Semantic Web.
>>
>> In Multimedia IR community, there are many obvious HCI implications
>> from many of the underlying machine processing development.  For
>> example, interaction designers for personal photo management systems
>> (such as Flickr) have a chronic problem of getting the user annotate
>> as much as possible for their uploaded photos.  Manually adding
>> annotation for each photo is quite a burden and time-spender for the
>> user; but without annotation the access is severely limited because
>> the user cannot search properly, especially when the number of
>> photos is very large.  So we (HCI folks) have been devising
>> interaction mechanisms to encourage the users to add more annotation
>> for each photo by easy drag-and-drop labels, simple choice from a
>> provided vocabulary list, or allowing bulk (batch) annotation for
>> multiple photos, etc. But as the collection size of the photos grows
>> very fast, all these interaction techniques seem less and less
>> usable.
>>
>> Now, multimedia IR (highly technically-focused) comes up with
>> content-based image analysis techniques to automatically organise,
>> classify, annotate and label the photos.  For example, faces in a
>> newly-uploaded photo can be automatically detected and labeled as
>> long as there are already some photos that have been previously
>> labeled in the collection.  Imagine you upload 50-100 photos on the
>> Flickr and you can immediately search by name of the persons
>> appearing in those photos, without having to manually annotate each
>> photo.  The implication that this particular technical development
>> brings to HCI aspect is very obvious: it helps reduce the user's
>> manual annotation burden dramatically, and there is a very
>> significant effect in designing interfaces for photo management
>> systems.
>
> (Semi-)automated gathering of useful machine-processable data is a
> clear interface desire.  There is also preventing the loss of such
> data.  A common example is that mobile cameras put timestamps and
> setting information in JPG files.  Perhaps soon they will put GPS in
> as well, which can be pretty valuable meta-data for photo collections.
> Gather and processing these semantics is a clear need for reducing and
> removing redundancy in human annotation efforts.
>
> While valuable to HCI, such applications may require a new perspective
> on what is "the Semantic Web" for the SW community.  While "simple"
> capture of data like this is a clear need for applications in the
> category of those that use Semantic Web technologies, should SW see
> this data capture as a legitimate part of SW research?  "Semantic Web
> research" has been defined by the use of specific technologies and
> processes.  Can and should this definition be extended to include
> related technology that enables a "Semantic Web" category of
> interaction?  If it enables Web 3.0 interaction, is it thus "Semantic
> Web"?
>
>> In other words, this is one of the contributions the multimedia
>> field can offer to concerns currently held by the HCI community.  So
>> I imagine there must be similar kinds of obvious contributions that
>> Semantic Web technology can do for the currently available
>> interfaces for various application areas, in reducing user's burden
>> for example.
>>
>> For another example, HCI folks in the field of IR and Information
>> Visualisation are working on visualizing temporal media such as
>> video.  In designing efficient interaction schemes for a large movie
>> archive, we want to provide a good overview of each movie before the
>> user have to play the whole movie for 2 hours.  Video retrieval
>> community offers automatic judicious selection of keyframes from the
>> movie, thus summarizing the 2-hour temporal medium into a
>> single-page, 50 miniaturised thumbnails of visual summary fully
>> automatically.  It also offers automatic extraction of the most
>> interesting or exciting sequences of the 2-hour movie and generate
>> 1-minute video summary.  In designing user-interface for movie
>> browsing, such summarization techniques can be directly used to help
>> the user in quickly getting overview of each movie.  Again, the
>> contribution that this particular technology can offer to HCI aspect
>> is very clear and obvious: it adds to the designer's options in
>> supporting the user's task of getting quick overview of the movies.
>
> Simile Timeline provides a timeline-based interface to SW.  The
> E-Culture demo provides a timeline interface in the context of museum
> artworks annotated with SW technology.  Video summarization,  
> keyframing
> and hierarchy building are different, of course, but share a time- 
> based
> interface to semantic annotations.  It makes a category and several
> subcategories of interface forms to the Semantic Web.
>
>> The recurring question in my case has been on how to take advantage
>> of these newly emerging technologies in multimedia in order to
>> support the end-users
>
> Here, of course, the challenge is to make sure that a technology
> that is newly made available also has a real use.
>
>> , but the answers to these have been relatively
>> straightforward because what the specific multimedia technology is
>> trying to do is often naturally such that HCI people can take away
>> and use to solve their problems.
>
> What brings about this user-centrism?  Is it intended by the
> developers?
>
>> I can imagine the HCI folks for
>> Semantic Web technology will probably think in the same way.
>
> On the other hand, they make this SW developers are building  
> interfaces
> based on the underlying format rather than what real users want to
> see, such as with the Big Fat Graph problem [Karger and schraefel,
> SWUI 2006]
>
>> There
>> are also chain-reaction issues, in which once we adopt a technology
>> to reduce a main problem, another problems (although usually less
>> severe as the original one) occurs as an HCI challenge.  Hopefully
>> I'll have a chance to explain these later.
>
> Looking forward to these explanations,
> Lloyd
>
>>
>> Hyowon
>
>

----------------------------------
n - max wilson
e - mlw05r@ecs.soton.ac.uk
w - www.ecs.soton.ac.uk/~mlw05r
t - +44 (0) 2380 598367
----------------------------------