Re: 90 minutes with HCI researchers - what would you discuss?

Hyowon Lee wrote:
>> What is in it for HCI?  The need for HCI methods in Semantic Web
>> research and practice is obvious: we now have all this great machine
>> processing of knowledge, now we need to have users make the most of
>> it.  But can you describe what unique contributions the Semantic Web
>> field has to offer to concerns currently held by the HCI community?
>
> I have been working in the area of HCI for Multimedia Information
> Retrieval area for a few years now, and I found the above question
> very similar to what I have always been asking myself.  Here's my
> explanation, hopefully it can throw some light by comparison to
> Semantic Web.
>
> In Multimedia IR community, there are many obvious HCI implications
> from many of the underlying machine processing development.  For
> example, interaction designers for personal photo management systems
> (such as Flickr) have a chronic problem of getting the user annotate
> as much as possible for their uploaded photos.  Manually adding
> annotation for each photo is quite a burden and time-spender for the
> user; but without annotation the access is severely limited because
> the user cannot search properly, especially when the number of
> photos is very large.  So we (HCI folks) have been devising
> interaction mechanisms to encourage the users to add more annotation
> for each photo by easy drag-and-drop labels, simple choice from a
> provided vocabulary list, or allowing bulk (batch) annotation for
> multiple photos, etc. But as the collection size of the photos grows
> very fast, all these interaction techniques seem less and less
> usable.
>
> Now, multimedia IR (highly technically-focused) comes up with
> content-based image analysis techniques to automatically organise,
> classify, annotate and label the photos.  For example, faces in a
> newly-uploaded photo can be automatically detected and labeled as
> long as there are already some photos that have been previously
> labeled in the collection.  Imagine you upload 50-100 photos on the
> Flickr and you can immediately search by name of the persons
> appearing in those photos, without having to manually annotate each
> photo.  The implication that this particular technical development
> brings to HCI aspect is very obvious: it helps reduce the user's
> manual annotation burden dramatically, and there is a very
> significant effect in designing interfaces for photo management
> systems.

(Semi-)automated gathering of useful machine-processable data is a
clear interface desire.  There is also preventing the loss of such
data.  A common example is that mobile cameras put timestamps and
setting information in JPG files.  Perhaps soon they will put GPS in
as well, which can be pretty valuable meta-data for photo collections.
Gather and processing these semantics is a clear need for reducing and
removing redundancy in human annotation efforts.

While valuable to HCI, such applications may require a new perspective
on what is "the Semantic Web" for the SW community.  While "simple"
capture of data like this is a clear need for applications in the
category of those that use Semantic Web technologies, should SW see
this data capture as a legitimate part of SW research?  "Semantic Web
research" has been defined by the use of specific technologies and
processes.  Can and should this definition be extended to include
related technology that enables a "Semantic Web" category of
interaction?  If it enables Web 3.0 interaction, is it thus "Semantic
Web"?

> In other words, this is one of the contributions the multimedia
> field can offer to concerns currently held by the HCI community.  So
> I imagine there must be similar kinds of obvious contributions that
> Semantic Web technology can do for the currently available
> interfaces for various application areas, in reducing user's burden
> for example.
>
> For another example, HCI folks in the field of IR and Information
> Visualisation are working on visualizing temporal media such as
> video.  In designing efficient interaction schemes for a large movie
> archive, we want to provide a good overview of each movie before the
> user have to play the whole movie for 2 hours.  Video retrieval
> community offers automatic judicious selection of keyframes from the
> movie, thus summarizing the 2-hour temporal medium into a
> single-page, 50 miniaturised thumbnails of visual summary fully
> automatically.  It also offers automatic extraction of the most
> interesting or exciting sequences of the 2-hour movie and generate
> 1-minute video summary.  In designing user-interface for movie
> browsing, such summarization techniques can be directly used to help
> the user in quickly getting overview of each movie.  Again, the
> contribution that this particular technology can offer to HCI aspect
> is very clear and obvious: it adds to the designer's options in
> supporting the user's task of getting quick overview of the movies.

Simile Timeline provides a timeline-based interface to SW.  The
E-Culture demo provides a timeline interface in the context of museum
artworks annotated with SW technology.  Video summarization, keyframing
and hierarchy building are different, of course, but share a time-based
interface to semantic annotations.  It makes a category and several
subcategories of interface forms to the Semantic Web.

> The recurring question in my case has been on how to take advantage
> of these newly emerging technologies in multimedia in order to
> support the end-users

Here, of course, the challenge is to make sure that a technology
that is newly made available also has a real use.

> , but the answers to these have been relatively
> straightforward because what the specific multimedia technology is
> trying to do is often naturally such that HCI people can take away
> and use to solve their problems.

What brings about this user-centrism?  Is it intended by the
developers?

> I can imagine the HCI folks for
> Semantic Web technology will probably think in the same way.

On the other hand, they make this SW developers are building interfaces
based on the underlying format rather than what real users want to
see, such as with the Big Fat Graph problem [Karger and schraefel,
SWUI 2006]

> There
> are also chain-reaction issues, in which once we adopt a technology
> to reduce a main problem, another problems (although usually less
> severe as the original one) occurs as an HCI challenge.  Hopefully
> I'll have a chance to explain these later.

Looking forward to these explanations,
Lloyd

>
> Hyowon

Received on Monday, 7 May 2007 12:02:15 UTC