Re: 90 minutes with HCI researchers - what would you discuss?

It's worth repeating that the semantic web is as much of a knowledge systems architecture issue as anything- which requires a mega disciplinary approach. We've been working on it for a long time- pre commercialization of the Internet and each evolution since. I think we've resolved most of the issues to reach the minimal acceptable standard from the organization's perspective, which requires a holistic approach, but not necessarily the public Web. 

The public web is far more difficult to achieve in large part because sharing knowledge very often clashes directly with the legal and fiduciary responsibility of the organization. 

And of course the individual knowledge worker often has direct conflicts to include others stealing their work, or taking credit for it either internally or externally- a systemic problem in virtual and real worlds, but the virtual world has made it much easier, and in many ways rewarded the wrong type of behavior. 

So to me discussing technical interoperability without including alignment of incentives, particularly relative to standardization and adoption, is frankly a waste of time and energy.

The broader issues involved have long been a problem across most disciplines in tech transfer, representing in part why it can be so challenging. The academic culture and incentives where many innovations occur is dramatically different than most of society. Knowledge systems architecture IMO takes all of this and much more into consideration- to transfer knowledge also means a transfer of resources with associated investment and cost. That can be a far more complex issue for everyone outside of the minority charged with sharing knowledge, than technical interoperability. 

How long do we expect intelligent individuals to share their knowledge when they lack payment enjoyed by their colleagues, for example? Or to share within a large organization that rarely if ever rewards (psychologically or financially) those who contribute far more than the average? Does anyone really expect global organizations to share knowledge from say a $5bn investment in R&D without compensation? The latter could well be defined as an illegal act in much of the world, depending on the type of organization. Somehow these issues often get lost in the discussion, but are none-the-less essential if wide adoption is the goal.

I recently discussed this semi privately with Vint Cerf, where among other things I suggested that the semantic web is a much different prospect than his/colleague's fine work with IP and TBL's in the original W3 (I'm having difficulty adopting 1-3.0), which were based on the purity of numbers, and language was left to the user. The incentives for adoption of IP and HTTP were entirely different- indeed opposite in many respects from an economic perspective.

While some have come very far in their attempts to use AI in translating human languages, and the more recent standards are making solutions to these issues more possible than in the earlier attempts, we are still left with the structural disincentives for sharing knowledge on the public Internet/Web, both at the organizational level as well as individual. From a public web perspective, does it matter if AI formulas can automatically define and translate ontologies if negative consequences exist for those who share their most valuable knowledge? 

Within especially large organizations, and formal relationships, absolutely. With the public which includes armies of competitive intelligence folks who will use the information to destroy a person's financial security, or organization, no.

As I and no doubt many others have often said, Karma has been slow to adopt to the network era- Karma is network challenged. Notice I carefully selected the word slow. Like any other medium, sustainable economics and associated modeling with incentives properly aligned must precede wide & sustainable adoption. .02- MM 







----- Original Message ----- 
  From: Max Wilson 
  To: Lloyd.Rutledge@cwi.nl 
  Cc: public-semweb-ui@w3.org ; Hyowon Lee 
  Sent: Tuesday, May 08, 2007 3:10 AM
  Subject: Re: 90 minutes with HCI researchers - what would you discuss?


        But can you describe what unique contributions the Semantic Web
        field has to offer to concerns currently held by the HCI community?


  in terms of this question specifically. the issues we are facing, while trying to design user friendly (not data oriented systems as Lloyd mentions below) information systems is suddenly the wealth of knowledge that can be accessed.


  suddenly the interconnected machine readable web could be used to get whatever information the user wants. how do you build that kind of interaction into an informatino system without having a sparql box in the ui? before, our information systems were designed around particular information stores, and interfaces were designed to convey useful information in a predictable way. things like the tabulator have started producing interactions that allow users to go endlessly through the semantic web, but i think its just the tip of the iceberg for producing free form interactions with such a broad network of unpredicatble data.


  so now HCI design has to convey useful information in an unpredictable way - we dont know necessarily what sources are available or what users want to ask of them.  we want to design decent interactions that let people go on tangents, ask complex questions, etc. these are opportunities that are being created by semantic web technologies that will help bring the benefits of the semantic web to real users.


  its the openness and endlessness that is sourcing many new and interesting hci (and IR) issues. it challenges many of our basic hci assumptions (you might want to read [Wilson & schraefel, swui 2006]), which are based on a closed world assumption. 


  max




  On 7 May 2007, at 13:01, Lloyd Rutledge wrote:




    Hyowon Lee wrote:
        What is in it for HCI?  The need for HCI methods in Semantic Web
        research and practice is obvious: we now have all this great machine
        processing of knowledge, now we need to have users make the most of
        it.  But can you describe what unique contributions the Semantic Web
        field has to offer to concerns currently held by the HCI community?


      I have been working in the area of HCI for Multimedia Information
      Retrieval area for a few years now, and I found the above question
      very similar to what I have always been asking myself.  Here's my
      explanation, hopefully it can throw some light by comparison to
      Semantic Web.


      In Multimedia IR community, there are many obvious HCI implications
      from many of the underlying machine processing development.  For
      example, interaction designers for personal photo management systems
      (such as Flickr) have a chronic problem of getting the user annotate
      as much as possible for their uploaded photos.  Manually adding
      annotation for each photo is quite a burden and time-spender for the
      user; but without annotation the access is severely limited because
      the user cannot search properly, especially when the number of
      photos is very large.  So we (HCI folks) have been devising
      interaction mechanisms to encourage the users to add more annotation
      for each photo by easy drag-and-drop labels, simple choice from a
      provided vocabulary list, or allowing bulk (batch) annotation for
      multiple photos, etc. But as the collection size of the photos grows
      very fast, all these interaction techniques seem less and less
      usable.


      Now, multimedia IR (highly technically-focused) comes up with
      content-based image analysis techniques to automatically organise,
      classify, annotate and label the photos.  For example, faces in a
      newly-uploaded photo can be automatically detected and labeled as
      long as there are already some photos that have been previously
      labeled in the collection.  Imagine you upload 50-100 photos on the
      Flickr and you can immediately search by name of the persons
      appearing in those photos, without having to manually annotate each
      photo.  The implication that this particular technical development
      brings to HCI aspect is very obvious: it helps reduce the user's
      manual annotation burden dramatically, and there is a very
      significant effect in designing interfaces for photo management
      systems.


    (Semi-)automated gathering of useful machine-processable data is a
    clear interface desire.  There is also preventing the loss of such
    data.  A common example is that mobile cameras put timestamps and
    setting information in JPG files.  Perhaps soon they will put GPS in
    as well, which can be pretty valuable meta-data for photo collections.
    Gather and processing these semantics is a clear need for reducing and
    removing redundancy in human annotation efforts.


    While valuable to HCI, such applications may require a new perspective
    on what is "the Semantic Web" for the SW community.  While "simple"
    capture of data like this is a clear need for applications in the
    category of those that use Semantic Web technologies, should SW see
    this data capture as a legitimate part of SW research?  "Semantic Web
    research" has been defined by the use of specific technologies and
    processes.  Can and should this definition be extended to include
    related technology that enables a "Semantic Web" category of
    interaction?  If it enables Web 3.0 interaction, is it thus "Semantic
    Web"?


      In other words, this is one of the contributions the multimedia
      field can offer to concerns currently held by the HCI community.  So
      I imagine there must be similar kinds of obvious contributions that
      Semantic Web technology can do for the currently available
      interfaces for various application areas, in reducing user's burden
      for example.


      For another example, HCI folks in the field of IR and Information
      Visualisation are working on visualizing temporal media such as
      video.  In designing efficient interaction schemes for a large movie
      archive, we want to provide a good overview of each movie before the
      user have to play the whole movie for 2 hours.  Video retrieval
      community offers automatic judicious selection of keyframes from the
      movie, thus summarizing the 2-hour temporal medium into a
      single-page, 50 miniaturised thumbnails of visual summary fully
      automatically.  It also offers automatic extraction of the most
      interesting or exciting sequences of the 2-hour movie and generate
      1-minute video summary.  In designing user-interface for movie
      browsing, such summarization techniques can be directly used to help
      the user in quickly getting overview of each movie.  Again, the
      contribution that this particular technology can offer to HCI aspect
      is very clear and obvious: it adds to the designer's options in
      supporting the user's task of getting quick overview of the movies.


    Simile Timeline provides a timeline-based interface to SW.  The
    E-Culture demo provides a timeline interface in the context of museum
    artworks annotated with SW technology.  Video summarization, keyframing
    and hierarchy building are different, of course, but share a time-based
    interface to semantic annotations.  It makes a category and several
    subcategories of interface forms to the Semantic Web.


      The recurring question in my case has been on how to take advantage
      of these newly emerging technologies in multimedia in order to
      support the end-users


    Here, of course, the challenge is to make sure that a technology
    that is newly made available also has a real use.


      , but the answers to these have been relatively
      straightforward because what the specific multimedia technology is
      trying to do is often naturally such that HCI people can take away
      and use to solve their problems.


    What brings about this user-centrism?  Is it intended by the
    developers?


      I can imagine the HCI folks for
      Semantic Web technology will probably think in the same way.


    On the other hand, they make this SW developers are building interfaces
    based on the underlying format rather than what real users want to
    see, such as with the Big Fat Graph problem [Karger and schraefel,
    SWUI 2006]


      There
      are also chain-reaction issues, in which once we adopt a technology
      to reduce a main problem, another problems (although usually less
      severe as the original one) occurs as an HCI challenge.  Hopefully
      I'll have a chance to explain these later.


    Looking forward to these explanations,
    Lloyd




      Hyowon






  ----------------------------------
  n - max wilson
  e - mlw05r@ecs.soton.ac.uk
  w - www.ecs.soton.ac.uk/~mlw05r
  t - +44 (0) 2380 598367
  ----------------------------------