RE: Annotation working group

Libby,

I'm sorry that it's taken me this long to respond.  I've read your article on aggregation of RDF annotations and I am still following some of the links from that article, from your email, and from the DESIRE project web site.

Comments are interspersed, below.

-bryan

-----Original Message-----
From:	Libby Miller [SMTP:Libby.Miller@bristol.ac.uk]
Sent:	Wednesday, May 10, 2000 1:49 PM
To:	Bryan Thompson
Cc:	'Julie Gibson'; www-annotation@w3.org; annotate@cog-tech.com; phil.cross@bristol.ac.uk; m.l.poulter@bristol.ac.uk; Dan Brickley
Subject:	RE: Annotation working group



Hi Bryan,

Thanks for having a look at the site. The desire project is ending in
July, although some of the 'recommendations' work will be continued with
the MedCertain project [1].

You asked about my particular interests in the design of annotation
servers. I am principally interested in annotations of things that can
be given a URI, and specifically html pages. These annotations could be 
as brief as giving a thumbs up or thumbs down to an html page. However,
I'd like to be able to model annotations in such a way that an
annotation is generalizable to part of a document, or any other resource
(e.g. a plumber). RDF [2] is potentially good for this, as it assigns a
unique identifier to any resource, to which you can attatch other
resources by arcs, labeling the resource or making statments about it.

What I've been contentrating on is trying to model annotations in
node/arc diagrams, and then store them in a fairly generic RDF triple
store in a relational database. At the moment I'm trying to generalise
the APIs and store sufficiently so that I can store calendar information
using the same process. There's a lot of this sort of thing going on in
RDF, all linked from [2].

I have been thinking about structured annotations based on a model of argument-based reasoning (e.g., Toulmin, S. (1958). The uses of argument. Cambridge, UK: Cambridge University Press.).  I have been thinking more along the lines defining a restricted subset of XML standard for TopicMaps (www.topicmap.com) that provides an XML schema for argument-based annotations.  (This by no means precludes other bases for defining annotation systems.)  I am also interested in providing computation support for reasoning with arguments over (e.g., X/HTHML web) content.

Our background is in cognitive models of how people reason about uncertainty (e.g., Cohen, M.S., Freeman, J.T. and Wolf, Steve. (1996). Meta-recognition in time-stressed decision making: Recognizing, critiquing, and correcting. Human Factors (38,2), pp. 206-219. and others at http://www.cog-tech.com/papers.html).  We have also been working with computational models of the cognitive models, e.g., www.icsi.berkeley.edu/~shastri/shruti and www.cog-tech.com/hybrid.

I'm also curious about the display of annotations data, and the
connected question of whether people will use annotations servers. The
current display metaphor we tend to use here is that of "shared
bookmarks", although some colleagues of mine have experimented with
using the "what's related" feature of Netscape to serve up comments
about a site. 

I like the inline method of www.crit.org.  There is also the Open Law annotation engine at http://cyber.law.harvard.edu/, which uses a "[1]" style notation in the content flow and a side bar of the annotations.  Managing aggregation, consensus (and difference), and scale are issues that will shape the success of views into a collaborative annotation system.

What we found here with the shared bookmrks idea at least, is that
people tend to use annotations servers more if they are used in small
communities. This has to do with issues of trust coupled with the lack
of detail about a resource provided by a minimal user interface for
inpout of annotations. So in our system, because not much information is
provided about the annotation by the annotator, you need to know
contextual information about that person. There was a report which
touches on these issues from Desire [3]. However, using RDF you can
aggregate annotations by slurping them from the web using XML, and I've
been wondering how we can overcome this issues of trust (and display) in
aggregation in a sensible way.

We have also done some modeling of trust, albeit in a different context (e.g., Cohen, Marvin S., Raja Parasuraman, and Jared T. Freeman. Trust In Decision Aids: A Model And Its Training Implications. Proceedings, 1998 Command and Control Research and Technology Symposium. Department of Defense C4ISSR Cooperative Research Program, 1998., which is also available at www.cog-tech.com/papers.html).

My perspective on annotation aggregation has been quite different, probably since we are dealing with a specific kind of structured annotation -- argument models.  Argument models label evidence - conclusion relationships, and you may identify rebuttals, i.e., conditions under which the argument is known not to hold.  Aggregation for arguments appears to have several facets:

(1) community and the means to select groups of individuals known to be broadly cooperating on a set of related issues.  This addresses your trust issue to some extent.
(2) a consensus policy, that is, the means by which you combine possibly divergent opinions and collapse the variety into a consensus on some issue (or model of an issue).
(3) attentive focus of those arguments, in a potentially vast body of arguments, that critically intersect some key issues of interested to a user.  www.crit.org did some work with contextures for an argument model.  This provided a visualized abstraction of the key relationships between a core issue and the evidence and conclusions drawn from that issue.  
(4) the explicit support for abstraction, in the sense of creating a vertical movement towards an increasing theoretical or invariant model of some increasingly broad set of issues.  For example, the relationships among information processing, cybernetics, economics, evolutionary systems, etc.  A problem in one of these domains can be linked via abstractions so as to facilitate transference of knowledge from other domains that share a common theoretical basis.

I just looked at your resume and it sounds like you're doing very
advanced stuff. I would be very interested to know more.


At this moment I am working to identify people and organizations interested in creating an extensible open source project for "commercial grade" annotation servers.  I.e., annotation servers that are capable of scaling to large Internet populations, provide fine-grained annotation, support annotation interchange, interfaces for interoperation with intelligent agents, etc.  I like the thought that some kinds of aggregation and consensus management could be abstracted into separate, and interoperable, services.  The more we factor the problem, the more success we are likely to have.

-bryan


Libby


[1] http://medpics.org/medcertain/
[2] http;//www.w3.org/RDF/ 
[3] http://www.desire.org/html/research/deliverables/D3.1/qualratings/doc0002.htm



On Wed, 10 May 2000, Bryan Thompson wrote: 

> Libby,
> 
> I am going through the DESIRE project site now.  This is a wonderful project 
> and I am amazed that I have not encountered this effort before.  
> Can you expand on your particular interests in the design of annotation 
> systems?
> 
> -bryan
>

Received on Sunday, 14 May 2000 16:31:46 UTC