Re: Bringing annotation to the browser

Thank you, Gerben, for the excellent summary and commentary.

I agree that exposing annotation to Web authors should be out of scope for any initial implementations: fragment directives are a mechanism to do that.

Text Fragments specifically uses a "fragment directive <>" approach in which the selector ("Text Fragment") is "*currently not web-exposed"*.  The approach mentioned in that spec actually does expose selectors to Web authors (you can link to them, create History entries with them, etc.), but in way that defers some of the problems you've mentioned.

(The _Selectors and States as Fragment Identifiers_ note is full of ideas that are workable within the fragment directive approach.)

I would also note: CSS is not fully controlled by author stylesheets, user agent stylesheets, or user stylesheets.  It's designed carefully, and designs are carefully revisited (see *user-select*, *:visited*, etc.).

> For web annotation, perhaps even some small changes could unblock other people (like myself) from implementing annotation capabilities through extensions; e.g. it might already help if Chromium implemented the Sidebar API <>

The Sidebar API is, in many ways, the ideal primitive for enabling this.

> provide a way to draw lines and highlights in an overlay on top of a page without modifying its DOM.

Google Chrome's Gleam API proposal <> (for Web extensions) was aimed squarely at this.

Mozilla's "Firefox Screenshots <>" took a very practical approach to implementing a selector creation user interface.  It essentially involved a privileged extension, which could itself become a Web Extension API for creating selectors.  With work, rendering annotations could also be implemented in the same API (and with the same approach).

(Firefox Screenshots appears to have been killed by market economics, unrelated to the merits of the project or its approach.)

> As said, I would be curious to hear the thoughts from others in this group. Is the interest in web annotation from Chromium developers (even if coming ‘late to the party’) an opportunity for web annotation? Or a hopeless/misguided mission? Or…?

This is a difficult time to get feedback from diverse parties, which means that a responsible implementation would be focused on future extensibility, and avoiding mistakes.  This is an opportunity for *the Web*, because it is an opportunity for users (like the Internet, the Web is for users).

geppy (*she/her*)

On Mon, May 24, 2021, at 10:39 AM, Gerben wrote:
> Hi everyone,

> I already sent David feedback privately, which I will share below. I am curious if anyone feels similar or has other remarks to share. As you can read below, I suggested David to discuss the plan in this Community Group, which has pursued highly similar ideas for several years. But the lack of response so far, to what sounds to me like a vague but exciting proposal, makes me wonder if anybody is still reading this list, and if anyone is still interested in the topic…

> Firstly, some possibly relevant context for those who missed it: David and his team already got a significant annotation-related change into the Chromium browser over the last year: Text Fragments <>, defining a special type of fragment identifier to point to (≈ scroll to and highlight) a quoted phrase inside the document. E.g. pointing at the words “illustrative example” on page looks like this: (try open it in a recent Chromium-based browser <>)



> Note for the Web Annotation-savvy: these Text Fragments are more or less a compacted version of serialising a TextQuoteSelector (or a RangeSelector containing two TextQuoteSelectors) into a URL, as was envisioned in the Selectors and States as Fragment Identifiers <> idea (that never made it into the Web Annotation recommendations). For comparison, here is a pointer at the same words using that latter serialisation:



> Having the ability to point at an arbitrary selection in web pages seems a small but important step to annotating web pages.

> Back to the point, here is my response to David’s request for feedback on the mentioned explainer <>:

>> Making annotations part of the web has been a vision of mine for many years (like for many others). Time and again the conclusion has been that this should be part of user agents, so interest from Chromium opens lots of possibilities.

>> Overall, I think your outline of your proposal is well-informed and reasonable, and seems to head roughly in the direction that I myself have been wanting to move things too; see e.g. my current project <> funded by the EU’s Next Generation Internet project, or my blog post <> from when I was an intern at Hypothesis (2014). I would be glad to exchange notes and collaborate in some way, join a call, or whatever works.

>> Below some thoughts from reading the explainer.

>> *Avoiding the “web graffiti” problem*

>> I am very pleased to see that it is a non-goal to make *“a centralized “comments section” for the web”*. Many previous annotation projects would create a single central annotation store, and show all annotations to every reader. Then website publishers would often be furious about losing control over the presentation of their content. Moreover, lots of this content would be of low interest to most readers. Moderation does not scale well, a single party as moderator is a bad idea anyhow, etcetera. (I wonder why you do not list Google Sidewiki <> among the previous efforts. Perhaps it is rather not spoken about among Googlers? ;))

>> Given this history, I recommend being extra clear about the approach, because you’ll have many people asking “what about spam/harassment/…?”. When talking about my project visions I often have to make clear that, unlike in many other/previous projects, people will only see annotations when they asked for them, from sources they selected.


>> Consequently, I think the first item+subitems in your list of challenges are problematic:

>>> “Decentralized moderation: Whose responsibility is this? …”

>> We could just as well ask: who moderates the WWW? That seems the wrong question. Boldly put: *If the annotation system requires a moderation system, it is a bad system.* The challenge should be how not to need moderation; otherwise it is doomed to fail — probably even before launching, it will be killed by a storm of criticism and bad publicity (harassment, disinfo and social media’s moderation failures are hot topics these days).

>> I would therefore strongly consider limiting the design and discussion to ] and use cases that do not pull annotations from just anywhere & anyone, but only get annotations the reader asked for: 
>>  * Personal private notes — stored in your browser, much like bookmarks
>>  * Sharing a document+annotations — e.g. I add notes to a page and share the annotated page with you. (technically, perhaps I point you to a collection of Annotations targeting the same document; your browser renders its target document while showing the annotations besides it)
>>  * Group collaboration — a group’s annotates a specific document (Google Docs-style but for any web page)
>>  * Pulling notes from sources one explicitly follows (I call this the “Twitter/RSS feed model” to annotiation — imagine seeing annotations from, or boosted by, people & organisations you follow — and only those people)
>> The last option of these sounds similar to your concept of “Annotation Sources”. Note however that this may create a huge problem for reader privacy, as a query for annotations is made to every annotation source upon every page you visit (while having annotations enabled). The only solution I can think of is that the browser (or a delegated service) retrieves *all* annotations from sources it follows and stores them locally, to not reveal which pages it visits.

>> *Publisher’s control*


>>> “Authors should have some control over which annotations can be shown (at least by default) when their page is being viewed.
>>> …
>>> This mechanism should give control to pages over third-party annotations on their content, but it shouldn’t limit what a user can do and see on their own device.”

>> I suppose some concerns of authors may come up, and compromises could be made (e.g. a page could *hint* to disable annotation), but as you write in some points in the explainer this is ultimately a front-end feature that users should control. If the graffiti concerns are avoided as just described, then one could argue that *every* annotation is something the user does and sees on their own device.

>> Therefore I would avoid the web page’s interference, and avoid some open questions like* **“Can the user prevent the page from filtering annotations?”*. A web page cannot tell the browser to avoid bookmarking it either.

>> Except for the page’s ability to block, I do like the schematic drawing with the “aggregated annotations” though. I often frame annotation as a *display of back-links*: My browser *knows* about a small section of the web, and when I visit a page, it displays (in a sidebar) other pages that link to that page. This by itself may be a concept worth working out; but in combination with “precise links” (text fragment links), we get web annotation.

>> *Making annotations available to the web page*

>> Best to be avoided, I think.

>> It seems impossible to let pages control annotation presentation without opening a can of worms, especially privacy issues. A page should not get access to the annotations’ content, and which words the annotation targets is already private content — so it would be impossible to let a page position the annotation without leaking information to the page.

>> If the annotations are public, then the web page can already fetch and display them without the browser’s aid; like a website can already load a hypothesis sidebar. I think the main point of web annotation is to view the web beyond&between the web pages themselves, without requiring the cooperation of those pages. So unless I miss something, I think this whole idea of involving the page in helping to annotate itself may best be forgotten about completely.


>> One way in which pages can however help the browser in showing annotations on the page: using good semantic html, specifying canonical and alternate URLs, using rel=bookmark links in <article>s, and so forth. This helps the browser to know which document(s) it is looking at — a point I do not see being mentioned in your explainer (many documents may appear under numerous URLs; e.g. sometimes query parameters matter, sometimes they do not).

>> (also, another challenge not mentioned in the explainer: the web lacks versioning or history, hence annotations can become ‘orphaned’)

>> *Browser extensions*

>> The case for web pages is very different from that of browser extensions. I like that you take extensions into consideration in the explainer.. Perhaps it is even okay if initially, for example, the browser itself can only display annotations, and browser extensions and CMSes will invent ways to create annotations.

>> A good example of this approach may be the work <> Mozilla did to enable distributed web protocols in Firefox: rather than implementing e.g. the IPFS or Hypercore protocol, they developed new WebExtension APIs that would let browser extensions implement such protocols (e.g. have extensions register protocol handlers and create UTP connections).

>> For web annotation, perhaps even some small changes could unblock other people (like myself) from implementing annotation capabilities through extensions; e.g. it might already help if Chromium implemented the Sidebar API <>, and would provide a way to draw lines and highlights in an overlay on top of a page without modifying its DOM..

>> Given that there may be many ways in which annotations could work, and experimentation may be needed, such an approach would nicely allow various experiments to slowly develop the concepts.

>> Nevertheless, to make annotation more part of the web, one should be able to share an annotation with other people without them having to install browser extensions; so at least a basic browser support for displaying a pair of document+annotations would be fantastic.

>> *Web platform*

>> I really like you are thinking how to make annotation part of “the web platform”, and that you are looking at web standards to implement. It would indeed be sad if this ends up being just a feature specific to Chromium (I hope Text Fragments gets wider adoption too). It needs a whole ecosystem around it (annotation publication software, annotation sources, …), that would ideally include existing annotation projects. It would be great if the millions of annotations made in Hypothesis and elsewhere could be displayed in the browser too. To get the relevant parties to discuss this, perhaps we can liven up the discussion in the W3C Open Annotation Community Group <> again?

>> As an overarching thought, I think the main questions this explainer provokes are not technical, but rather about the goal and the process towards it. Will the concept be developed by engineers, or with rounds of user research and feedback?  Will this be a collaborative effort to change the web at large, or Google using its dominant browser to push a feature through and wait for others to follow? Will it end up being a single product experience, or a framework that enables many browsers, browser extensions, publishers, CMSes and other parties to try out a variety of annotation(-ish) systems and use cases? I guess you might not have the answers ready either, but I am curious how you plan to go about.

> As said, I would be curious to hear the thoughts from others in this group. Is the interest in web annotation from Chromium developers (even if coming ‘late to the party’) an opportunity for web annotation? Or a hopeless/misguided mission? Or…?

> Kind regards,

> — Gerben


> On 12/05/2021 17.42, David Bokan wrote:
>> Hi everyone, 
>> Myself and a few colleagues from the Chrome team have been considering ways to bring some annotation use cases to the browser by default. This requires a lot of thought about what kind of  APIs and controls are given to page authors and users as well as the broader implications on the web ecosystem (e.g. security, privacy, UI, etc.)
>> We want to do this in a way that's open and integrates with existing specs and work on annotations. We've put up a public explainer <>; there's no concrete proposal yet, it's all very early stages. There are a few rough ideas though and explains how we're thinking about the challenges. We'll continue to develop the ideas there if you'd like to participate or just follow along.
>> Given this group's interest in annotations, I'd like to invite thoughts and feedback, particularly if anyone has any experiences with similar efforts in the past. 
>> Thanks!
>> David Bokan

Received on Tuesday, 1 June 2021 05:22:14 UTC