Re: Bringing annotation to the browser

(Apologies for being a bit late to the discussion… my overzealous filtering
on email, stuck this in a lower priority bucket till I just noticed it.)

I'll note that David reached out to me as well, and left a Hypothesis
annotation-as-invitation on the github document, so I went ahead and left
my remarks there.  (This link
<https://hyp.is/go?url=https%3A%2F%2Fgithub.com%2Fbokand%2Fweb-annotations&group=__world__>
will take you to them)

Will respond to a few of Gerben's points below...



On Mon, May 24, 2021 at 10:39 AM, Gerben <mail@treora.com> wrote:

> Hi everyone,
>
> I already sent David feedback privately, which I will share below. I am
> curious if anyone feels similar or has other remarks to share. As you can
> read below, I suggested David to discuss the plan in this Community Group,
> which has pursued highly similar ideas for several years. But the lack of
> response so far, to what sounds to me like a vague but exciting proposal,
> makes me wonder if anybody is still reading this list, and if anyone is
> still interested in the topic…
>

Yes!

Firstly, some possibly relevant context for those who missed it: David and
> his team already got a significant annotation-related change into the
> Chromium browser over the last year: Text Fragments
> <https://wicg.github.io/scroll-to-text-fragment/>, defining a special
> type of fragment identifier to point to (≈ scroll to and highlight) a
> quoted phrase inside the document. E.g. pointing at the words “illustrative
> example” on example.org page looks like this: (try open it in a recent
> Chromium-based browser <https://caniuse.com/url-scroll-to-text-fragment>)
>
>
> https://example.org/#:~:text=use%20in-,illustrative%20examples,-in%20documents
>
> Note for the Web Annotation-savvy: these Text Fragments are more or less a
> compacted version of serialising a TextQuoteSelector (or a RangeSelector
> containing two TextQuoteSelectors) into a URL, as was envisioned in the Selectors
> and States as Fragment Identifiers
> <https://w3c.github.io/web-annotation/selector-note/#frags> idea (that
> never made it into the Web Annotation recommendations). For comparison,
> here is a pointer at the same words using that latter serialisation:
>
>
> https://example.org/#selector(type=TextQuoteSelector,exact=illustrative%20examples,prefix=use%20in%20,suffix=%20in%20documents)
>
> Having the ability to point at an arbitrary selection in web pages seems a
> small but important step to annotating web pages.
>

Completely agree.  A fact which bodes well for the much expanded vision
he's now laying out in this new doc.

Back to the point, here is my response to David’s request for feedback on the
> mentioned explainer <https://github.com/bokand/web-annotations>:
>
> Making annotations part of the web has been a vision of mine for many
> years (like for many others). Time and again the conclusion has been that
> this should be part of user agents, so interest from Chromium opens lots of
> possibilities.
>
> Overall, I think your outline of your proposal is well-informed and
> reasonable, and seems to head roughly in the direction that I myself have
> been wanting to move things too; see e.g. my current project
> <https://nlnet.nl/project/WebAnnotation/> funded by the EU’s Next
> Generation Internet project, or my blog post
> <https://web.hypothes.is/blog/supporting-open-annotation/> from when I
> was an intern at Hypothesis (2014). I would be glad to exchange notes and
> collaborate in some way, join a call, or whatever works.
>
> Below some thoughts from reading the explainer.
>
> *Avoiding the “web graffiti” problem*
>
> I am very pleased to see that it is a non-goal to make *“a centralized
> “comments section” for the web”*. Many previous annotation projects would
> create a single central annotation store, and show all annotations to every
> reader. Then website publishers would often be furious about losing control
> over the presentation of their content. Moreover, lots of this content
> would be of low interest to most readers. Moderation does not scale well, a
> single party as moderator is a bad idea anyhow, etcetera.
>
>
Well put.

Given this history, I recommend being extra clear about the approach,
> because you’ll have many people asking “what about spam/harassment/…?”.
> When talking about my project visions I often have to make clear that,
> unlike in many other/previous projects, people will only see annotations
> when they asked for them, from sources they selected.
>
>
Completely agree.


> Consequently, I think the first item+subitems in your list of challenges
> are problematic:
>
>
> “Decentralized moderation: Whose responsibility is this? …”
>
>
> We could just as well ask: who moderates the WWW? That seems the wrong
> question. Boldly put: *If the annotation system requires a moderation
> system, it is a bad system.* The challenge should be how not to need
> moderation; otherwise it is doomed to fail — probably even before
> launching, it will be killed by a storm of criticism and bad publicity
> (harassment, disinfo and social media’s moderation failures are hot topics
> these days).
>
>
We're probably thinking the same thing, but I would put it a bit
differently.  I think the moderation responsibility lies with services, of
which there will be many.  Subscribe to the ones that you want to — just as
you visit the web pages that you want to.  If the user-agent (browser) is
de-facto a moderation agent, then that is def an issue.


> I would therefore strongly consider limiting the design and discussion to
> ] and use cases that do not pull annotations from just anywhere & anyone,
> but only get annotations the reader asked for:
>
>
>    - Personal private notes — stored in your browser, much like bookmarks
>
> Or stored w/ a service.  After all, some will want their personal notes
available on whichever browser / computer they're using.

>
>
>    - Sharing a document+annotations — e.g. I add notes to a page and
>    share the annotated page with you. (technically, perhaps I point you to a
>    collection of Annotations targeting the same document; your browser renders
>    its target document while showing the annotations besides it)
>
> +1


>
>    - Group collaboration — a group’s annotates a specific document
>    (Google Docs-style but for any web page)
>
>
+1


>
>    - Pulling notes from sources one explicitly follows (I call this the
>    “Twitter/RSS feed model” to annotiation — imagine seeing annotations from,
>    or boosted by, people & organisations you follow — and only those people)
>
>
Yes— I'd modify the language to say "... from *services* and sources one
explicitly follows…" but yes I agree w/ this.


>
>
> The last option of these sounds similar to your concept of “Annotation
> Sources”. Note however that this may create a huge problem for reader
> privacy, as a query for annotations is made to every annotation source upon
> every page you visit (while having annotations enabled).
>
>
While true— I think this is something that should be within the reader's
control. If they want to have the browser alert them to annotations on a
document from services that they're subscribed to, then that is a right
that should be granted to them.

If a reader doesn't want to communicate their IP to annotation services,
they can use a VPN.  Perhaps there might even be annotation-specific vpns
that don't require every connection in and out of your computer to be
proxied.

The only solution I can think of is that the browser (or a delegated
> service) retrieves *all* annotations from sources it follows and stores
> them locally, to not reveal which pages it visits.
>
>
Not quite sure I understand this distinction.


> *Publisher’s control*
>
>
> “Authors should have some control over which annotations can be shown (at
> least by default) when their page is being viewed.
> …
> This mechanism should give control to pages over third-party annotations
> on their content, but it shouldn’t limit what a user can do and see on
> their own device.”
>
>
> I suppose some concerns of authors may come up, and compromises could be
> made (e.g. a page could *hint* to disable annotation), but as you write
> in some points in the explainer this is ultimately a front-end feature that
> users should control.
>
>
Thank you Gerben.  Completely agree.  The hint might be a worthwhile
feature— though ultimately it feels like something which wouldn't work in
the way it was intended.  As a parallel example, if a web page said "don't
tweet about this" would I respect that?  Hmm… probably depends on what I
felt about the page owner and the rationale for such a request. A request
like this on a public web page is a little bit like shouting something from
a rooftop and at the same time saying "please don't talk about this with
anyone else".

If the graffiti concerns are avoided as just described, then one could
> argue that *every* annotation is something the user does and sees on
> their own device.
>
>
Yes.


> Therefore I would avoid the web page’s interference, and avoid some open
> questions like* “Can the user prevent the page from filtering
> annotations?”*. A web page cannot tell the browser to avoid bookmarking
> it either.
>
>
Agree.  btw… one strategy that is employed by some web pages are the
wordpress extensions that inject shrapnel into the DOM to interfere w/
anchoring.  (They don't however prevent annotations from orphaning and thus
being discovered at the root of those pages).

Except for the page’s ability to block, I do like the schematic drawing
> with the “aggregated annotations” though. I often frame annotation as a *display
> of back-links*: My browser *knows* about a small section of the web, and
> when I visit a page, it displays (in a sidebar) other pages that link to
> that page. This by itself may be a concept worth working out; but in
> combination with “precise links” (text fragment links), we get web
> annotation.
>
> *Making annotations available to the web page*
>
> Best to be avoided, I think.
>
>
Agree.


> It seems impossible to let pages control annotation presentation without
> opening a can of worms, especially privacy issues. A page should not get
> access to the annotations’ content, and which words the annotation targets
> is already private content — so it would be impossible to let a page
> position the annotation without leaking information to the page.
>
>
When we use the DOM or shadow DOM to store ranges, then presumably that
information is leaking?  (Or maybe not when shadow DOM is used...)


> If the annotations are public, then the web page can already fetch and
> display them without the browser’s aid; like a website can already load a
> hypothesis sidebar.
>
>
My thought exactly.

I think the main point of web annotation is to view the web beyond&between
> the web pages themselves, without requiring the cooperation of those pages.
> So unless I miss something, I think this whole idea of involving the page
> in helping to annotate itself may best be forgotten about completely.
>
>
Yes.


> One way in which pages can however help the browser in showing annotations
> on the page: using good semantic html, specifying canonical and alternate
> URLs, using
> rel=bookmark
> links in
> <article>
> s, and so forth. This helps the browser to know which document(s) it is
> looking at — a point I do not see being mentioned in your explainer (many
> documents may appear under numerous URLs; e.g. sometimes query parameters
> matter, sometimes they do not).
>
>
+1


> (also, another challenge not mentioned in the explainer: the web lacks
> versioning or history, hence annotations can become ‘orphaned’)
>
> *Browser extensions*
>
> The case for web pages is very different from that of browser extensions.
> I like that you take extensions into consideration in the explainer.
>
>
Me too :)

Perhaps it is even okay if initially, for example, the browser itself can
> only display annotations, and browser extensions and CMSes will invent ways
> to create annotations.
>
>
Or maybe the browser provides primitive services, like fast anchoring and
fuzzy matching, that can be used by extensions or its own implementation,
etc.


> A good example of this approach may be the work
> <https://github.com/mozilla/libdweb> Mozilla did to enable distributed
> web protocols in Firefox: rather than implementing e.g. the IPFS or
> Hypercore protocol, they developed new WebExtension APIs that would let
> browser extensions implement such protocols (e.g. have extensions register
> protocol handlers and create UTP connections).
>
> For web annotation, perhaps even some small changes could unblock other
> people (like myself) from implementing annotation capabilities through
> extensions; e.g. it might already help if Chromium implemented the Sidebar
> API
> <https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/user_interface/Sidebars>,
> and would provide a way to draw lines and highlights in an overlay on top
> of a page without modifying its DOM.
>
> Given that there may be many ways in which annotations could work, and
> experimentation may be needed, such an approach would nicely allow various
> experiments to slowly develop the concepts.
>
>
Yes


> Nevertheless, to make annotation more part of the web, one should be able
> to share an annotation with other people without them having to install
> browser extensions; so at least a basic browser support for displaying a
> pair of document+annotations would be fantastic.
>
>
Yes!


>
> *Web platform*
>
> I really like you are thinking how to make annotation part of “the web
> platform”, and that you are looking at web standards to implement. It would
> indeed be sad if this ends up being just a feature specific to Chromium (I
> hope Text Fragments gets wider adoption too). It needs a whole ecosystem
> around it (annotation publication software, annotation sources, …), that
> would ideally include existing annotation projects. It would be great if
> the millions of annotations made in Hypothesis and elsewhere could be
> displayed in the browser too. To get the relevant parties to discuss this,
> perhaps we can liven up the discussion in the W3C Open Annotation
> Community Group <https://www.w3.org/community/openannotation/> again?
>
> As an overarching thought, I think the main questions this explainer
> provokes are not technical, but rather about the goal and the process
> towards it. Will the concept be developed by engineers, or with rounds of
> user research and feedback?  Will this be a collaborative effort to change
> the web at large, or Google using its dominant browser to push a feature
> through and wait for others to follow? Will it end up being a single
> product experience, or a framework that enables many browsers, browser
> extensions, publishers, CMSes and other parties to try out a variety of
> annotation(-ish) systems and use cases? I guess you might not have the
> answers ready either, but I am curious how you plan to go about.
>
> As said, I would be curious to hear the thoughts from others in this
> group. Is the interest in web annotation from Chromium developers (even if
> coming ‘late to the party’) an opportunity for web annotation? Or a
> hopeless/misguided mission? Or…?
>

I think this is a fabulous (conceptual) effort, and I'm really excited by
the kind of thinking that I see in David's document. With very minor
quibbles, some of which you've highlighted above, I think he gets the spirt
of this exactly right.


> Kind regards,
>
> — Gerben
>
>
> On 12/05/2021 17.42, David Bokan wrote:
>
> Hi everyone,
>
> Myself and a few colleagues from the Chrome team have been considering
> ways to bring some annotation use cases to the browser by default. This
> requires a lot of thought about what kind of  APIs and controls are given
> to page authors and users as well as the broader implications on the web
> ecosystem (e.g. security, privacy, UI, etc.)
>
> We want to do this in a way that's open and integrates with existing specs
> and work on annotations. We've put up a public explainer
> <https://github.com/bokand/web-annotations>; there's no concrete proposal
> yet, it's all very early stages. There are a few rough ideas though and
> explains how we're thinking about the challenges. We'll continue to develop
> the ideas there if you'd like to participate or just follow along.
>
> Given this group's interest in annotations, I'd like to invite thoughts
> and feedback, particularly if anyone has any experiences with similar
> efforts in the past.
>
> Thanks!
> David Bokan
>
>