Re: Rough Draft of Robust Anchoring: the RangeFinder API

Hey Doug,

> On 25 Feb 2015, at 15:01 , Doug Schepers <schepers@w3.org> wrote:
> 
> Hey, Ivan–
> 
> Thanks for the feedback. Replies inline...
> 
> On 2/25/15 12:49 PM, Ivan Herman wrote:
>> 
>> (Caveat 1: I am currently on the plane, queueing this comment; I am
>> not sure when I will be online again, ie, when it will be mailed...)
>> (Caveat 2: this type of Interface design is not really my technical
>> area, so apologies if what I say is fundamentally incorrect:-)
>> 
>> Some questions came to my mind while reading this.
>> 
>> - the querySelector is defined as an xpath selector. Shouldn't we
>> define a similar mechanism for CSS selectors? (Note that the charter
>> lists the CSS selector document as our input, too[1]).
> 
> That's not correct. There are 2 separate criteria attributes, 'querySelector' [1] for CSS selectors, and 'xpathSelector' [2] for XPath selectors.

Oops, right. Sorry for the noise.

> 
>> - a similar question for SVG. The annotation model has an SVG "area
>> selector", shouldn't that be reflected in this document? And you know
>> SVG way better than I do, I am sure that there is a possibility for
>> other types of selectors for SVG.
> 
> You can't currently (and probably never will) be able to express such things as a DOM range. And the RangeFinder API deals exclusively with DOM Ranges.
> 

O.k. This is an important thing to emphasize, ie, it is not the solution for everything under the sun:-)

> What you can do is supplement the range with additional "selectors" or "indicators". So, for example, if your app wants to highlight an area on an image, RangeFinder would find the image element itself, and once you have that, your app can take additional selectors defined in the WA Data Model and apply those as some sort of overlay on top of the image.

I am still wondering (but this is just jet-lagged musing at an airport...) whether some of the concepts used for DOM cannot be generalized. The idea of this 'prefix', ie, the '32 characters before the selection' may be seen as defining an interval on a one dimensional space which are the lines of text; a bounding box in SVG may be considered as something similar in 2D. But, again, this is just musing...

> 
> Note that RangeFinder doesn't do any highlighting itself. It just locates and returns the range. I have a separate proposal for how you can highlight a range in CSS, which I've not yet written up.

Sure, of course.

> 
> But speaking of SVG, you can search for arbitrary markup, like an SVG element, as a string using the 'markup' attribute [3], or using the 'querySelector' or 'xpathSelector' attributes.
> 

Yes, that is of course possible, I agree. And, actually, the CSS selection is also usable.

> 
> On a side note, I am not convinced that the SVG area selector defined in the data model spec is quite right yet.
> 
> I have some thoughts on SVG as an area selector (and more), which you can see in action (with some very simple spec-like prose) in my Annoshapes project [4].
> 
> 
>> Again another use case is selectors in media like video
> 
> That's possible using the 'fragment' attribute [5]
> 
> 
>> (- In general, the various selector specifications, ie, in this
>> document and in the annotation model should probably be reconciled,
>> also in term of their textual definitions.)
> 
> Yes, I hope that RangeFinder might help inform the selectors in the Data Model, in the same way that the Data Model informs the RangeFinder spec.
> 
> Note that annotation and robust anchoring/deep linking are only 2 uses for the RangeFinder API; it's also intended to provide a general client-side find-in-page mechanism for developers.
> 
> Thus, I'm not yet certain that we should reconcile the terminology, because the domain of the RangeFinder API is not the same as that of the Data Model spec; where possible, I'm using terminology derived from client-side features, DOM ranges, and APIs. For example, I used 'querySelector' for the CSS-based selector, inspired by the querySelector() DOM method, because that's going to be easy for developers to understand and remember; but the Data Model spec doesn't seem to mention CSS-based (or XPath) selectors at all; it only mentions CSS as a styling mechanism.
> 

Which is perfectly fine of course, because we are talking about DOM related methods. But I do believe we should make an attempt to reconcile these two. The API return information and that information, in the model, specify the target of an annotation through the selectors; it would be a bit disturbing if these two followed a different terminology and philosophy. Just my 2 cents...

> Now that I think of it in terms of RangeFinder, the Data Model seems a bit odd in its use of selectors, both in which ones it includes (or excludes), and in the way it describes and organizes them. It doesn't quite reconcile with client-side selectors; maybe that's okay, and the current Data Model selectors are more useful for the server-side/linked-data case;

I do not believe so. If the data model cannot be used for client side then we have a problem. At least in my view.

> or maybe we should look at expanding or changing the Data Model notion of selectors, at the risk of making it different than the Open Annotation Data Model.

If it is different than the output of the OA CG's document, then so be it. But we may be able to reconcile things by harmonizing the terminology and the definitions.

> 
> 
>> - Maybe an answer to the previous questions is related to a more
>> general question: I do not understand in the WebIDL spec how you
>> provide 'extension points', ie, that users could plug in their own
>> selection/search methods. I think we all agree that providing such
>> extension points is important
> 
> I agree that it's important. The short answer is, the 'customSelector' attribute is meant as the extension point for custom searches, and the intention is that the developer would pass in their own JavaScript function that uses additional arbitrary searching criteria and algorithms on the content, in some as-yet-undefined way.

Ah, o.k. It is probably my non-existent WebIDL knowledge, but, at the moment, it says:

"customSelector of type DOMString"

but then this should rather be some sort of a call back, right?

> 
> The longer answer is that you may be conflating the larger problem set with what is possible with RangeFinder in a couple of significant ways:
> 
> 1) as mentioned before, how you highlight or render a selection is not part of RangeFinder, that's part of the webapp; see my earlier image-area example;
> 

Correct, and I do not think I said anything on that effect...

> 2) some selectors are not only not expressible as a DOM range, but they aren't even client-side operations; for example, I might have a geo-coordinate selection on a dynamic map; when the page containing the map loads, getting the map element (probably an iframe) is the only part that's in the province of the RangeFinder; communicating with the map application so that it serves up the requested tiles that contain the geo-coordinates is completely outside the scope of the RangeFinder API, and so it wouldn't be appropriate to provide an extension point for that.

O.k., understood. The question is, however: do we try to attack some of these use cases as well, or should we restrict ourselves to the simple case for now? (I do not have the answer...)

> 
> 
>> - I do not have a clear model in my mind how the the interface works
>> in terms of asynchronity/promises. Shouldn't the search method also
>> return a Promise? What runs asynchronously and which are just
>> synchronous calls? (Probably an example in Javascript could help.)
> 
> Possibly. I'm torn between simplicity of understanding and use and flexibility and performance.
> 
> If anyone agrees we should be returning a Promise, I'd appreciate a suggestion about how to spec that out.
> 
> 
>> - The digital publishing world may have a separate use case: if I
>> make a search for a text, which itself is presented in a paginated
>> way, I would like to know on which *page* the result is. At this
>> moment the information about pagination is difficult to get to, and
>> the solutions are highly implementation dependent. However, the CSS
>> WG has just started the so-called Houdini project that, if my
>> understanding is correct, should provide a scripting interface to the
>> information related to CSS structures (including paging). I just
>> wonder how these mechanisms could be combined with RangeFinder. (Note
>> that the Houdini project is at its infancy, and our answer may very
>> well be that, in the first phase, we concentrate on DOM-related
>> information only, leaving the CSS stuff for later. But it is worth
>> some thought nevertheless...)
> 
> Yes, that's an interesting use case. I don't know how or even if it can be solved today,

Each 'reading system' does it differently, afaik, and with what is usually described an undescribeable hack:-( But I do not know the details.

> or if not, how it will be done in the future, but either way, that's really more of a DOM Ranges/CSS feature, not a feature of the RangeFinder API; the only thing the RangeFinder will do is find and return the range; it's up to other parts of the Web Platform to define how you get the scroll/page position of that range, how you navigate to it, how you style it, etc.

Ok, I understand the restriction of RangeFinder, and I am fine with this. However, this certainly _is_ possibly important use case for digital publishing which would deserve looking into.

Thanks Doug

Ivan


> 
> 
>> Before I forget, b.t.w.: thanks for writing this down!
> 
> Glad to finally have it out of my brain and onto "paper"!
> 
> 
> [1] http://w3c.github.io/web-annotation/api/rangefinder/#widl-RangeFinder-querySelector
> [2] http://w3c.github.io/web-annotation/api/rangefinder/#widl-RangeFinder-xpathSelector
> [3] http://w3c.github.io/web-annotation/api/rangefinder/#widl-RangeFinder-markup
> [4] https://shepazu.github.io/annoshape/annoshape.html
> [5] http://w3c.github.io/web-annotation/api/rangefinder/#widl-RangeFinder-fragment
> [6] http://w3c.github.io/web-annotation/api/rangefinder/#widl-RangeFinder-customSelectorhttp://w3c.github.io/web-annotation/api/rangefinder/#widl-RangeFinder-customSelector
> 
> 
> Regards–
> –Doug
> 
>> Cheers
>> 
>> Ivan
>> 
>> [1] http://www.w3.org/community/cssselfrags/
>> 
>> 
>>> On 25 Feb 2015, at 24:48 , Doug Schepers <schepers@w3.org> wrote:
>>> 
>>> Hi, folks–
>>> 
>>> Just a quick note. Rob asked me to move this file, to keep the
>>> deliverables organized. It's now located at:
>>> 
>>> http://w3c.github.io/web-annotation/api/rangefinder/
>>> 
>>> Even this is a temporary location, though... I'll be moving it to
>>> specs.webplatform.org soon, and adding the annotation capability to
>>> it.
>>> 
>>> Feel free to review, but be aware that the URL is transitory.
>>> 
>>> Regards– –Doug
>>> 
>>> On 2/24/15 1:33 PM, Doug Schepers wrote:
>>>> Hi, folks–
>>>> 
>>>> After talking about Robust Anchoring with many people over the
>>>> course of the last couple years (!), with encouragement and good
>>>> criticisms, I've refined my notion of what's needed for a
>>>> client-side API for Robust Anchoring.
>>>> 
>>>> I've drawn up a strawman of my current thinking for an API
>>>> called RangeFinder [1].
>>>> 
>>>> It's very rough in places, but I'd appreciate any feedback on the
>>>> spec as it stands. I'd greatly appreciate any thoughts or
>>>> opinions on it at this stage.
>>>> 
>>>> I'm not sure it's mature enough for this yet, but at some point,
>>>> I'd like to engage the research and academic communities and the
>>>> experts who've published on text search algorithms, to polish
>>>> this up and make it not quite as embarrassing as it is currently.
>>>> If anyone knows who we should contact in that regard, please
>>>> chime in. This is a great opportunity to leverage all that
>>>> research in the service of Web developers and browsers!
>>>> 
>>>> [1] http://w3c.github.io/web-annotation/rangefinder-api/
>>>> 
>>>> Regards– –Doug
>>>> 
>>> 
>>> 
>> 
>> 
>> ---- Ivan Herman, W3C Digital Publishing Activity Lead Home:
>> http://www.w3.org/People/Ivan/ mobile: +31-641044153 ORCID ID:
>> http://orcid.org/0000-0003-0782-2704
> 
> 
> 


----
Ivan Herman, W3C
Digital Publishing Activity Lead
Home: http://www.w3.org/People/Ivan/
mobile: +31-641044153
ORCID ID: http://orcid.org/0000-0003-0782-2704