Re: Expressing complex regions with media fragments - use cases + possible solution

On Thu, Sep 9, 2010 at 4:35 PM, Davy Van Deursen
<davy.vandeursen@ugent.be>wrote:

> Hi Silvia, Jack,
>
> >>> On Wed, Sep 8, 2010 at 7:00 PM, Jack Jansen <Jack.Jansen@cwi.nl>
> wrote:
> >>>
> >>> On 8 sep 2010, at 02:51, Silvia Pfeiffer wrote:
> >>> > I have, however, a pretty big caveat with standardising this
> approach: right now we are discussing with the browser vendors on
> how to present spatial media fragment URIs. There is a preference to use
> them for splicing pictures, i.e. for rendering only the
> referenced image or video region.
> >>> >
> >>> > I do not believe that matches your intentions here. IIUC your
> intentions here are to only have a means to provide annotations
> to regions. I think this is more of a "image map" type approach than an
> "image splicing" approach - correct me if I'm wrong.
> >>>
> >>> We've got to be careful here: browser vendors tend to think that the
> whole world revolves around the browser:-) A media fragment
> is basically nothing more than a specification of a portion of a video
> (audio, image) resource, and even though we give guidelines
> on how to present such a fragment in the browser that doesn't mean it's the
> only application of media fragments. I would assume that
> the Media Annotation folks couldn't care less about presentation: they just
> want to be able to point at something inside the video.
> >>>
> >>>
> >>> Except: everyone wants to see their results in the browser. So, if the
> browser vendors agree to present a spatial media fragment
> URI as an image slice, then you can do as many annotations as you want with
> such a URI, it will never be presented as an image with
> highlights by the Web browser. So, as long as we are talking about a
> presentation in a Web browser, we are actually talking about
> the same application doing the same thing with the same URI.
> I'm 100% in agreement with Bernhard here: our main task is in defining how
> to address subparts of media items, very similar to how
> #xmlid addresses subparts of an XML document. For practical reasons, we
> also define guidelines on how we think some classes of
> applications should render some of our MF-based addresses, but that is much
> less important. Moreover, we can provide no more than
> guidelines.
> >>
> >>There's also a problem with your (Silvia) reasoning that reasoning about
> "the browser" is similar to reasoning about "the
> operating system": the browser is simply a platform on which an application
> runs. Clearly, if someone types an MF-url into the
> location bar, something consistent should happen. But more often than not
> the MF-url will be used in the context of an application
> (either client-side or server-side), and this application will know best
> what to do. An annotation-viewing app will most probably
> want to show the whole original resource with some form of highlighting on
> the portion selected by the MF.
> >>
> >>As an aside: this means I also disagree with Ian's WHATWG mail
> <
> http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2010-August/028163.html>:
> we define addressing, but the semantics of using
> those addresses should be formally explained in the specification
> referencing our work. In case of HTML it would be best if it
> provided a default rendering semantic (such as "crop to the specified
> area") with a CSS-based or attribute-based method to override
> this ("draw the whole resource, and make the area selected by the MF
> available to scripts through the DOM").
> >
> >How about we define or at least propose the semantics for use in the HTML
> <video> and <audio> elements and the HTML address bar.
> Then we can leave it to the other use cases to do what they want with the
> URIs?
>
> I fully agree with Jack: ' we can provide no more than guidelines '. For
> the browser case, something consistent should indeed
> happen. A proposal might be as follows:
> - HTML address bar: choose either to crop or to highlight the fragment (I
> would prefer to crop the fragment (along all axes))
> - <video> and <audio> elements: I think both rendering modes should be
> available here (i.e., both cropping and highlighting).
> However, I don't think the MF spec should foresee this switch (as proposed
> in our last phone conf [1]). More specifically, in this
> case, the switch could be provided by the HTML spec, for example by adding
> an attribute to the <video> or <audio> element indicating
> how fragments should be rendered.
>


I was under the impression we were discussing to put the extra information
for deciding how to render it on the URL.
Adding a new attribute to the HTML5 media elements is an interesting
alternative. I don't think it will fly, though, because what is the meaning
of this attribute if the URL does not have a spatial fragment? And what is
the browser to do if the author forgets to add the attribute?
I think we may even find it hard to get the browser vendors to implement one
version of it, not to talk of a second one.
Also, we may find that they will only accept spatial fragments for images
(just like Bernhard is saying, too).

I'd go as far as saying that if it is possible to write the specification of
the region highlight in a document rather than in a url as Bernhard has done
in his examples, there isn't a need for reference URLs and the svg-fragment
example is perfectly fine. Just use SVG filters:

<?xml version="1.0" ?>
<r:RDF xmlns:r="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
      xmlns:a="http://www.w3.org/2000/10/annotation-ns#"
      xmlns:d="http://purl.org/dc/elements/1.1/">
 <r:Description>
 <a:annotates r:resource="http://serv1.example.com/some/page.html"/>
<x:svg-fragment rdf:parseType="Literal">
<svg:defs xmlns:svg="http://www.w3.org/2000/svg">
 <svg:filter id="c1">
      <svg:symbol id="Polygon" xmlns:svg="http://www.w3.org/2000/svg">
             <svg:polygon fill="none" points="52,203 0,124 186,0 238,80
52,203 " stroke="rgb(255,0,0)" stroke-width="0" xmlns:svg="
http://www.w3.org/2000/svg"/>
      </svg:symbol>
  </svg:filter>
</svg:defs>
<svg:img height="270px" width="480px" "http://example.com/image1.jpg"
filter="url(#c1)"/>
</x:svg-fragment>
</r:Description>
</r:RDF>

You can define the mask in that same file (as done above) or in a different
file, so you will basically get what you want for free with the @filter and
relative url there.

Wouldn't that do it?

Cheers,
Silvia.