Re: [W3C Media Annotation WG] Request for Expert Review (API)

Hi Chris,

Thanks for your feedback on the review. I will put some more comments
inline where there needs to be clarifications.

Group feedback will indeed be of value!


On Thu, Apr 15, 2010 at 1:43 AM, Chris Poppe <Chris.Poppe@ugent.be> wrote:
>
> 2. Section 2.1: Language
>
> I would not define Language as a part of the return value of some of
> the properties - it is very unlikely that a annotation uses different
> languages for different properties. I would only manage the language
> on the level of the metadata source level.
>
> [CP] It could be the case that different annotations are available for one
> media resource (e.g. 3 MPEG-7 files). I agree that within one annotation
> typically the language remains the same, but I would not be surprised to see
> multi-lingual annotations appear in the future (e.g., a review by a French
> man on a Hungarian movie).

Even in this case, your annotation will be, e.g. a title and an author
and a genre and a publisher - these will all be given in the same
language, e.g. all in English. It seems we are mixing two different
uses of "language" here - one that describes the language in which the
metadata is given and one that is actually metadata about the content.
You might want to separate these.


> 3. Section 2.2: setContext
>
> I would suggest renaming that function. Also, your parameters are not
> quite right:
>
> boolean setMAResource ( in DOMString mediaResource, in optional
> MetadataSource[] metadataSources);
>
> (note the replacement of "Object" with "MetadataSource").
>
> [CP] Any suggestions on the new name? (setSources, setMediaResource +
> setMetadataSources, ...)?

My suggestion was setMAResource (see the interface description I gave).



> 3. Section 2.3
>
> You need to make sure that the full definition of "getProperty" is
> printable (for me, it went off the right edge of my page).
>
> Also, you only return one MAObject IIUC, so don't make the return value a
> list.
> [CP] Well we want to include the possibility that you can get back
> information from different annotations (EXIF, DC, MPEG-7) even of the same
> standard (2 independently created DC annotations).

I would totally want several calls for that, each requesting a different format.


> Furthermore, for some
> properties you could get back different values (e.g., if a movie has two
> titles), different languages (French and English title) and so on.

These are already part of the structures that the individual elements
return. Some of them return lists, others just single elements. They
should not return multiple MAObjects since a MAObject relates to one
set of resources.


> Also, I believe that the API should be independent of a format and
> thus the sourceFormat and the subtype parameters are not useful.
>
>
> [CP]The idea was that in some cases an application might want to get back
> metadata of a specific format only. In any case, when not including a
> sourceformat you get back the metadata regardless the underlying format. The
> subtype allows to refine a search for metadata (e.g., requesting for the
> contributors gives both producers and actors. Using the subtype you could
> ask for only those contributors that are actors).

IIUC the idea of Media Annotations API is to provide an API
independent of the source format. If you wanted to directly use an
existing format, you'd use an API that is provided for that particular
format. Is is impossible to return in a MAObject data in the unmapped
format of the underlying media resource - all you can do is return a
text string of all the annotations. If that is really desired, that
should be a different API call, something like
getMetadataResource(file).

I have now understood the use of "subtype" - but you will need to
register a lot of default strings there for all the elements.


> Also, I believe the API should be independent of the language in use -
> if you really wanted language selection to be made on the metadata
> files, then this should be handled at a different level.
> [CP] If nothing is filled in for the language argument it is in fact
> language independent. Do you think that the optional language argument is
> really an issue?

See above - we need to be clear which use case we are satisfying with
the "language" argument and rather than making it an argument, I would
make it a part of the return struct from the function calls.



OK.


> Thus, it should probably look like:
>
> MAObject getProperty(in DOMString propertyName, in optional DOMString
> fragment);
>
> Note that I am not quite sure how to represent in Web IDL that the
> MAObject is the union of all the possible return values for all
> getProperty requests. I don't think how it is currently done is quite
> correct, since it only returns the parent class, not the child
> classes.
>
> [CP] Well, all possible return values (depending on the requested
> propertyName) are objects that implement specific interfaces (which inherit
> from the MAObject interface). I assume(d) that this can be represented in
> WebIDL as such.

May well be. :-) I'm not a WebIDL expert.


> Further, I would suggest as an example for how to introduce this into
> a UA: the  following example for how to introduce it into HTML5:
>
> interface HTMLMediaElement : HTMLElement {
>  ...
>  getter MAObject getProperty(in             DOMString propertyName,
>                                            in optional unsigned long
> trackIndex);
>  ...
> }
>
> (I don't think any other type of fragment than the track index makes
> sense for HTML5).
> [CP] Adding such an example seems like a good idea to me. However, I am not
> sure about the trackIndex, or the absence of the other arguments (as
> mentioned above).

Feel free to adapt the suggestion - you will anyway have to expect the
HTML WG to adapt it further when you introduce it there.


> 4. Section 2.3.1.1 Identifier
>
> Your examples are all a bit incomplete in their JSON representation. I
> will write a full example for this section - the others need to be
> changed logically to this, too, though I will only indicate this where
> it may be confusing.
>
> [CP]I agree, this is being worked on. I believe your example is an excellent
> starting point!

OK.


> Example:
<..>
> Usage as a UA call in JavaScript in a Web Browser:
>
> video = document.getElementByTagName("video")[0];
> id = video.getProperty("identifier");
>
> Resulting in:
> id.value =
> "http://www.w3c.org/2008/WebVideo/Annotations/wiki/Image:MAWG-Stockholm-2009
> 0626.JPG"
> and id.type = "URI"
> and the other values as above in JSON
>
> [CP] I am assuming that this example is in the case of the getProperty has
> been added to HTML5?

Yes.

> However, in the current state it would rather be:
> id[0].value =
> "http://www.w3c.org/2008/WebVideo/Annotations/wiki/Image:MAWG-Stockholm-2009
> 0626.JPG"
> id[0].type = "URI"

What is id[0] supposed to be?
"video" is the video resource.
"video.getProperty" provides the media annotations information for
that resource for the specified dimension, which is "identifier".
"identifier" returns a struct consisting of value and type. There can
only be one such struct. Thus, it's id.value and id.type that address
the elements in the struct.


> 5. Section 2.3.1.2 Title
>
> * remove "Language" from the inheritance list - should not be per
> property but per metadata resource
>
> * Example should be:
> title.value = "MAWG Stockholm 20090626"
> title.type = "Video title"
>
>
> [CP] I agree. However, for the type, the fact that it is a title of a video
> is typically not always described in the metadata itself. Would you suggest
> that the application needs to deduce this information somehow? (e.g., from
> the fact that the mediaelement is a HTMLVideoElement like in HTML5?).

"type" could of course be empty. I just assumed it was easy to deduct
that it is a video title from the given resource.



> 6. Section 2.3.2.1 Contributor
>
<..>

> * in this example, you have a set of return values, so the spec needs
> to look different:
>
> interface Contributors {
>    attribute DOMString value;
>    attribute DOMString type;
> }
>
> interface Contributor: MAObject {
>    attribute Contributors[] contributor;
> }
>
> * JSON reply thus looks something like:
>
> {    "Contributor": {
>           // This first lot is inherited from MAObject, though I am
> unclear what the values should be
>            "unstructuredValue": ???,
>            "uri" : ???,
>            "sourceFormat": ???,
>            "fragmentIdentifier": ???,
>            "mappingType": ???,
>            "contributor": [ {
>                     "value": "http://individuals.example.com/Contributor1",
>                     "type": "editor"
>                     },
>                     {
>                     "value": "http://individuals.example.com/Contributor2",
>                     "type": "producer"
>                     }]
>     }
> }
>
> [CP] As mentioned before we could always have a set of return values.

I would recommmend against a set of MAObjects be created as a return
value for a single function call.


> 7. Section 2.3.2.2 Creator
>
> * replace "StringObject" with "Creator"
>
> [CP] StringObject was used for each property that just holds a string. It
> might indeed be better to define appropriate interfaces.
> Will change this if the group agrees.

I think they have been changed elsewhere - I was pointing out an
inconsistency here.


> 8. Section 2.3.2.3 CreationDate
>
> * you talk about both "CreationDate" and "Date" here. If you want this
> value to have a type of e.g. "creation date" and "publish date", it
> should not be called "CreationDate", but only "Date".
> [CP] I agree that "createDate" as propertyname is not a good name to
> represent both "Create Date" and "Publish Date" subtypes. I am not fond of
> the use of "date" as propertyname either since it sounds a bit too generic.

If you allow multiple types of dates to be used in this dimension then
you need to have subtypes here and the function needs to be generic.


> 22. Section 2.5.3 getOriginalData
>
> * the return value should not be a DOMString list - it's simply a
> single, very long string for the requested source format - not for all
> possible formats
> [CP] We assume that for one format multiple annotations can be available
> (e.g., different MPEG-7 files). So this is the reason for the list of
> DOMStrings

Parsing of those formats and presentation of their content in native
form is not the aim of the Media Annotations API. Rather, an abstract
API to deal with all formats is the goal. So, you should not provide
helper functions for every possible media format. That should be
provided through a custom library for those formats.



Cheers,
Silvia.

Received on Thursday, 15 April 2010 00:19:34 UTC