Re: JSON metadata cues?

On Fri, Oct 18, 2013 at 3:48 PM, Brendan Long <self@brendanlong.com> wrote:

>  I'm looking at implementing the CableLabs "Exposing In-Band Media
> Container Tracks in HTML5<http://html5.cablelabs.com/tracks/media-container-mapping.html>"
> spec, and it seems to conflict with the idea behind DataCue*, so I'm
> wondering how people would feel about a cue with no .data attribute, and a
> getCueAsObject(), returning a JavaScript object.
>

I don't see any conflict. And don't see any need for not having a data
attribute, or for having getCueAsObject().


>
> The reason this would be helpful if that, at least in GStreamer, getting
> raw binary data is actually relatively difficult, and we easily have access
> to structured data.
>

Really? I don't think so. Just use

new DataCue(*start*, *end*, stringToUint16Array(*jsonText*))


For example, we have
GstMpegTsSection<http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-bad-libs/html/gst-plugins-bad-libs-Base-MPEG-TS-sections.html>for
MPEG-TS metadata and
> GstToc<http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/gstreamer-GstToc.html>for the table of contents (which converts nicely to WebVTT already). I
> assume other kinds of metadata will also be exposed this way, not as raw
> binary streams, like the spec seems to expect.
>
> With structured data, it would be nice if we could just throw it into a
> JSON object, like so:
>
> interface JSONCue : TextTrackCue <http://www.w3.org/html/wg/drafts/html/master/single-page.html#texttrackcue> {
>       readonly attribute DOMString   text <http://www.w3.org/html/wg/drafts/html/master/single-page.html#dom-datacue-text>; /* JSON */
>       readonly attribute object getCueAsObject(); /* JavaScript object */
> };
>
> You really don't need the syntactic sugar of adding a getCueAsObject()
method, but if you really want this method, e.g., to cache the object
decoded from JSON, then just add yourself:

cue.getCueAsObject = function() {
  if (!this.cachedObject)
    this.cachedObject = JSON.parse(this.text);
  return this.cachedObject;
}

In this case, there really isn't much point in defining a new JSONCue
interface; just use DataCue.



> The advantages are:
>
>    - It's easier to do with GStreamer.
>
> No it isn't. There is no requirement to populate *data*.

>
>    - It should be easier for JS developers to work with.
>
> Not particularly. See above.


>
>    - At least in MPEG-TS, tracks aren't easily separated, so providing a
>    metadata track without parsing it doesn't really make sense. This may not
>    apply to other formats though.
>
> Of course you have to parse it. DataCue is not expected to contain a TS
byte stream, a stream of TS packets, a stream of TS sections, etc.

Any implementation that uses DataCue should document the circumstances in
which it is constructed, what the binary data buffer contains, and how the
text field, if non null, represents that data.

Implementations would normally used an internal type that effectively
represents a subclass of DataCue. But this doesn't mean there is a need to
export a JS version of that subclass when DataCue will do.

>
>    -
>
> The disadvantage is:
>
>    - It may be harder to create in non-GStreamer media pipelines.
>
> My question is, does this make sense for other implementors, or is this
> only easier for GStreamer? We *could* rebuild the binary format from the
> structured data if we needed to, I just want to make sure we're not doing
> something unnecessarily complicated if providing structured data makes more
> sense for other media pipelines too.
>
> * I realize we could just use DataCue and put JSON in the text and data
> fields, but that doesn't seem to be what it was meant for, so I don't want
> to do that unless the concensus is that it's "the right thing to do".
>

Why do you think it wasn't meant for this? That was what it was intended to
do. Again, implementations that populate a track with DataCue instances
should document how they are using DataCue in this regard.