Re: HTML5 support for track metadata

From: Brendan Long <B.Long@cablelabs.com<mailto:B.Long@cablelabs.com>>
Date: Tuesday, March 4, 2014 at 1:10 PM
To: "public-inbandtracks@w3.org<mailto:public-inbandtracks@w3.org>" <public-inbandtracks@w3.org<mailto:public-inbandtracks@w3.org>>
Subject: Re: HTML5 support for track metadata
Resent-From: <public-inbandtracks@w3.org<mailto:public-inbandtracks@w3.org>>
Resent-Date: Tuesday, March 4, 2014 at 1:11 PM

The wiki says:
Set trackMetadata to the concatenation of: the string “video/mp2t”, the stream type byte and the descriptor bytes bytes, expressed in hexadecimal using uppercase ASCII hex digits.
https://www.w3.org/community/inbandtracks/wiki/Main_Page#In-band_Metadata_Attribute_Alternative

Is that what you're suggesting adding?

No. I am proposing extending HTML5 support for the 3rd option, i.e. Using inBandMetadataTrackDispatchType for metadata tracks and kind for all other tracks.

If so, wouldn't it be better to not mix the two types of data, and have one attribute give the format (MPEG-TS), and one attribute give the metadata (as an ArrayBuffer if possible, so we can avoid unnecessary base64 encoding).

Is there any reason not to use DataCues in metadata tracks like we discussed previously?

DataCues would be used for metadata tracks, as stated here http://www.w3.org/html/wg/drafts/html/CR/embedded-content-0.html#guidelines-for-exposing-cues-in-various-formats-as-text-track-cues.



On Tue, 2014-03-04 at 19:50 +0000, Bob Lund wrote:
Hi CG,


I think that the existing HTML5 CR spec [1] is very close to supporting the use cases that are being discussed in the CG. I propose submitting several HTML5 bugs to close the gap and I’m interested in your thoughts.


An alternative described in the CG wiki is to use kind and inBandMetadataTrackDispatchType attributes to expose track metadata: kind is used for all tracks except metadata text tracks and inBandMetadataTrackDispatchType is used for metadata text tracks. This covers the use cases that have been described in the CG but requires some additions to existing HTML5 CR sections:


1) Additions to the table "Return values for AudioTrack.kind() and VideoTrack.kind()” in [2] describing how @kind should be set for various track types. [3] shows the new additions for MPEG-2 TS and DASH media resources.


2) Text track equivalent of the table "Return values for AudioTrack.kind() and VideoTrack.kind()” in [2]. [4] shows such an equivalent table for setting @kind for text tracks in MPEG-2 TS and DASH. This could go in the HTML5 CR spec here [5].


3) Guidelines for creating in-band metadata text track cues. Here is the start for MPEG-2 TS [5]. This table could go here [6].


4) Additional definition for DASH describing how to set inBandMetadataTrackDispatchType[7].


Does anyone see a reason not to file bugs to add 1-4 above? These changes are consistent with the direction already taken in [1]. Making these changes wouldn’t preclude further work in the CG and would address use cases that have been identified so far.


Bob


[1] http://www.w3.org/TR/html5/
[2] http://www.w3.org/TR/html5/embedded-content-0.html#audiotracklist-and-videotracklist-objects
[3] https://www.w3.org/community/inbandtracks/wiki/Main_Page#Audio_and_video_kind_table
[4] https://www.w3.org/community/inbandtracks/wiki/Main_Page#Text_kind_table
[5] https://www.w3.org/community/inbandtracks/wiki/Main_Page#Guidelines_for_creating_metadata_text_track_cues
[6] http://www.w3.org/TR/html5/embedded-content-0.html#sourcing-in-band-text-tracks
[7] https://www.w3.org/community/inbandtracks/wiki/Main_Page#Exposing_a_Media_Resource_Specific_TextTrack