Re: HTML5 support for track metadata

On 3/9/14, 11:50 PM, "Silvia Pfeiffer" <silviapfeiffer1@gmail.com> wrote:

>Hi Bob & Clift,
>
>I have actually wondered about the need for exposing in-band track
>metadata here myself
>https://www.w3.org/community/inbandtracks/wiki/Main_Page#Exposing_In-band_
>Track_Metadata
>.
>
>I believe the biggest discussion that was had here wrt this topic is
>to expose the PMT. I agree with Clift and Bob that exposing the PMT in
>a separate track is not necessary, since the list of tracks in the
>media element (.audioTracks, .videoTracks and .textTracks) as well as
>the attributes on the track objects provide sufficient information
>about the PMT.
>
>Thus, we have to discuss whether there is a use case for providing a
>generic means for exposing other in-band metadata text tracks as
>specified here:
>https://www.w3.org/community/inbandtracks/wiki/Main_Page#Guidelines_for_cr
>eating_metadata_text_track_cues
>
>What other tracks does a MPEG-2 TS have that are not audio, video,
>captions/subtitles, or the PMT?

The other track types are: region_rating_table used in parental controls,
SCTE 35 used for ad insertion, EISS as in enhanced TV. I have been told by
Nielsen rating that they want to embed proprietary data in a private user
track to support collecting information on Web TV platforms.

>
>Cheers,
>Silvia.
>
>
>
>
>On Wed, Mar 5, 2014 at 9:11 AM, Bob Lund <B.Lund@cablelabs.com> wrote:
>>
>>
>> From: <Clift>, Graham <Graham.Clift@am.sony.com>
>> Date: Tuesday, March 4, 2014 at 2:17 PM
>> To: Bob Lund <b.lund@cablelabs.com>, "public-inbandtracks@w3.org"
>> <public-inbandtracks@w3.org>
>> Cc: "Ota, Takaaki" <Takaaki.Ota@am.sony.com>, "Wu, Max"
>> <Max.Wu@am.sony.com>, "Nejat, Mike" <Mahyar.Nejat@am.sony.com>, Brant
>> Candelore <brant.candelore@am.sony.com>
>> Subject: RE: HTML5 support for track metadata
>>
>> Hi CG,
>>
>>
>>
>> I believe that option 3 is the best approach because it requires least
>> change to HTML5 specification, covers the use cases and will be the most
>> likely to pass the HTML WG.
>>
>>
>>
>> As to how to handle events (with minimal change to HTML spec), I was
>> thinking the following:
>>
>>
>>
>> When an application sees a change in the PMT there should also be a
>> corresponding change to one or more of the
>> TextTrackList/VideoTrackList/AudioTrackList 's.
>>
>> Seems then it should be sensible to tie the PMT change in
>> inBandMetaDataDispatchType to an onchange event in the TextTrackList
>>object.
>> This would be sufficient for the purpose tracking PMT data and if cues
>>are
>> needed then they could be generated easily by the application thus
>> eliminating the only advantage that option 1 has IMHO.
>>
>>
>> Changes in the PMT caused by adding/removing elementary streams should
>> result in one of onhange, onddrack, onemovetrack events. Are there any
>>use
>> cases where the PMT for an existing track changes?
>>
>>
>>
>> BTW,
>>
>>
>>
>> As well as deciding how to handle the PMT data via the three
>>alternatives I
>> believe more work should be done on Item 3) (creating in-band metadata
>>text
>> tracks  cues). In particular there needs to be more clarification to
>>ensure
>> consistency across implementations. The two areas below is where I see
>>some
>> problems:
>>
>>
>>
>> a)       Handling private section payloads that are split across many TS
>> packets is not well defined.
>>
>>
>> My proposal defines this - a complete private section is returned in the
>> cue.
>>
>> Seems there are three possible approaches.
>>
>>                      i.                              Since the detection
>> method proposed is the 'payload_unit_start_indicator', this would
>>suggest
>> that the transport demux is collecting the individual payloads before
>> presenting them as a cue to the web application. If this is the case
>>then
>> how does the demux decide that the payload is complete? If it waits to
>>see
>> the next 'payload_unit_start_indicator' then the timing may be too late
>>to
>> be relevant.
>>
>>                    ii.                              If, on the other
>>hand,
>> the demux creates a cue for each payload entity then that could impact
>> performance.
>>
>>                   iii.                              Maybe it just waits
>>for
>> a period of time and if no more payloads packets are received then cue
>>what
>> we have. This approach would mean there is an expectation for the
>> application to handle variably fragmented cues.
>>
>>
>>
>> Leaving it up the UA to decide how to implement could be challenging
>>for web
>> application designers to allow for all possiblities.
>>
>> b)      There is no clarity of what the startTime means to the
>>application.
>> The spec says this is with respect to the media resource time which
>> presumably means the PTS from some audio/video PES. But which timing?
>>After
>> the private payload is sent or before private payload is sent? And what
>>is
>> the startTime for payloads split across TS packets with Video PES
>>packets in
>> between, especially when partial payload DataCues are supported as  (as
>>in
>> case iii above.) The option of leaving this up to the implementation is
>> unsatisfactory because of the potential for variations in the result.
>>
>>
>> I agree more guidelines in cue generation would remove ambiguity. I
>>think
>> startTime in the case of MPEG-2 TS metadata should be the current time
>>in
>> the media resource when the private section is received.
>>
>>
>>
>> Regards
>>
>>
>>
>> Graham Clift
>>
>>
>>
>>
>>
>> From: Bob Lund [mailto:B.Lund@CableLabs.com]
>> Sent: Tuesday, March 04, 2014 11:50 AM
>> To: public-inbandtracks@w3.org
>> Subject: HTML5 support for track metadata
>>
>>
>>
>> Hi CG,
>>
>>
>>
>> I think that the existing HTML5 CR spec [1] is very close to supporting
>>the
>> use cases that are being discussed in the CG. I propose submitting
>>several
>> HTML5 bugs to close the gap and I'm interested in your thoughts.
>>
>>
>>
>> An alternative described in the CG wiki is to use kind and
>> inBandMetadataTrackDispatchType attributes to expose track metadata:
>>kind is
>> used for all tracks except metadata text tracks and
>> inBandMetadataTrackDispatchType is used for metadata text tracks. This
>> covers the use cases that have been described in the CG but requires
>>some
>> additions to existing HTML5 CR sections:
>>
>>
>>
>> 1) Additions to the table "Return values for AudioTrack.kind() and
>> VideoTrack.kind()" in [2] describing how @kind should be set for various
>> track types. [3] shows the new additions for MPEG-2 TS and DASH media
>> resources.
>>
>>
>>
>> 2) Text track equivalent of the table "Return values for
>>AudioTrack.kind()
>> and VideoTrack.kind()" in [2]. [4] shows such an equivalent table for
>> setting @kind for text tracks in MPEG-2 TS and DASH. This could go in
>>the
>> HTML5 CR spec here [5].
>>
>>
>>
>> 3) Guidelines for creating in-band metadata text track cues. Here is the
>> start for MPEG-2 TS [5]. This table could go here [6].
>>
>>
>>
>> 4) Additional definition for DASH describing how to set
>> inBandMetadataTrackDispatchType[7].
>>
>>
>>
>> Does anyone see a reason not to file bugs to add 1-4 above? These
>>changes
>> are consistent with the direction already taken in [1]. Making these
>>changes
>> wouldn't preclude further work in the CG and would address use cases
>>that
>> have been identified so far.
>>
>>
>>
>> Bob
>>
>>
>>
>> [1] http://www.w3.org/TR/html5/
>>
>> [2]
>> 
>>http://www.w3.org/TR/html5/embedded-content-0.html#audiotracklist-and-vid
>>eotracklist-objects
>>
>> [3]
>> 
>>https://www.w3.org/community/inbandtracks/wiki/Main_Page#Audio_and_video_
>>kind_table
>>
>> [4] 
>>https://www.w3.org/community/inbandtracks/wiki/Main_Page#Text_kind_table
>>
>> [5]
>> 
>>https://www.w3.org/community/inbandtracks/wiki/Main_Page#Guidelines_for_c
>>reating_metadata_text_track_cues
>>
>> [6]
>> 
>>http://www.w3.org/TR/html5/embedded-content-0.html#sourcing-in-band-text-
>>tracks
>>
>> [7]
>> 
>>https://www.w3.org/community/inbandtracks/wiki/Main_Page#Exposing_a_Media
>>_Resource_Specific_TextTrack
>>
>>

Received on Monday, 10 March 2014 21:22:10 UTC