RE: [media] change proposals for issue-152

I think there are some real benefits for the handling of in-band tracks in Sean's proposal, in particular with respect to metadata text tracks. Supporting markup for in-band text track usage, especially metadata, makes it much more explicit how these types of tracks will be handled:
- It is explicit which cue types are to be handled
- It explicitly supports multiple metadata tracks
- It allows metadata tracks to be typed by the user agent via the @type attribute on the contained <source>
- It supports type independent use of metadata tracks. For example, the following shows how ad insertion might work with different formats of insertion point metadata:

<cues id="adInsertion" timeline="v1" kind="metadata">
      <source src="video.webm" type="application/x-scte35" />
      <source src="video.webm" type="application/x-madeupType" />
      <source src="video.mp4" type="application/x-scte35" />
      <source src="video.mp4" type="application/x-madeupType" />

I'd suggest one slight (at least I think it's slight) change. Rather than the following shown in Sean's proposal:

<cues id="captions" timeline="v1" kind="captions" srclang="en" label="Captions">
      <source src="video.webm#track=captions" type="text/vtt" />
      <source src="video.mp4#track=captions" type="application/ttml+xml" />

How about:

<cues id="captions" timeline="v1" kind="captions" srclang="en" label="Captions">
      <source src="video.webm" type="text/vtt" />
      <source src="video.mp4" type="application/ttml+xml" />

i.e., drop the #track selector in <source> since it appears redundant with the @kind, @srclang and @label attributes in <cues>? This alternative also allows any permutation of these attributes to be used in selecting in-band tracks.

Another comment is that usage of text tracks with non-video media seems to be realistic. The example given was captions for audio. Another example might be audio advertisement insertion. While this could be accomplished with the "tracks" as proposed, their use with anything other than video is not described.


Bob Lund

> -----Original Message-----
> From: public-html-a11y-request@w3.org [mailto:public-html-a11y-
> request@w3.org] On Behalf Of Silvia Pfeiffer
> Sent: Sunday, March 27, 2011 12:04 PM
> To: HTML Accessibility Task Force
> Subject: Re: [media] change proposals for issue-152
> 
> Dear Media Subgroup,
> 
> As per decision by the chairs of the HTML WG, we have until Friday to
> continue discussions and come to an agreement on a common change
> proposal: http://lists.w3.org/Archives/Public/public-
> html/2011Mar/0614.html
> .
> 
> I would recommend to approach this by going one by one through the
> differences of at least the three proposals that have originated from
> this group and either come up with a unified new one, or modify one to
> match all our requirements. The discussions between Ian's proposal and
> ours is one that I would like to see happening afterwards on the main
> list.
> 
> Here are the three proposals under discussion:
> 
> (1) http://lists.w3.org/Archives/Public/public-html/2011Feb/0363.html
> (Frank's original proposal)
> (2) http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Change_Proposal
> (write-up from the F2F)
> (3)
> http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Change_Proposal_2
> (Sean's proposal)
> 
> Since (2) and (3) are very similar and emerged because we didn't have
> time to finish that discussion at the F2F, I would like to address this
> first.
> 
> So, proposal 3 suggests that rather than having the <track> element as a
> dependent element underneath the <video> element, we should make the
> <track> functionality available as a prime element to Web pages under
> the name <cues>.
> 
> While I can follow the reasoning for this proposal to stem from the idea
> of having a unified approach to all possible types of content that can
> appear in a multi-track resource, I believe this is actually taking the
> purity reasons a step too far. I have two main issues with this
> approach:
> 
> 1. Captions and more generally time-aligned text for videos, are always
> slaved to video content. They do not make sense to exist as an entity by
> themselves. The times in the cues of a cue file only become real when
> attached to a video. A <cues> element on a Web page without a video
> would never display anything because the cues would never become active.
> 
> 2. There should be a default rendering of cues and it should be attached
> to the video viewport. Rendering in other positions on the page should
> be regarded as custom rendering and the 20% case - easy to achieve, but
> not the common use case. Proposal 3 puts this on its head, making
> rendering in other positions on the page the default and rendering on
> top of the video a problem of the Web page author.
> 
> For these reasons mainly, I would refrain from trying to re-define how
> the <track> element works.
> 
> 
> Best Regards,
> Silvia.
> 
> 
> 
> 
> 
> On Mon, Mar 21, 2011 at 10:03 PM, Sean Hayes <Sean.Hayes@microsoft.com>
> wrote:
> > No. we didn't agree on pulling it out. We did however have a unified
> approach on them all being tracks. I disagree with the move of treating
> them differently, either they are all tracks or they are all top level
> elements.
> >
> > I will submit my proposal separately then, I'll also rename <track> to
> <cues>. I agree that we are not done on this issue.
> >
> > -----Original Message-----
> > From: Silvia Pfeiffer [mailto:silviapfeiffer1@gmail.com]
> > Sent: 22 March 2011 04:53
> > To: Sean Hayes
> > Cc: HTML Accessibility Task Force
> > Subject: Re: [media] change proposals for issue-152
> >
> > Hi Sean,
> >
> > I don't think we had all agreed on pulling <track> out into a
> > stand-alone tag. I certainly disagree with that move.
> >
> > So, given the lack of time, we will certainly have to submit both
> > proposals now and leave it to the chairs to decide how to move on. I
> > don't think we've finished discussions on this topic though and
> > believe that given time we could still converge. But we'll now have to
> > wait and see what the chairs will allow to happen.
> >
> > Cheers,
> > Silvia.
> >
> >
> > On Tue, Mar 22, 2011 at 1:56 PM, Sean Hayes <Sean.Hayes@microsoft.com>
> wrote:
> >> [4] Is an interesting CP, it does however fail to capture one
> >> fundamental premise of the proposal that we had on the table when I
> >> left at 2pm. Namely that the handling of text tracks should be as far
> >> as possible the same as the handling of a media track. This would
> >> imply under this formulation that <track> (meaning a text track)
> >> should be promoted to the same level as audio and video, and with the
> >> same model, so no it doesn't have my agreement that it is an
> >> equivalent to [1]
> >>
> >> I am editing a copy of the page with those changes.
> >>
> >> -----Original Message-----
> >> From: public-html-a11y-request@w3.org
> >> [mailto:public-html-a11y-request@w3.org] On Behalf Of Silvia Pfeiffer
> >> Sent: 22 March 2011 00:47
> >> To: HTML Accessibility Task Force
> >> Subject: [media] change proposals for issue-152
> >>
> >> Hi media subgroup,
> >>
> >> A quick status update on where we stand wrt change proposals for
> issue-152.
> >>
> >> Over the weekend we have worked on a change proposal, which we called
> >> the "San Diego Thought Experiment", see [1].
> >>
> >> It is based on the ideal markup representation of a multitrack media
> >> resource, where external resources are represented in markup as
> >> tracks of a main resource. As we worked through the markup changes,
> >> the IDL/JavaScript API changes, the CSS changes and the rendering
> >> approaches, we realized that the implementation of this ideal
> >> representation would replicate far too many existing codepaths and at
> >> the same time introduce complex layout requirements that would be
> >> unrealistic to expect to be implemented.
> >>
> >> We came to the conclusion that the approach of keeping separate
> >> <video> and <audio> elements around and synchronizing them to a
> >> "master" element (i.e. the "main" resource) would be far easier to
> >> implement and just as powerful. So, we picked up the existing option
> >> 6 [2] and continued designing from there to see if that would be
> >> achievable in a simpler manner.
> >>
> >> Note that in the meantime Ian has also submitted a change proposal
> >> which is highly interesting [3].
> >>
> >> Since Eric and I promised to put the change proposal that was the
> >> outcome of the F2F discussions together, we've worked on this today
> >> and it's now in a readable state [4]. We are going to submit that
> >> proposal as an additional change proposal to the main working group
> >> late today. I don't know if it has general task force approval and
> >> it's too late to do a poll for this. But we certainly want to submit
> >> a change proposal within the deadline. Others should be free to
> >> submit their own if they disagree.
> >>
> >> Cheers,
> >> Silvia.
> >>
> >>
> >> [1]
> >> http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Media_API#.2810.2
> >> 9_HTML_Accessibility_Task_Force_proposal_-_.22The_San_Diego_Thought_E
> >> xperiment.22 [2]
> >> http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Media_API#.286.29
> >> _Synchronize_separate_media_elements_through_attributes
> >> [3] http://lists.w3.org/Archives/Public/public-html/2011Mar/0436.html
> >> [4]
> >> http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Change_Proposal
> >>
> >>
> >>
> >
> >

Received on Tuesday, 29 March 2011 20:01:40 UTC