RE: [media] change proposals for issue-152

Hi Mark. This is an early snapshot of a work in progress, I'll make it easier to follow and more complete over the next couple of days; but it is a Wiki, so feel free to add your comments in directly.

From: Mark Watson [mailto:watsonm@netflix.com]
Sent: 31 March 2011 17:15
To: Sean Hayes
Cc: Silvia Pfeiffer; HTML Accessibility Task Force
Subject: Re: [media] change proposals for issue-152

Sean,

If by CP3 you mean http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Change_Proposal (it would be good to make the CP numbers hyperlink), it does not support discovery of in-band tracks and neither does CP4.

Discovery of in-band tracks means a page author who does not know a priori which tracks will be present in the resource (e.g. because they have just been given a URL for a resource to render) needs to find the list of tracks that are in there. So there needs to be some "HTMLMediaElement.tracks" attribute containing the list of tracks. You may then create markup for the ones you want to render, or it may be possible to enable rendering of a track directly from the objects in HTMLMediaElement.tracks (as in Ian's proposal)

This is supported already for in-band text tracks.

I'm happy to update the page everyone agrees this is a correct understanding.

...Mark


On Mar 31, 2011, at 8:04 AM, Sean Hayes wrote:


I've made a start at http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Issue_Tracking

-----Original Message-----
From: public-html-a11y-request@w3.org<mailto:public-html-a11y-request@w3.org> [mailto:public-html-a11y-request@w3.org] On Behalf Of Sean Hayes
Sent: 31 March 2011 09:09
To: Silvia Pfeiffer
Cc: HTML Accessibility Task Force
Subject: RE: [media] change proposals for issue-152

I think it would definitely be worth starting a wiki page (or set of pages) that is essentially a "defect list" of issues related to the multi-track proposals, of which track styling is one. Then when we mark each item as consensus we can issue a bug at that point with detailed change.



-----Original Message-----
From: Silvia Pfeiffer [mailto:silviapfeiffer1@gmail.com]
Sent: 31 March 2011 08:50
To: Sean Hayes
Cc: HTML Accessibility Task Force
Subject: Re: [media] change proposals for issue-152

Sean, all,

In today's media a11y meeting we discussed Sean's issue that tracks
cannot be moved by CSS into a different screen region. As the solution
we agreed that a ::track pseudo-selector would be an acceptable
solution and better than introducing a cues element, because in this
way we can also provide styling to in-band text tracks.

Do we need a wiki page to put details of this proposed change together
or can we just register a bug?

Cheers,
Silvia.


On Wed, Mar 30, 2011 at 9:00 AM, Sean Hayes <Sean.Hayes@microsoft.com<mailto:Sean.Hayes@microsoft.com>> wrote:

" ...the default rendering, which should make immediate sense to the user and cover the 80% use case. Text should by default be rendered on top of the main video's viewport "
Well I'm not sure we know what the 80% use case really is on the web. If we are transplanting TV content this might be true, but in a web case actually having the text appear in a div immediately after the video in the block layout direction might actually make more sense.

"Additional video tracks should by default be rendered next to the main video"
Again I'm not sure this is true. Until we get reliable  use data, I think that complete replacement of the main video is actually just as likely, if not more so.

I fully expect that in 99% of cases CSS will be used anyway, so to me the default arrangement argument is an extremely weak one, especially that in the absence of CSS, user agents can pretty much do what they want to present content that "represents" something.

 "> Can you describe in the "embedded in viewport" model how I spread captions across two videos placed side by side.
This can only be done using CSS. This is true for all existing proposals and therefore not an objection to any of them."
No. If captions are rendered into a viewport CSS alone cannot do this. Only if captions are rendered into a containing box is this possible with CSS.

"As I said: I can appreciate that we might need the networkState for text tracks. That is a separate issue from the one we are discussing here"
I can appreciate your desire to divide and conquer, and in general I'd agree but in this case however this feature needs to be properly integrated, otherwise it's going to look like a baling-wire and twine lash up. I think that we need to consider a lot of external issues to get this right.

" the @timeline attribute makes the relationship definition easily prone to faulty markup "
I agree, and actually I'm rather coming to prefer Ian's controller and mediagroup model. Although I think the way he represents tracks doesn't work.

" If you want to change the way in which the <audio> element works in general, that is another discussion to have "
Well again as I said we need to consider a lot of seemingly external issues, however I want to be clear that I actually *don't* want to change audio, or indeed *video* (neither IE nor FF implement audio content in the video element today). I want to just have another element that can be in audio's mediagroup that does have a visual layout.

"When there is no video element on the page, do the cues continue to display on a timeline?"
They would display at time=0, although if not in a mediagroup, by default in  a zero width/height element.

"Do they have default controls?"
It's not a requirement that they be able to play independently, so no; but they could.

"What if there are captions that are only activated 5min in - does the user have to
wait for 5min to display them?"
Obviously yes, unless they scrub or use some other trick play.

" they don't want to introduce a new layout engine for the video viewport, when the CSS layout engine already provides what is needed for multiple videos "
Precisely. Why not use that same layout engine for the text frame too.

"I still maintain that the most typical layouts of multitrack video are: tiled, picture-in-picture, and as a scrollable list. I would personally like to see these encoded in CSS and thus make multiple videos layed out by just choosing one CSS value."

I don't think a fixed menu is likely to be very useful, I do agree that CSS is a clumsy tool to use for layout, however IMO that has been a general problem with CSS for years. I believe there are some moves to fix this in CSS3.