- From: Silvia Pfeiffer <silviapfeiffer1@gmail.com>
- Date: Sat, 19 Feb 2011 08:41:53 +1100
- To: Philip Jägenstedt <philipj@opera.com>
- Cc: "public-html@w3.org" <public-html@w3.org>
On 19/02/2011, at 3:31 AM, Philip Jägenstedt <philipj@opera.com> wrote: > On Fri, 18 Feb 2011 17:08:28 +0100, Mark Watson <watsonm@netflix.com> wrote: > >> >> On Feb 18, 2011, at 2:08 AM, Philip Jägenstedt wrote: >> >>> On Thu, 17 Feb 2011 18:43:49 +0100, Mark Watson <watsonm@netflix.com> >>> wrote: >>> >>>> >>>> On Feb 17, 2011, at 7:17 AM, Philip Jägenstedt wrote: >>>> >>>>> On Wed, 16 Feb 2011 18:47:22 +0100, Mark Watson <watsonm@netflix.com> >>>>> wrote: >>>>> >>>>>> >>>>>> On Feb 16, 2011, at 12:02 AM, Philip Jägenstedt wrote: >>>>>> >>>>>>> On Wed, 16 Feb 2011 03:31:47 +0100, Silvia Pfeiffer >>>>>>> <silviapfeiffer1@gmail.com> wrote: >>>>>>> >>>>>>>> On Wed, Feb 16, 2011 at 12:08 PM, Jonas Sicking <jonas@sicking.cc> >>>>>>>> wrote: >>>>>>>>> On Tue, Feb 15, 2011 at 4:19 PM, Silvia Pfeiffer >>>>>>>>> <silviapfeiffer1@gmail.com> wrote: >>>>>>>>>> On Wed, Feb 16, 2011 at 5:36 AM, Mark Watson <watsonm@netflix.com> >>>>>>>>>> wrote: >>>>>>>>>>> Hi Philip, >>>>>>>>>>> >>>>>>>>>>> Just a quick note that the "alternative" vs "additional" >>>>>>>>>>> distinction >>>>>>>>>>> is not always completely clear. Video with different camera angles >>>>>>>>>>> (gimmiky or not) could be considered as an alternative, or could >>>>>>>>>>> be >>>>>>>>>>> rendered as picture-in-picture, or multiple thumbnail videos could >>>>>>>>>>> show beside the main video (some sports sites already do this kind >>>>>>>>>>> of >>>>>>>>>>> thing). >>>>>>> >>>>>>> Sure, but all of those modes should be achieved by the author making >>>>>>> it >>>>>>> happen with CSS. At the risk of making a strawman argument, I honestly >>>>>>> can't see browsers allowing the user to change the rendering of the >>>>>>> page >>>>>>> to achieve PiP or something like that when the author hasn't provided >>>>>>> for >>>>>>> it, messing with the layout like that seems both weird and unlikely to >>>>>>> be >>>>>>> useful. Of course we can have User JavaScript and User CSS to do that >>>>>>> kind >>>>>>> of thing, though. >>>>>> >>>>>> I was assuming that the "author" of the content - who labels the tracks >>>>>> - might not be the same as the "author" of the webpage that is >>>>>> rendering >>>>>> the content. So the first author should not assume that (say) multiple >>>>>> views are alternatives, because some webpages might be able to view >>>>>> them >>>>>> both as PIP. >>>>> >>>>> Since the tracks are labeled using the attribute of the <track> >>>>> attribute, >>>>> it will be the page author that has to do the work to support some >>>>> specific video display, be that PiP, overlay or something else. >>>> >>>> That would be the case for track objects created as a result of <track> >>>> elements, but what about in-band tracks ? The page author does the work >>>> for PIP etc., of course, but the media author should not assume that >>>> such capabilities are or are not available on the pages where their >>>> media might be used: they should just label the tracks and let the page >>>> to whatever it is capable of. >>> >>> I don't think we should spend much time making extra in-band video tracks >>> work more than barely, if at all, since the extra bandwidth needed to have >>> multiple in-band video tracks makes it quite unlikely the feature would be >>> used to any greater extent. >> >> A track declared within an adaptive streaming manifest (e.g. a DASH manifest or take-your-pick of various proprietary adaptive streaming solutions) would be an in-band track but would only be fetched when actually needed. > > Good point. > >>> If they should work at all, my position is that the only thing you should >>> be able to do with in-band video tracks is switch between them, in other >>> words what I've called alternative tracks. Either having some kind of >>> layout information in the file itself or having HTML markup to target >>> individual tracks of the same resource seems like unjustified complexity >>> and spec/implementation effort not very well spent. >> >> I think people do imagine that adaptive streaming manifests would declare all the tracks needed for a presentation - including sign language tracks that are additional rather than alternative. Such manifests have to be useful in environments other than HTML and so need to included everything. I don't think we should ask people to re-author them in HTML for use in HTML environments. > > I quite disagree, designing something to work both in browsers and non-browsers means that we can't make good use of whatever existing capabilities browsers already have. In this case, I think we should rely on CSS and only CSS to achieve the desired rendering of multitrack video. Any default rendering we could provide is unlikely to fit well enough in with the overall design of the page that people will want to use it. > > Some samples Silvia collected in <http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Media_Rendering> demonstrate quite clearly IMO the variety of styles we can expect to see. > >> I guess what I am saying is that Option (1) in the wiki write-up should be supported in order to provide support for adaptive streaming. The questions are: >> (1) whether this should be the only way to declare such additional/alternative tracks or whether an HTML markup way is also required (and I think that it is) > > I don't see how this approach could give us the flexibility in styling that is necessary. How do you envisage getting a visual end result similar to <http://www.w3.org/WAI/PF/HTML/wiki/File:Transhud.jpg> (note the fancy borders around the overlayed video) using a manifest approach? The exercise isn't really finished yet. What I am trying to figure out is whether there are common patterns. Of course we need to also allow for the author to change and adapt the styling for their site. But I believe we also need a default display mechanism just like the controls. That should not be fancy. I think it is early to draw conclusions, but we seem to see two fundamentally different layouts: pip (picture-in-picture) and side-by-side. The default should be without any fancy borders, just thrown either on top of the main video or tiled. Either should have a single control. That's all I can tell from preliminary looks. > >> (2) what should that markup be >> (3) how to define the API for discovering and manipulating these tracks in a way that is common for in-band (from a file or from an adaptive streaming manifest) and explicitly marked up tracks. > > From a markup and API perspective a multitrack manifest file would be treated the same as a multitrack WebM file, right? Yes, absolutely! Silvia. > > -- > Philip Jägenstedt > Core Developer > Opera Software >
Received on Friday, 18 February 2011 21:42:44 UTC