[whatwg] <video> feedback

On Thu, 7 Jun 2012, Kit Grose wrote:
> On 06/06/2012, at 7:44 AM, Ian Hickson wrote:
> > On Fri, 13 Jan 2012, Kit Grose wrote:
> >> 
> >> I'd argue that while we did receive in WebM "a common codec" it does 
> >> not enjoy the sort of universal adoption required to be able to 
> >> mandate its support in the spec, so on that logic, I think 
> >> establishing a declarative fallback mechanism is probably required to 
> >> prevent a situation where you cannot write a robust HTML5 page with 
> >> video and without resorting to JS.
> > 
> > I don't think it's time to give up yet, but maybe I'm overly 
> > optimistic...
> 
> Is there any reason why it wouldn't be prudent to render the content of 
> the <video> element as HTML if the video cannot be played, given that 
> fallback content in the video element is already supported for legacy 
> browsers in the spec:
>
> > Content may be provided inside the video element. User agents should 
> > not show this content to the user; it is intended for older Web 
> > browsers which do not support video, so that legacy video plugins can 
> > be tried, or to show text to the users of these older browsers 
> > informing them of how to access the video contents.
> 
> How are legacy UAs without <video> support appreciably different from a 
> UA with restrictive or limited <video> support? Surely the author's 
> intent in either case is delivering the content in a different way or 
> delivering appropriate alternate content.

There are several problems here. One is that designing <video> to fall 
back to rendering its contents in certain cases leads to a very flickery 
experience -- e.g. on a slow network:

   <video>
    <!-- here the browser doesn't know of a <source>, so show fallback? -->
    <source ...>
    <!-- ok now we know of a <source>, so show the video, even though we 
         haven't yet downloaded it? -->
    <!-- but maybe now we know the <source> is not supported, so show the 
         fallback again? -->
    <source ...>
    <!-- ah, now we have a video we know we support, so show it -->
   </video>
   <!-- oops, turns out the one we support is broken after all, show 
        fallback? -->

It just doesn't work very well.


> Even if we eventually get a common codec (which I—perhaps overly 
> pessimistically—feel is unlikely), authors are already using the <video> 
> element without supplying whatever that eventual codec will be, and 
> users are suffering without reasonable fallbacks.

Right now two codecs support everyone, and one of those is what you'd have 
to use for the fallback anyway, so when would you need to show fallback if 
the browser supports <video> at all?


> As it stands, it's almost better (and certainly easier) for authors to 
> default to Flash video and use the existing, declarative fallback 
> behaviour of the <object> element to use a <video> element instead. That 
> can't be in the best interest of the open web.

Why not? So long as they provide a non-proprietary non-encumbered 
alternative, there doesn't see to be much difference what the default is.


On Mon, 11 Jun 2012, Ralph Giles wrote:
>
> Recently, we've been considering adding a 'tags' or 'metadata' attribute 
> to HTML media elements in Firefox, to allow webcontent access to 
> metadata from the playing media resource. In particular we're interested 
> in tag data like creator, title, date, and so on.
> 
> My recollection is that this has been discussed a number of times in the 
> past, but there was never suffient motivation to support the interface. 
> Our particular motivation here is webapps that present a media file 
> library. While it's certainly possible to parse the tag data out 
> directly with javascript, it's more convenient if the HTML media element 
> does so, and the underlying platform decoder libraries usually provide 
> this data already.

My recommendation would be to develop a specification for this (or use the 
one(s) already available for this purpose), and in that specification 
define how it is added to HTMLMediaElement, much as you suggest:

> partial interface HTMLMediaElement {
>   [...]
> };

(I don't have the bandwidth to define how to extract this kind of thing 
from each video format. Even trying to define what little the spec already 
says has required hours of reading obscure specifications and that's 
without even testing to see if those specs match reality.)


On Fri, 15 Jun 2012, Charles Pritchard wrote:
> 
> It's a bit of extra work and won't be "compatible" with the img setsrc 
> technique. I think this is just a rehash of having Image.toBlob() and 
> video.frameToBlob() made available.

(I do not know what this refers to. The e-mail in question was top-posted 
with untrimmed context below. If you would like a detailed reply, I 
encourage you to follow the quoting style suggested in the FAQ:

   http://wiki.whatwg.org/wiki/FAQ#Should_I_top-post_or_reply_inline.3F

Thanks.)


On Mon, 18 Jun 2012, Simon Pieters wrote:
> On Fri, 15 Jun 2012 18:01:09 +0200, Ian Hickson <ian@hixie.ch> wrote:
> > > 
> > > When preload=none, step 2 of 
> > > http://www.whatwg.org/specs/web-apps/current-work/multipage/the-video-element.html#concept-media-load-resource 
> > > should not be optional.
> > > 
> > > The effective (internal) preload state should be defined.
> > > 
> > > It should also be defined that with preload=metadata, readyState 
> > > should never go beyond HAVE_CURRENT_DATA, even for a data: URL or 
> > > otherwise fully cached resource.
> >
> > Making it non-conforming for a user agent to aggressively cache resources,
> > especially if the user has asked for it, is a non-starter.
> 
> Aggressively caching does not necessarily need to be exposed to scripts.

This doesn't make any sense. Making the API just flat out lie about what 
is cached would just mean that everyone has to remove the attribute to 
figure out what's actually going on. It would turn preload=metadata from a 
hint saying to avoid network traffic, to a flag that cripples the API. And 
it wouldn't, as far as I can tell, make any difference to compatibility. 
There will be browsers that don't preload with preload=auto, for instance, 
so it's not like authors can end up depending on 'loadeddata' or 'canplay' 
firing to start playback with no other way to start it. Plus, even if a 
page did end up waiting for a caching even that never comes to start 
playback, the user can just tell the UA to start playback instead of 
waiting for the page to do it, and then all the events will fire.


> > > > As far as I can tell, the spec is as detailed as it can be here 
> > > > given the range of possible implementation strategies that we need 
> > > > to allow.
> > > > 
> > > > Could you give a concrete example of what you are concerned about?
> > > 
> > > <video src=x preload=none onsuspend="makeSiteWork()"></video>
> > 
> > Then we should stop firing "suspend" in the preload=none case,
> 
> No.
>
> > or fire it in every case if preload=none, even if the UA immediately 
> > unsuspends.
> 
> Maybe.

Neither of these seem like a good idea to me, They're both lying. 
'suspend' should fire when the network traffic suspends (or when it 
finishes, or when it doesn't start). Having it fire at times where we're 
still getting data, is silly.


> > But I'm not convinced anyone is going to hook into onsuspend in this 
> > way. There'd be no point as far as I can tell, and it's more 
> > complicated than the alternative (just run the code straight away 
> > without waiting).
> 
> You're not convinced that people on the Web do things in ways more 
> complicated and fragile than necessary, just because it's possible?

Not for cases such as this, no. And if they do, it'll still work, you just 
have to wait for the entire video to download.



On Wed, 20 Jun 2012, Silvia Pfeiffer wrote:
> 
> I found that browsers don't provide a default tabfocus on media elements 
> nor do they provide keyboard interactivity. I had to put explicit 
> @tabindex attributes onto the media elements to allow them to at least 
> receive focus. This is particularly irritating in a screenreader.

Please file such bugs directly with the browser vendors.


> As the video is specified right now, it is not a tabfocusable element 
> [1] and only interactive [2] when it has controls.
> [1] http://www.whatwg.org/specs/web-apps/current-work/multipage/editing.html#focusable
> [2] http://www.whatwg.org/specs/web-apps/current-work/multipage/elements.html#interactive-content-0

The category "interactive content" doesn't mean it's interactive, per se, 
it just means it's non-conforming for the element to be placed in other 
interactive content.

The UA controls of a media element themselves should always be focusable 
and interactive. They're separate from the <video> and <audio> elements.


On Wed, 20 Jun 2012, Simon Pieters wrote:
>
> http://www.whatwg.org/specs/web-apps/current-work/multipage/editing.html#focus
> 
> I guess it should add "media elements that are exposing a user 
> interface" and finally have a "may" for just about anything, since this 
> is UI and browsers should be allowed to make things focusable if they 
> want. It is unusual for the spec to have UI requirements at all.

I will track this as part of:

   https://www.w3.org/Bugs/Public/show_bug.cgi?id=17554
   https://www.w3.org/Bugs/Public/show_bug.cgi?id=17555


On Mon, 9 Jul 2012, adam k wrote:
> 
> i have a 25fps video, h264, with a burned in timecode.  it seems to be 
> off by 1 frame when i compare the burned in timecode to the calculated 
> timecode.  i'm using rob coenen's test app at 
> http://www.massive-interactive.nl/html5_video/smpte_test_universal.html 
> to load my own video.
> 
> what's the process here to report issues?  please let me know whatever 
> formal or informal steps are required and i'll gladly follow them.

Depends on the browser. Which browser?


> i'm aware that crooked framerates (i.e. the notorious 29.97) were not 
> supported when frame accuracy was implemented.  in my tests, 29.97DF 
> timecodes were incorrect by 1 to 3 frames at any given point.
> 
> will there ever be support for crooked framerate accuracy?  i would be 
> more than happy to contribute whatever i can to help test it and make it 
> possible.  can someone comment on this?

This is a Quality of Implementation issue, basically. I believe there's 
nothing inherently in the API that would make accuracy to such timecodes 
impossible.


On Tue, 17 Jul 2012, Mark Callow wrote:
>
> The spec. for CanvasRenderingContext2D.drawImage says draw nothing when 
> a video element's readyState is HAVE_NOTHING or HAVE_METADATA. I was 
> wondering why this was chosen vs. drawing the poster. A search in the 
> list archive didn't turn up any discussion or explanation.

It's error-handling behaviour. The reason you'd draw a <video> is to do 
frame capture, but if you're in HAVE_NOTHING or HAVE_METADATA then you 
don't have the current frame.

If you want to draw the poster, you can do that easily enough already.


On Wed, 18 Jul 2012, Mark Callow wrote:
> 
> Speaking of which, how does one specify the "poster frame" for an 
> animated image? There is no poster attribute on an img element. Is this 
> something specified in the file format? A quick search didn't reveal any 
> such for animated GIF or APNG files.

It's hypothetically a feature of the animated image format. In practice I 
think only MNG supports that (not sure though).


On Wed, 25 Jul 2012, Charles Pritchard wrote:
> > 
> > Why? onmetadataloaded provides a sufficiently stable stuation: either 
> > the poster img or video frame is then loaded (if @preload is not 
> > "none") or it's black (if @poster is not set and @preload is "none").
> 
> Seems like a good place to spec this. +1.
> 
> I'm not up to speed on the video spec, but if this behavior for poster 
> is not in there, consider proposing it to the editor(s).

I'm not really clear on what the use case is here.


On Thu, 26 Jul 2012, Cyril Concolato wrote:
> 
> [SVG and WebVTT]

I couldn't work out what you were proposing, sorry. Could you elaborate on 
your use case? Maybe it's already possible.


On Fri, 24 Aug 2012, Dumez, Christophe wrote:
> 
> The MediaController interface in the latest video element
> specification [1] has a number of events, for example:
>            attribute EventHandler onemptied;
>            attribute EventHandler onloadedmetadata;
>            attribute EventHandler onloadeddata;
> 
> However, the MediaController interface does not inherit EventTarget
> [2]. Shouldn't it be added?

Done.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'