[media-and-entertainment] Frame accurate seeking of HTML5 MediaElement

tidoust has just created a new issue for https://github.com/w3c/media-and-entertainment:

== Frame accurate seeking of HTML5 MediaElement ==
I've heard a couple of companies point out that one of the problems that makes it hard (at least harder than it could be) to do **post-production of videos** in Web browsers is that there is no easy way to process media elements on a frame by frame basis, whereas that is the usual default in Non-Linear Editors (NLE).

The `currentTime` property takes a time, not a frame number or an SMPTE timecode. Conversion from/to times to/from frame numbers is doable but supposes one knows the framerate of the video, which is not exposed to Web applications (a generic NLE would thus not know about it). Plus that framerate may actually vary over time.

Also, internal rounding of time values may mean that one seeks to the end of the previous frame instead of the beginning of a specific video frame.

Digging around, I've found a number of discussions and issues around the topic, most notably:
1. An long thread from 2011 on Frame accuracy / SMPTE, which led to improvements in the precision of seeks in browser implementations:
https://lists.w3.org/Archives/Public/public-whatwg-archive/2011Jan/0120.html
2. A list of use cases from 2012 for seeking to specific frames. Not sure if these use cases remain relevant today:
https://www.w3.org/Bugs/Public/show_bug.cgi?id=22678
3. A question from 2013 on whether there was interest to expose "versions of currentTime, fastSeek(), duration, and the TimeRanges accessors, in frames, for video data":
https://www.w3.org/Bugs/Public/show_bug.cgi?id=8278#c3
4. A proposal from 2016 to add a rational time value for seek() to solve rounding issues (still open as of June 2018):
https://github.com/whatwg/html/issues/609

There have probably been other discussions around the topic.

I'm raising this issue to collect practical use cases and requirements for the feature, and gauge interest from media companies to see a solution emerge. It would be good to precisely identify what does not work today, what minimal updates to media elements could solve the issue, and what these updates would imply from an implementation perspective.

Please view or discuss this issue at https://github.com/w3c/media-and-entertainment/issues/4 using your GitHub account

Received on Monday, 11 June 2018 13:58:35 UTC