[Bug 27249] New: Correct the “All or Nothing” Approach Currently Implemented For HTMLMediaElement / MSE

https://www.w3.org/Bugs/Public/show_bug.cgi?id=27249

            Bug ID: 27249
           Summary: Correct the “All or Nothing” Approach Currently
                    Implemented For HTMLMediaElement / MSE
           Product: HTML WG
           Version: unspecified
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: HTML5 spec
          Assignee: dave.null@w3.org
          Reporter: karan@karanlyons.com
        QA Contact: public-html-bugzilla@w3.org
                CC: mike@w3.org, public-html-admin@w3.org,
                    public-html-wg-issue-tracking@w3.org

(Apologies for the length and possibly poor nomenclature of the below.)

Currently if you use an HTMLMediaElement:
    1) The browser automatically determines the file types / codecs of the
sources and chooses the “best” source to play.
    2) The browser automatically parses the metadata of the chosen source in
order to enable abilities like seeking, get the duration, etc.
    3) The browser implements some version of source buffering, but offers you
know control over it. Not even at the very least starting and stopping
buffering, and at most, even doing things like setting how much to buffer
ahead, etc.
    4) The browser exposes *none* of this known metadata (file type, codec,
file structure for seeking, bitrate, etc.) with the exception of the height,
width, and duration.

If you use MSE:
    1) The browser expects you to know ahead of time if the data you’re putting
into a SourceBuffer is decodable by the browser.
    2) The browser expects you to know the metadata associated with the source
in order to set the duration and implement seeking *yourself*.
    3) The browser implements no buffering.

Those are your only two options. If you want to be able to do something as
simple as starting or stopping buffering, you need to know all the associated
metadata for your file (file type, codec, stco atom in the case of QT
containers, height, width, duration, perhaps more). Your options are either to
use a prepared manifest of all that required data, which is only possible if
you know what content you’re to play ahead of time, or determine all that
information dynamically, for which as a developer you’re now forced to use far
less performant javascript, *despite the fact that the browser already
implements fast compiled functions to do all this work*.


This “all or nothing” approach permeates both specs. As one example,
HTMLMediaElement will tell you if it can play a source with canPlayType(),
which requires a string of the format “filetype; codecs”. However it provides
no way to *get* the format of a file, despite the fact that it can quite
clearly determine this information (how else does it find the “best” source to
play?).

Another example is that readyState change events, though fired based on buffer
data, contain no snapshot of the TimeRanges buffered, despite the fact that
those TimeRanges can and will change in time between the event being triggered
and an event listener running and iterating through HTMLMediaElement.buffered.

MSE seems to have less of these problems, though an obvious one is the
requirement for a type string with addSourceBuffer(), despite—again—no way to
*get* the type of a buffer/file.


This honestly seems fairly difficult to fix since it’s a bug in the thinking
behind the implementations and not just some aspect of the implementations
themselves. Moreover, it requires more to be changed in the HTMLMediaElement
spec than MSE’s. But I do think it’s very important that these shortcomings are
addressed.

At the very least allowing greater control over the buffer (whereby “greater” I
just mean the ability to start and stop the buffer, and *perhaps* control its
length) would allow most developers to stick with HTMLMediaElement as opposed
to being forced to use the more complex MSE.

At the most there needs to be a far greater degree of transparency in all the
work the browser does regarding media. Metadata, file type, codec information,
etc. should *all* be exposed to the developer, along with functions for
determining them from an arbitrary SourceBuffer/ArrayBuffer/Blob/etc. The aim
should be to ensure that the handling of media can continue to be dynamic (e.g.
no manifests) and performant (e.g. no javascript based container parsing to get
metadata) regardless of where along the continuum of HTMLMediaElement <-> MSE
the developer chooses to reside.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

Received on Wednesday, 5 November 2014 14:39:37 UTC