Re: [MSE] Summary of 'Appending URLs' thread

On Jul 24, 2012, at 3:35 PM, Steven Robertson wrote:

> On Tue, Jul 24, 2012 at 9:44 AM, Aaron Colwell <acolwell@google.com> wrote:
>> 1. Prevent media data from being surfaced into the JS layer when no
>> modifications are necessary.
> 
> JavaScript will end up needing to read at least some of the data in
> several common scenarios. For example, YT will be using DASH VOD
> profile for static content, which packs the subsegment-to-byte mapping
> in a single SIDX at the head of the one and only DASH segment (where
> DASH "subsegments" are MSE "media segments").

I think "Prevent" above should be "avoid a requirement for".

I'd like to keep the existing append approach and *add* an approach which allows the media data to flow directly from XHR to SourceBuffer for the cases where the application doesn't want to see it.

It's even possible that we can extend the XHR+Blob stuff so that part of the response can be exposed to the JS and part of it send directly to the SourceBuffer - for example getting a BLOB response, slicing it into two BLOBs, reading one into an ArrayBuffer and passing the other to SourceBuffer.

> We also have prototypes
> which would require the same on a per-segment basis for live streams,
> meaning either an extra request per stream every five seconds to cover
> duplicate ranges, or passing all data through JavaScript.
> 
> This isn't a counterpoint to that goal, of course; using JS is less
> efficient and it's nice to avoid that step when possible.
> 
>> 2. Provide a mechanism for the SourceBuffer to control the append rate so it
>> can avoid memory overflow in mobile devices.
>> 3. Give the SourceBuffer access to media data as it is being received
>> instead of waiting until the whole transfer is complete.
>> 
>> I still believe these goals are important, but I'm not sure they should all
>> be solved with a single solution. I'm starting to believe that goal 2 needs
>> a distinct mechanism especially if we decide to have both URL based &
>> Uint8Array based appending methods. I think goals 1 & 3 could be solved via
>> special interactions with XHR or Blobs(Mark Watson's idea from a side email
>> conversation).
> 
> Wholeheartedly agree that goal 2 is distinct - and important! Here's
> why we (YT) think this:
> 
> We expect that many device implementations will use simple cache
> eviction policies (such as "discard the entire GOP under the playback
> head" or "drop everything outside of [t, t+20)"). We also expect that
> 'appendUrl()' would encourage some devices to use something like
> libcurl on the media stack to control rate, and thus forego any
> resource caching.
> 
> Some features (an Instant Replay button on live streams) and entire
> applications (an interactive nonlinear video presentation which
> accepts and reacts to input events) require very low latency to be
> effective. In the presence of the expected limitations of devices,
> implementing this would require applications to manually implement
> precaching and cache eviction strategies to cover these corner cases.
> 
> Since rebuffer events are the single most important QoS factor, even
> "ordinary" streaming applications will likely pay close attention to
> resource acquisition and caching, and may decide to provide a
> secondary caching implementation in JavaScript to deal with poor
> source buffer cache eviction policies.
> 
> Regardless of the actual strategy used to provide data to the source
> buffer, providing application-level feedback regarding buffer capacity
> and free memory information can help applications and platforms
> cooperate, rather than making authors and implementors fight to
> override each others' desired caching strategies.
> 
>> Here are my questions for the group:
>> 
>> Do people agree with the goals I've outlined? If not, what should be
>> changed?
>> 
>> Should we drop appendUrl() in favor of an alternate approach?
> 
> append(XMLHttpRequest) seems to offer a reasonable path to
> implementation and most if not all of the features authors need. It
> also more strongly suggests that resource requests should go through
> the browser.
> 
> It's likely that YT will use manual fetching on devices for reasons
> described above; as long as a path remains to append bytes from
> JavaScript without too much memory explosion, and there's a mechanism
> for getting feedback to keep JS heap size within platform constraints,
> our needs should be met there.
> 
> Steve
> 
>

Received on Tuesday, 24 July 2012 23:29:55 UTC