Re: [whatwg] Memory management problem of video elements

On Tue, Aug 19, 2014 at 3:54 PM, duanyao <duanyao@ustc.edu> wrote:
> 于 2014年08月19日 20:23, Philip Jägenstedt 写道:
>
>> On Tue, Aug 19, 2014 at 11:56 AM, duanyao <duanyao@ustc.edu> wrote:
>>>
>>> If the media element object keeps track of its current playing url and
>>> current position (this requires little memory), and the media file is
>>> seekable, then
>>> the media is always resumable. UA can drop any other associated memory of
>>> the media element, and users will not notice any difference except a
>>> small
>>> delay
>>> when they resume playing.
>>
>> That small delay is a problem, at least when it comes to audio
>> elements used for sound effects. For video elements, there's the
>> additional problem that getting back to the same state will require
>> decoding video from the previous keyframe, which could take several
>> seconds of CPU time.
>>
>> Of course, anything is better than crashing, but tearing down a media
>> pipeline and recreating it in the exact same state is quite difficult,
>> which is probably why nobody has tried it, AFAIK.
>
> UA can pre-create the media pipeline according to some hints, e.g. the video
> element is becoming visible,
> so that the delay may be minimized.
>
> There is a load() method on media element, can it be extended to instruct
> the UA to recreate
> the media pipeline? Thus script can reduce the delay if it knows the media
> is about to be played.

load() resets all state and starts resource selection anew, so without
a way of detecting when a media element has destroyed its media
pipeline to save memory, calling load() can in the worst case increase
the time until play.

> Audios usually eat much less memory, so UAs may have a different strategy
> for them.
>
> Many native media players can save playing position on exit, and resume the
> playing from that position on the next run.
> Most users are satisfied with such feature. Is recovering to "exact same
> state" important to some web applications?

I don't know what is required for site compat, but ideally destroying
and recreating a pipeline should get you back to the exact same
currentTime and continue playback at the correct video frame and audio
sample. It could be done.

> I'm not familiar with game programing. Are sound effects small audio files
> that are usually
> played as a whole? Then it should be safe to recreate the pipeline.

There's also a trick called audio sprites where you put all sound
effects into a single file with some silence in between and then seek
to the appropriate offset.

Philip

Received on Wednesday, 20 August 2014 07:52:32 UTC