Re: State transitions for media elements

On Thu, 18 Sep 2008, Dave Singer wrote:
>
> So, here is a brief revision, obviously derived from the existing 
> document but, we hope, avoiding these issues and supporting more (we 
> hope, all) protocols:
> 
> Network state:  documents whether the network is being used (e.g. for a 
> network activity indicator)
> 
> Empty:  initial state, or state when there is a failure that'll need 
> some action to escape from
> 
> Idle:  the URI is known, but the UA has no need to use the network right 
> now (e.g. download resource for which 'enough' is cached, streaming 
> resource which is not active...)
> 
> Loading: the network is being used right now (you can show an activity 
> indicator)
> 
> Loaded:  for a loadable resource, we've both loaded it all and don't 
> intend to unload it (you could disconnect and walk away)
> 
> There is an event, Stalled, which is fired once during Loading if data 
> doesn't seem to be arriving after a reasonable timeout (as now).
>
> 
> Media state:  documents what you can do with the media.  (Each state is 
> a superset of the one preceding).
> 
> Empty: initial state
> 
> Metadata_loaded:  enough data has been loaded that a well-defined set of 
> questions can now be answered as well as they ever could be (e.g. 
> duration, width/height, codecs used, and so on).
> 
> Can_display (or Can_display_at_current_time; currently called 
> can_display_current_frame):  the UA has done all it can or intends to do 
> for the media resource to be displayable at the current_time.  For a 
> downloadable resource, this means that the current video frame (if 
> applicable) can be painted, at least one sample of audio (if applicable) 
> played, and so on.  For a streaming resource, it may mean very little 
> more than that if you are waiting for something before you displayed the 
> media element, stop waiting: it won't get any more displayable.
> 
> Can_play:  if playback were requested, the UA expects it would be able 
> to actually start within a reasonable period and play a reasonable 
> amount (before a stall, for example).  For a downloadable protocol, that 
> means that at least some data ahead of current_time is available; for a 
> streaming protocol, that if playback was requested, playback would start 
> 'soon'.
> 
> Can_play_through:  if playback was requested, the UA is reasonably 
> confident that it could play to the end without a playback stall.  
> (This state might never get entered if the network bandwidth is 
> insufficient and the resource cannot be cached, either because of cache 
> limitations or because it's a streaming service)

I also added a state for "all the metadata is known and video has been 
shown at least once so there is no need to show placeholder images like 
the poster frame".

The states thus are:

   NETWORK_EMPTY
   NETWORK_IDLE
   NETWORK_LOADING
   NETWORK_DONE

...and:

   HAVE_NOTHING
   HAVE_METADATA
   HAVE_SOME_DATA
   HAVE_CURRENT_DATA
   HAVE_FUTURE_DATA
   HAVE_ENOUGH_DATA

I'm not good at naming constants so if anyone has a truly better idea, 
please let me know. Ideas should be self-consistent, intuitive, have a 
common leading prefix, be relatively short, have correct grammar, and not 
be cute. If you're not sure, don't suggest it. I don't want to wade 
through three dozen sets of names tomorrow. :-P


> Play_request state:  documents what has been asked of the media.  We 
> need state+events for this because UAs can display a play/pause 
> controller that the scripts cannot 'see'.
> 
> Empty:  initial state
> 
> Pause_requested:  the UA has been asked to pause playback
> 
> Play_requested:  the UA has been asked to play
> 
> (This could probably be a single boolean if we don't need the empty 
> initial state).

How is this different from the "paused" boolean attribute?


> Actual playing is reflected by the is_playing property and the 
> Rate_changed event.
> 
> Rate_changed gets dispatched if either of
> a) is_playing changes value (between true and false)
> or
> b) is_playing is true and the current playback rate changes
> 
> Specifically:
> a) for a streaming protocol, after a play_request, the network connection is
> opened, data is requested, some amount of de-jitter buffer accumulated, and
> then is_playing changes to true and a Rate_changed event happens
> b) for a download or streaming protocol, if the buffer runs dry while playing,
> is_playing changes to false and the Rate_changed event is dispatched.

You could implement these on top of the existing events, but1 I guess we 
could add these events too. Do people really want them?


On Mon, 22 Sep 2008, Philip Jägenstedt wrote:
> 
> In particular I've noticed that if the file is available locally (in 
> cache or file://) one will know that it is fully loaded before actually 
> reading any data. But since network states must go through 
> LOADED_METADATA and LOADED_FIRST_FRAME (which require actually reading 
> data) reporting that state has to wait until later. It wouldn't be a big 
> issue implementation-wise, but is a symptom of the mixing of separate 
> issues into a single state.

As defined, the networkState will never be set to NETWORK_LOADED before 
the metadata and first frame have been obtained explicitly. Is that ok?


> I'm thinking that the EMPTY state isn't needed at all, the lowest state 
> could be IDLE. Currently the EMPTY state is used in several algorithms 
> to detect a "fresh" media element, but as far as I can see an empty flag 
> would suffice, and it would not need to be exposed via the API at all.

It seems like it is useful to expose what the browser thinks the state is. 
If people think we should drop NETWORK_EMPTY and just have NETWORK_IDLE, 
though, I'm certainly open to changing the spec to that. What do people want?

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'