Re: MediaStream and HTMLMediaElement


On 11/06/15 10:37, Adam Bergkvist wrote:
> Hi
> I've done some investigating. My initial thought is that this will
> require some more work.
> /Adam
> =================
> # srcObject attribute description in the Media Capture and Streams spec
> "This attribute overrides both the src attribute and any <source>
> elements. Specifically, if srcObject is specified, the User Agent MUST
> use it as the source of media, even if the src attribute is also set or
> <source> children are present."
> Comment: Covered.
> "If the value of srcObject is replaced or set to null the User Agent
> MUST re-run the media element load algorithm"
> Comment: Covered
> # Modifications/restrictions mentioned in the Media Capture and Streams
> spec:
> "Whenever the User Agent runs the media element load algorithm, if
> srcObject is specified, the User Agent must immediately go to the
> resource fetch phase of the algorithm."
> Comment: The HTML5.1 document simply says that the "media element load
> algorithm" should be run when the srcObject is set. This will be a
> change in behavior since step 3 (of the "media element load algorithm")
> will fire an "abort" event (networkState is always NETWORK_IDLE). Step 4
> will queue a task that fires an additional event ("emptied") (also
> because of the network state).

I'm not sure the above is a problem. The resource is local, so network 
state should be ignored. We should probably take this to the list.

> One way to avoid these events would be to
> force the NETWORK_EMPTY state ahead of running the "media element load
> algorithm". The state would be set back to NETWORK_IDLE later in the
> "resource fetch algorithm".
> "Whenever the User Agent runs the media element load algorithm, and
> reaches the resource fetch phase of this algorithm, if it determines
> that the media resource in question is a MediaStream, it MUST
> immediately abort the resource selection algorithm, and set the
> media.readyState to HAVE_NOTHING if the MediaStream is inactive, or
> HAVE_ENOUGH_DATA if it is active."
> Comment: The HTML5.1 document doesn't prevent the "resource selection
> algorithm" from running for srcObject=MediaStream. Instead, the
> algorithm has a special mode (mode=object) if the element has an
> "assigned media provider object". The "resource selection algorithm"
> also updates the networkState to values not supported by the attribute
> table in the MediaCap spec (NETWORK_NO_SOURCE and NETWORK_LOADING).

We should update the table. I think we should just remove the lines 
related to MediaElement readyState and networkState since that is now 
spelled out in 5.1.

> "Let the media resource, represented by the MediaStream object, indicate
> to the media element load algorithm that all audio tracks and all live
> video tracks (represented by a MediaStreamTrack with the readyState
> attribute set to live) should be enabled. This allows the media element
> load algorithm to set AudioTrack.enabled, VideoTrack.selected and
> VideoTrackList.selectedIndex accordingly."
> Comment: Covered ("if the user agent has information that would
> facilitate the selection of specific audio tracks to improve the user's
> experience")
> "If a MediaStreamTrack is removed from a MediaStream, played by a media
> element, the corresponding AudioTrack or VideoTrack MUST be removed as
> well."
> Comment: Not covered (at least directly).

I think this is covered in

> "The User Agent MUST NOT buffer data from a MediaStream. When playing,
> the User Agent MUST always play the current data from the stream."
> Comment: Not covered (at least directly).

I agree.

> "When the MediaStream state moves from the active to the inactive state,
> the User Agent MUST raise an ended event on the media element and set
> its ended attribute to true. Note that once ended equals true the media
> element will not play media even if new Tracks are added to the
> MediaStream (causing it to return to the active state) unless autoplay
> is true or the JavaScript restarts the element, e.g., by calling play()."
> Comment: Not covered (at least directly).

Yes, we need to add that - if we want that behavior.

There are some other things I've detected in addition:

* HTML5.1 srcObject seem to be thinking "blob", things like "when the 
entire resource has been loaded" makes no sense
* Not clear what should happen if the MediaStream is inactive when loaded.

I'll update the PR, but won't be ready for the call today.


Received on Thursday, 18 June 2015 13:00:38 UTC