Re: MediaStream and HTMLMediaElement

In-line.

On 11/06/15 10:37, Adam Bergkvist wrote:
> Hi
>
> I've done some investigating. My initial thought is that this will
> require some more work.
>
> /Adam
>
> =================
>
> # srcObject attribute description in the Media Capture and Streams spec
>
> "This attribute overrides both the src attribute and any <source>
> elements. Specifically, if srcObject is specified, the User Agent MUST
> use it as the source of media, even if the src attribute is also set or
> <source> children are present."
>
> Comment: Covered.
>
> "If the value of srcObject is replaced or set to null the User Agent
> MUST re-run the media element load algorithm"
>
> Comment: Covered
>
> # Modifications/restrictions mentioned in the Media Capture and Streams
> spec:
>
> "Whenever the User Agent runs the media element load algorithm, if
> srcObject is specified, the User Agent must immediately go to the
> resource fetch phase of the algorithm."
>
> Comment: The HTML5.1 document simply says that the "media element load
> algorithm" should be run when the srcObject is set. This will be a
> change in behavior since step 3 (of the "media element load algorithm")
> will fire an "abort" event (networkState is always NETWORK_IDLE). Step 4
> will queue a task that fires an additional event ("emptied") (also
> because of the network state).

I'm not sure the above is a problem. The resource is local, so network 
state should be ignored. We should probably take this to the list.

> One way to avoid these events would be to
> force the NETWORK_EMPTY state ahead of running the "media element load
> algorithm". The state would be set back to NETWORK_IDLE later in the
> "resource fetch algorithm".
>
> "Whenever the User Agent runs the media element load algorithm, and
> reaches the resource fetch phase of this algorithm, if it determines
> that the media resource in question is a MediaStream, it MUST
> immediately abort the resource selection algorithm, and set the
> media.readyState to HAVE_NOTHING if the MediaStream is inactive, or
> HAVE_ENOUGH_DATA if it is active."
>
> Comment: The HTML5.1 document doesn't prevent the "resource selection
> algorithm" from running for srcObject=MediaStream. Instead, the
> algorithm has a special mode (mode=object) if the element has an
> "assigned media provider object". The "resource selection algorithm"
> also updates the networkState to values not supported by the attribute
> table in the MediaCap spec (NETWORK_NO_SOURCE and NETWORK_LOADING).

We should update the table. I think we should just remove the lines 
related to MediaElement readyState and networkState since that is now 
spelled out in 5.1.

>
> "Let the media resource, represented by the MediaStream object, indicate
> to the media element load algorithm that all audio tracks and all live
> video tracks (represented by a MediaStreamTrack with the readyState
> attribute set to live) should be enabled. This allows the media element
> load algorithm to set AudioTrack.enabled, VideoTrack.selected and
> VideoTrackList.selectedIndex accordingly."
>
> Comment: Covered ("if the user agent has information that would
> facilitate the selection of specific audio tracks to improve the user's
> experience")
>
> "If a MediaStreamTrack is removed from a MediaStream, played by a media
> element, the corresponding AudioTrack or VideoTrack MUST be removed as
> well."
>
> Comment: Not covered (at least directly).

I think this is covered in 
http://www.w3.org/TR/html51/semantics.html#dom-media-audiotracks

>
> "The User Agent MUST NOT buffer data from a MediaStream. When playing,
> the User Agent MUST always play the current data from the stream."
>
> Comment: Not covered (at least directly).

I agree.

>
> "When the MediaStream state moves from the active to the inactive state,
> the User Agent MUST raise an ended event on the media element and set
> its ended attribute to true. Note that once ended equals true the media
> element will not play media even if new Tracks are added to the
> MediaStream (causing it to return to the active state) unless autoplay
> is true or the JavaScript restarts the element, e.g., by calling play()."
>
> Comment: Not covered (at least directly).

Yes, we need to add that - if we want that behavior.

There are some other things I've detected in addition:

* HTML5.1 srcObject seem to be thinking "blob", things like "when the 
entire resource has been loaded" makes no sense
* Not clear what should happen if the MediaStream is inactive when loaded.

I'll update the PR, but won't be ready for the call today.

>
>

Received on Thursday, 18 June 2015 13:00:38 UTC