Re: [mediacapture-main] Clarification needed on ended HTMLMediaElement with MediaStream becoming active (#519)

Maybe, because mediacapture-main no longer says "set its ended attribute", but leaves this to the media element spec instead. That means the media element spec also dictates when to unset the ended attribute.

It says this about having ended playback (which is when `ended` returns true):
```
A media element is said to have ended playback when:
-    The element's readyState attribute is HAVE_METADATA or greater, and

-    Either:
        The current playback position is the end of the media resource, and
        The direction of playback is forwards, and
        The media element does not have a loop attribute specified. 

     Or:
        The current playback position is the earliest possible position, and
        The direction of playback is backwards. 
```

If we've ended playback and the stream becomes active/audible (video/audio), that might mean that the `end of the media resource` is again +inf?

Looking at "media resource" [the spec](https://html.spec.whatwg.org/multipage/media.html#media-elements:media-resource) says:
> A media resource can have multiple audio and video tracks. For the purposes of a media element, the video data of the media resource is only that of the currently selected track (if any) as given by the element's videoTracks attribute when the event loop last reached step 1, and the audio data of the media resource is the result of mixing all the currently enabled tracks (if any) given by the element's audioTracks attribute when the event loop last reached step 1.

So the end of the media resource can only be extended if when the MediaStream becomes active, those new AudioTrack and VideoTrack instances become enabled or selected, respectively.

There are AudioTracks and VideoTracks becoming enabled/selected without the application interacting, in the [resource selection algorithm](https://html.spec.whatwg.org/multipage/media.html#concept-media-load-resource) with mode `local`, specifically in the [media processing steps list](https://html.spec.whatwg.org/multipage/media.html#media-data-processing-steps-list) that are run whenever new data becomes available:
> Whenever new data for the current media resource becomes available, queue a task to run the first appropriate steps from the media data processing steps list below.

Note that the entire resource selection algorithm is aborted when the entire media resource is aborted:
> When the current media resource is permanently exhausted (e.g. all the bytes of a Blob have been processed), if there were no decoding errors, then the user agent must move on to the final step below. This might never happen, e.g. if the current media resource is a MediaStream.

The final step:
> Final step: If the user agent ever reaches this step (which can only happen if the entire resource gets loaded and kept available): abort the overall resource selection algorithm.

I guess the remaining question is whether a MediaStream becoming inactive/inaudible means that it is exhausted? I guess the `This might never happen, e.g. if the current media resource is a MediaStream.` is the answer, and means that a MediaStream never becomes permanently exhausted. Then this means that an ended media element where its (MediaStream) current media resource becomes active, will stop being ended, advance the readyState, select tracks anew, etc.

Can someone sign off on this making sense? If so I think we can close this.

Though it would probably be useful with a note in mediacapture-main noting this, because it took some digging to get through.

-- 
GitHub Notification of comment by Pehrsons
Please view or discuss this issue at https://github.com/w3c/mediacapture-main/issues/519#issuecomment-553845455 using your GitHub account

Received on Thursday, 14 November 2019 11:23:53 UTC