RE: TV Control API and getUserMedia

Hi Harald,

Some more comments below:

> Den 12. jan. 2017 11:07, skrev Chris Needham:
> > Thanks for your reply, and apologies for the delay in following up.
>
> No problem - glad that the token was not lost!
>
> >>> 2. Channel constraint at the track level?
> >>> --
> >>> Similarly, it does not seem natural to apply the constraint to change the channel at the track level. The channel "constraint" rather seems to apply at the MediaStream level, as it is going to affect all tracks. Are we trying to push the constraint model too far?
> >>
> >> It seems somehow strange to use a stream control to change the channel..
> >> I'd rather want to model this as getting a new stream from the new
> >> channel. Streams are not very heavy-weight objects; creating a new one
> >> is probably more sensible than re-sourcing an old one.
> >
> > This is something we've discussed in the TV Control WG, where we considered that creating a MediaStream involves allocating the underlying hardware resources for tuning to and decoding the media. It seemed to us that reusing the stream allowed for a simpler implementation, and provided a reasonable API for users.
>
> Streams and tracks are control surfaces; the underlying implementation
> will do what it has to do. In a WebRTC implementation, incoming decoder
> resources are allocated as a result of SetLocalDescription(), and the
> creation of tracks and streams is a side effect of that. Whether the
> resources belong to the track or to the peerconnection is rather
> academic; there's no testable effect of either model.
>
> I think it's reasonable to focus on providing a reasonable API for the
> users.

I agree, on both points.

> >>> The same comment applies to the "stop" method, which again is going to affect all tracks of the MediaStream in our case.
> >>
> >> I don't think it's necessary to reintroduce stream.stop(); it is a very
> >> short loop to stop all tracks in a stream.
> >
> > We don't really have a use case for stopping streams individually, so we may want a stop method in our MediaStream-derived interface, or an interface that acts as a parent to the MediaStream.
>
> By "parent", do you mean "upstream object", the "source" of the
> MediaStream's tracks?
> If you have a separate tuner object, stopping the tuner should ripple
> down through all the streams' tracks naturally.

I was meaning the TVTuner interface in Francois' "Alternative proposal" example, which has a reference to the stream. The interfaces look like this (simplified):

interface TVSource : EventTarget {
  Promuse<sequence<TVChannel>> getChannels();
  Promise<TVTuner> tuneToChannel(TVChannel channel);
};

interface TVTuner : EventTarget {
  Promise<void> stop();
  readonly attribute TVChannel?     currentChannel;
  readonly attribute TVMediaStream? stream;
};

As you say, the TVTuner::stop() method would stop all the tracks on its stream.

Note that we're likely to move away from the "tuner" name to avoid implying a particular kind of implementation.

> >>> 3. TV specific semantics?
> >>> --
> >>> While it works, is also seems counter-intuitive to "apply a constraint" to switch from one channel to another. Developers would rather expect something like a "tuneToChannel" method instead.
> >>
> >> If we don't retarget streams, this problem goes away.
> >
> > Yes, true.
> >
> >>> 4. Extending or wrapping MediaStream?
> >>> --
> >>> The TV Control API extends MediaStream to add buffering. This seems to preserve the spirit of MediaStreams and there has already been exchanges about that in the past.
> >>>
> >>> We may need to introduce other attributes at that level, such as an "isRecordable" property to tell whether the channel may be recorded. This is also where the "stop" method mentioned above would fit. Such extensions seem of different nature. We're wondering whether it could be preferable to introduce an extra layer that wraps the MediaStream and exposes TV specific properties.
> >>
> >> What are the buffering properties you want? Do you want to make the
> >> result seekable (infininte-size buffering), or do you want merely to
> >> ensure that buffers are deep enough that glitches don't happen?
> >
> > We want the stream to be seekable, to enable pause/resume for live TV and timeshifted playback, so essentially we're removing a lot of the constraints that MediaStream applies to the HTMLMediaElement. Here's a link to our previous discussion:
> >
> > https://lists.w3.org/Archives/Public/public-media-capture/2015Jun/0011.html
> >
> > and there are some specifics in my original email:
> >
> > https://lists.w3.org/Archives/Public/public-tvapi/2015Jun/0000.html
>
> Ouch, that's a lot of seekability.... is your model that the "tuner"
> produces a real-time stream and that the stream is preserved locally,
> but only back to the time that the "tuner" was created (the DVR model),
> or is it a model where the "tuner" is able to do either storage or
> calling upstream for data it doesn't have locally (the "on-demand" model)?

It's more the DVR model at this stage, so local buffering of the stream. The ability to switch to on-demand introduces a lot more complexity.

> >>> Alternative proposal
> >>> -----
> >>> All in all, an alternative proposal that separates the APIs, moves operations back to the source level, and creates a wrapping tuner class around MediaStream could lead to the following code:
> >>>
> >>> var source = null;
> >>> var secondChannel = null;
> >>> var tvTuner = null;
> >>>
> >>> navigator.tv.getSources()
> >>>     .then(function (sources) {
> >>>       source = sources[0];
> >>>       return source.getChannels();
> >>>     })
> >>>     .then(function (channels) {
> >>>       var channel = channels[0];
> >>>       secondChannel = channels[1];
> >>>       return source.tuneToChannel(channel);
> >>>     })
> >>>     .then(function (tuner)) {
> >>>       tvTuner = tuner;
> >>>
> >>>       // Render the stream
> >>>       document.getElementById('video').srcObject = tvTuner.stream;
> >>>
> >>>       // Switch the source to another channel, re-using the same tuner
> >>>       return source.tuneToChannel(secondChannel, tuner);
> >>>     })
> >>>     .then(function () {
> >>>       // Stop stream and release resource
> >>>       tvTuner.stop();
> >>>     });
> >>
> >> // Stop the channel
> >> document.getElementById('video').srcObject = null;
> >> // Since there are no more references to "stream", it will be garbage
> >> collected.
> >
> > Our preference is to have a more explicit mechanism for releasing resources, rather than only relying on GC, although we do realise we can't rely on users doing the right thing.
>
> Users will do what works for them, not what we want them to :-)
>
> document.getElementById('video').srcObject.videoTracks()[0].stop will
> work too of course.

Yes, this would also work.

Thanks for your input!

Chris


-----------------------------
http://www.bbc.co.uk
This e-mail (and any attachments) is confidential and
may contain personal views which are not the views of the BBC unless specifically stated.
If you have received it in
error, please delete it from your system.
Do not use, copy or disclose the
information in any way nor act in reliance on it and notify the sender
immediately.
Please note that the BBC monitors e-mails
sent or received.
Further communication will signify your consent to
this.
-----------------------------

Received on Friday, 13 January 2017 15:17:47 UTC