Re: TV Control API and getUserMedia

In my opinion, the first seems to work just fine: regard a TV stream as
just another camera and tune it with constraints. I'd suggest doing the
same with radio streams.

However, IMO you should get input from an implementer who would try to make
it work and find the limitations.

Best Regards,
Silvia.

On 17 Dec. 2016 1:03 am, "Francois Daoust" <fd@w3.org> wrote:

Dear Media Capture task force,

The TV Control Working Group is re-modeling the TV Control API
specification around sources. That new model is well aligned with the model
used in Media Capture and Streams. In fact, it would seem possible to
re-use Media Capture and Streams interfaces as-is. However, some details do
not seem entirely right when we do that, and we would like to get your
feedback on the following questions.


Context
-----
The TV Control API specification defines an API for sourcing audio and
video media, such as TV and radio from broadcast, IPTV, or other sources,
and allows presentation of the media using the <video> and <audio> HTML
elements:

  http://w3c.github.io/tvcontrol-api/

The API typically produces MediaStreams, with methods to switch from one
TV/radio channel to the other, and various classes to retrieve associated
channel and program metadata.

The current API is designed around the notion of tuners but the group
agreed to a re-design centered on the notion of sources. To keep things
simple, a TV source is something attached to a broadcast signal (which
could come through a cable, terrestrial antennas, satellites, etc.) that
can be tuned to a specific channel to produce a MediaStream, composed of a
set of MediaStreamTracks for that channel. Users may switch the source to
another channel at any time.

Looking at it from a getUserMedia perspective, it seems possible to
consider TV sources as input devices, and channels as a constraint that
could be applied to the main video track. This would lead to the following
code:

var source = null;

navigator.mediaDevices.enumerateDevices()
    .then(function (devicesInfo) {
      source = devicesInfo.find(function (deviceInfo) {
        // Or use "getCapabilities" to detect support for the "tvchannel"
        // constraint if kind cannot be extended?
        return deviceInfo.kind === 'tvsource';
      });
      return source.getChannels();
    })
    .then(function (channels) {})
      var channel = channels[0];
      secondChannel = channels[1];
      navigator.mediaDevices.getUserMedia({
        video: {
          deviceId: source.deviceId,
          tvchannel: channel
        }
      })
    })
    .then(function (stream) {
      tvStream = stream;

      // Render the stream
      document.getElementById('video').srcObject = tvStream;

      // Switch to another channel
      return tvStream.getVideoTracks()[0].applyConstraints({
        tvchannel: secondChannel
      });
    })
    .then(function () {
      // Stop stream
      tvStream.getVideoTracks()[0].stop();
    });


This triggers a few questions though.

Questions
-----

1. Attaching to "enumerateDevices"?
--
We need some way to enumerate the TV/radio sources but it feels a bit
strange to mix TV/radio sources with camera/microphone sources. Use cases
that want to get media from a camera are roughly disjoint from use cases
that want to tune to a particular TV channel. Shouldn't APIs rather be
separated?

If not, what would be the proper way to distinguish TV sources? Should we
introduce a new "tvsource" kind of source for instance?


2. Channel constraint at the track level?
--
Similarly, it does not seem natural to apply the constraint to change the
channel at the track level. The channel "constraint" rather seems to apply
at the MediaStream level, as it is going to affect all tracks. Are we
trying to push the constraint model too far?

The same comment applies to the "stop" method, which again is going to
affect all tracks of the MediaStream in our case.


3. TV specific semantics?
--
While it works, is also seems counter-intuitive to "apply a constraint" to
switch from one channel to another. Developers would rather expect
something like a "tuneToChannel" method instead.


4. Extending or wrapping MediaStream?
--
The TV Control API extends MediaStream to add buffering. This seems to
preserve the spirit of MediaStreams and there has already been exchanges
about that in the past.

We may need to introduce other attributes at that level, such as an
"isRecordable" property to tell whether the channel may be recorded. This
is also where the "stop" method mentioned above would fit. Such extensions
seem of different nature. We're wondering whether it could be preferable to
introduce an extra layer that wraps the MediaStream and exposes TV specific
properties.


Alternative proposal
-----
All in all, an alternative proposal that separates the APIs, moves
operations back to the source level, and creates a wrapping tuner class
around MediaStream could lead to the following code:

var source = null;
var secondChannel = null;
var tvTuner = null;

navigator.tv.getSources()
    .then(function (sources) {
      source = sources[0];
      return source.getChannels();
    })
    .then(function (channels) {
      var channel = channels[0];
      secondChannel = channels[1];
      return source.tuneToChannel(channel);
    })
    .then(function (tuner)) {
      tvTuner = tuner;

      // Render the stream
      document.getElementById('video').srcObject = tvTuner.stream;

      // Switch the source to another channel, re-using the same tuner
      return source.tuneToChannel(secondChannel, tuner);
    })
    .then(function () {
      // Stop stream and release resource
      tvTuner.stop();
    });

That alternative proposal re-uses MediaStreams, but is less aligned with
getUserMedia. What do you think?

Feedback through email is fine. Feel free to send feedback on GitHub if
preferred. This is being tracked by the following issue on our side:
 https://github.com/w3c/tvcontrol-api/issues/4

Note that we would be happy to schedule a call as needed to clarify
questions and thoughts.

Thanks,
Francois.