W3C home > Mailing lists > Public > public-media-capture@w3.org > August 2013

Re: Constructors for MediaStreamTrack subtypes - remove?

From: Harald Alvestrand <harald@alvestrand.no>
Date: Mon, 12 Aug 2013 09:30:25 +0200
Message-ID: <52088F11.2070305@alvestrand.no>
To: Stefan Håkansson LK <stefan.lk.hakansson@ericsson.com>
CC: "public-media-capture@w3.org" <public-media-capture@w3.org>
On 08/12/2013 08:34 AM, Stefan Håkansson LK wrote:
> On 2013-08-11 17:13, Harald Alvestrand wrote:
>> On 08/08/2013 03:35 PM, Stefan Håkansson LK wrote:
>>> On 2013-08-05 10:46, Harald Alvestrand wrote:
>>>> Hi,
>>>> at the moment, the spec contains constructors for VideoMediaStreamTrack
>>>> and AudioMediaStreamTrack - each taking a TrackConstraints set as parameter.
>>>> These were part of an idea to revisit the way one acquires media
>>>> (different from getUserMedia, which returns tracks) - but the chairs are
>>>> unable to tell whether any of the brief proposals floated in Lyon and
>>>> Boston have gathered consensus as "THE proposal", or indeed whether we
>>>> have consensus to go ahead with this API addition.
>>>> In the absence of such consensus, I suggest that we remove the
>>>> constructors from the spec at this time, so that it's clear that
>>>> implementations don't have to support these constructors that don't
>>>> produce useful objects at the moment.
>>>> If we converge on a specific proposal that makes these constructors
>>>> useful, I think it is easy to add them back at that time.
>>>> People who have a different opinion should reply to this message....
>>> Personally I think the possibility to create AudioMediaStreamTrack's and
>>> VideoMediaStreamTrack's (and out of them MediaStream's) is quite
>>> valuable in situations where the destination is a remote peer.
>>> It allows the application to set up the connection and media codecs etc.
>>> before (or in parallel to) asking for consent to use microphone and
>>> camera. This can enhance the user experience since it reduces the risk
>>> of audio clipping (as everything is already set up when the user accepts
>>> the use of camera and microphone). It also allows the app to not bother
>>> the user by asking for consent if for some reason the connection can't
>>> be established (or if the codecs supported do not match - we still don't
>>> have an MTI video codec).
>> That's where I get confused .... what if the properties of the stream
>> depend on the device chosen at GetUserMedia time? Such as Cullen's
>> standard example of a VP8-hardware camera and an H.264-hardware camera,
>> where you can choose one or the other? (Of course, if you can tell the
>> capabilities of the camera without opening it, we're back into
>> fingerprinting land, for which user prompts are supposed to be our
>> shield.... but "new" happens before the user prompt.)
>> So if one does this:
>> videoTrack = new VideoMediaStreamTrack(..)
>> videoStream = new MediaStream(videoTrack)
>> peerConnection.addStream(videoStream)
>> offer = peerConnection.createOffer(...)
>> will the offer contain all the codec configurations available to the
>> browser, only those codec configurations that the browser knows it can
>> support across all devices, some "browser default", or something else?
> To me, it would be only those codec configurations the browser knows it
> can support across all devices. That is all that can be done at this stage.

Two corner cases that come to mind:

1) The browser has two non-intersecting possible sets (the H.264 vs VP8 
camera case).
Should the offer contain no video?

2) The browser's "support across all devices" is outside the constraints 
given for the streams , but all reasonable results of an attachUserMedia 
result in a media stream that satisfies the constraints (for example, 
min size may be 640x480, transcoding may be supported only up to 
240x160, while both the VP8 camera and the H.264 camera are HD capable).

Should CreateOffer fail? Or create a "no video" offer?
> But for this to be possible, all devices must be able to deliver media
> in a more raw format than h.264/VP8 otherwise the browser would have to
> transcode. If this is not possible, then I agree this path is questionable.
I'm not sure I can parse those words into something I can translate into 


The format actually exchanged on the network once data flows will be in 
a codec format.
That codec format may be VP8, or something else - it will definitely not 
be in a "raw" format, so negotiating a "raw" format won't just be 
useless, it'll be harmful.
So that can't be what you intended to say - could you explain more?

>> And if one later does (my preferred naming - I see no reason to reuse
>> the name "getUserMedia):
>> navigator.attachUserMedia(videoStream, ....)
>> will a negotationNeeded event be fired if the capabilities of the source
>> are different from the capabilities previously negotiated? If it's more
>> capable than announced, less capable than announced, or only with
>> different preferences?
> I think it should fire if the attached device adds capabilities.
>> It's this kind of complexity that leads me to think that we don't have a
>> complete proposal for what I'm calling "attachUserMedia" yet.
>>> I also think it would be quite a minor addition to spec up how a
>>> MediaStream that contains only constructed audio and video tracks can be
>>> an argument to getUserMedia (but I don't know how big the implementation
>>> effort would be though).
>>>>            Harald
Received on Monday, 12 August 2013 07:30:16 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 16:26:18 UTC