Re: [Exposed] attribute on our APIs

So until now, we can summarize:

- There are people saying that they can imagine use cases where making
MediaStreamTrack objects available in a Worker is useful

- There are no people saying that making getUserMedia() available in a
Web Worker makes any sort of sense

- There is one browser maker (Mozilla) that has said that implementing
access to the MediaStreamTrack object in a Worker is a major engineering
effort - so it's not cheap.

Not sure what conclusions to draw from that....


Den 19. mai 2015 07:35, skrev Stefan Håkansson LK:
> On 19/05/15 01:03, Mathieu Hofman wrote:
>> Anne and Stefan seem to have had an exchange about "Make MediaStream
>> usable with postMessage" in October 2013. Did anything come out of
>> that?
> 
> What basically happened as far as I remember was that Anne and I had 
> some text that we (at least I) thought was OK. It was quite 
> straightforward.
> 
> However (and remember we've been trying to get to last call for a long 
> time :) ), in the interest of not delaying the LC I decided to stop 
> pushing this feature as there was little interest from others at the 
> time. This seems to have changed now.
> 
> It could be worth noting that we at the time had a brief discussion on 
> whether the MediaStream(Track)'s should be transferred or structurally 
> cloned, and ended up on cloning (just as you seem to have) since the 
> MediaStream(Track) API is just a control surface that is pretty 
> lightweight, the heavy media processing machinery executes in the 
> platform and is not cloned itself.


> 
> Stefan
> 
>>
>> ________________________________________ From: Mathieu Hofman Sent:
>> Monday, May 18, 2015 3:36 PM To: Harald Alvestrand;
>> public-media-capture@w3.org Cc: Iņaki Baz Castillo Subject: RE:
>> [Exposed] attribute on our APIs
>>
>> I'm not extremely familiar with how this would be specified, but here
>> is what I think is needed: - Define in the Media Capture and Stream
>> specification how MediaStream and MediaStreamTrack objects should
>> implement HTML5 structured cloning
>> (http://www.w3.org/TR/html5/infrastructure.html#structured-clone).
>> Since these objects already support cloning, it might simply be a
>> matter of mentioning that a structured clone is the result of
>> implicitly calling the .clone() method? - Make sure that the
>> MediaStream and MediaStreamTrack APIs are accessible from non-window
>> contexts. - Have the ImageCapture APIs accessible from non-window
>> Worker contexts as well.
>>
>> For the use case I mentioned previously, the work would be in both
>> APIs.
>>
>> The structured cloning change would mean that a source can be
>> accessed through track objects in different, potentially unrelated
>> contexts. In particular, a page could obtain access to a media source
>> and send its stream object to an iframe running in another domain.
>> From a user perspective, I do not think this is a concern since the
>> user already trusted the app to basically do anything it wants with
>> the captured source. However, I have no idea what the implications
>> are on browser implementations.
>>
>> This does however open the door to very interesting approaches, such
>> as: - Media source processed by background worker (my previously
>> described use case) - Media source acquired by installed browser
>> app/extension then "transferred" to regular page for sharing/display.
>> This could be an implementation path for some privileged "output"
>> streams such as tab capture. - Media displayed/rendered in other
>> windows of an app. E.g. allowing an app to pop out a video stream
>> outside of its main UI. (Inaki's use case)
>>
>> I'd be happy to help writing a specific proposal, I'm just not sure
>> how to do it right.
>>
>> Mathieu ________________________________________ From: Harald
>> Alvestrand [harald@alvestrand.no] Sent: Saturday, May 16, 2015 1:16
>> AM To: Mathieu Hofman; public-media-capture@w3.org Subject: Re:
>> [Exposed] attribute on our APIs
>>
>> Den 15. mai 2015 22:50, skrev Mathieu Hofman:
>>> Answering to myself here, I completely forgot about the
>>> ImageCapture API. I think it would be useful for this API to be
>>> available in a worker and be able to transfer a MediaStream(Track)
>>> object to a worker through postMessage.
>>>
>>
>> Thanks for the input!
>>
>> Would you be willing and able to provide a proposal for exactly what
>> we need to change in the spec in order for this to be possible?
>>
>> And - should this be done as part of the ImageCapture API work, or
>> as part of the Media Capture and Streams specification?
>>
>>> Mathieu ________________________________________ From: Mathieu
>>> Hofman Sent: Friday, May 15, 2015 1:41 PM To: Harald Alvestrand;
>>> public-media-capture@w3.org Subject: RE: [Exposed] attribute on our
>>> APIs
>>>
>>> Sorry for missing this earlier.
>>>
>>> One use case I could see is processing of the media video track in
>>> a worker through a canvas element. Canvas can be manipulated in a
>>> worker through the CanvasProxy object. However there is no way to
>>> get the video content into the canvas from the worker, since the
>>> only path right now is through attaching the media stream to a
>>> video element and a canvas drawImage operation with the video
>>> element as source. Because of this limitation it's probably fine to
>>> mark the API as exposed on Window only, but I'd like us to explore
>>> this use case and see if anything can be done to support it.
>>>
>>> Mathieu ________________________________________ From: Harald
>>> Alvestrand [harald@alvestrand.no] Sent: Tuesday, May 12, 2015 1:23
>>> PM To: public-media-capture@w3.org Subject: Re: [Exposed] attribute
>>> on our APIs
>>>
>>> Den 04. mai 2015 19:20, skrev Harald Alvestrand:
>>>> On 04/29/2015 05:17 PM, Harald Alvestrand wrote:
>>>>> Den 22. april 2015 19:18, skrev Martin Thomson:
>>>>>> On 22 April 2015 at 05:16, Harald Alvestrand
>>>>>> <harald@alvestrand.no> wrote:
>>>>>>> What considerations do people know of that dictate whether
>>>>>>> or not we should expose APIs to workers or not?
>>>>>> The primary consideration is whether we want to expose them
>>>>>> to workers.
>>>>>>
>>>>>> Maybe you should ask if anyone plans to implement X in
>>>>>> workers.
>>>>>>
>>>>> As amended by Martin:
>>>>>
>>>>> Does anyone plan to implement any part of the Media Capture and
>>>>> Streams API in workers?
>>>>>
>>>>>
>>>>
>>>> With all the responses I have seen so far:
>>>>
>>>> The conclusion is that nobody plans to implement any part of the
>>>> Media Capture and Streams API in workers, and we'll mark all our
>>>> APIs (where appropriate) with [Exposed=Window].
>>>>
>>>> I'll leave this open until Wednesday afternoon, and if the
>>>> conclusion still stands, will take that as a consensus position
>>>> and relay it to the editors for implementation.
>>>>
>>>> Harald
>>>>
>>>>
>>>
>>> The conclusion has remained standing.
>>>
>>>
>>
>>
>

Received on Tuesday, 19 May 2015 10:46:29 UTC