Re: WebRTC Next Version Use Cases: Now available!

Hello,


I research Immersive Telepresence over the Internet.

I read "WebRTC-NV Use Cases" and I'm really excited to read REQ-ID N25.

https://www.w3.org/TR/2018/WD-webrtc-nv-use-cases-20181211/


Since Immersive Telepresence is not commercial-level but research-level 
currently, I have not sent any request to WebRTC-NV discussion.

However, I think this document suggest that some people have similar 
interests, so I'll send a user report summarizing about my research area.

This report evaluates how much synchronization can be done with current 
WebRTC specifications in various network environments.

I hope you find it helpful.

https://docs.google.com/document/d/1lYQuXtIPJnLc-_C87Fi6nqqkxFisgdQawEXrwmUF_jA/edit#heading=h.t1936t1lbikd

#I'm not a native English speaker, so if you do not understand my 
report, please ask me.


Best Regards,

Toshiya Nakakura


On 2018/10/08 23:19, Stojiljkovic, Aleksandar wrote:
> Hello,
> > There might be a potential additional requirement: hardware 
> acceleration (say WebGL/WebGPU) should be available to process video 
> content.
>
> I started expanding this in Issue 17. 
> <https://github.com/w3c/webrtc-nv-use-cases/issues/17>
> Did you mean something else?
> Kind Regards,
> Aleksandar
> ------------------------------------------------------------------------
> *From:* Bernard Aboba [Bernard.Aboba@microsoft.com]
> *Sent:* Saturday, October 06, 2018 1:52 AM
> *To:* youenn fablet
> *Cc:* WebRTC WG
> *Subject:* Re: WebRTC Next Version Use Cases: Now available!
>
> Agree with your suggested next steps.  Sounds like you have
> already begun to drill down on the requirements. Understanding
> if they are real or just “nice to have” is very important.
>
> Would you like to help with the discussion at TPAC and/or with the slides?
>
> On Oct 5, 2018, at 2:15 PM, youenn fablet <yfablet@apple.com 
> <mailto:yfablet@apple.com>> wrote:
>
>> Thanks Bernard,
>>
>> This is a great document and I like the list of requirements.
>> I am curious about the next steps you would like to see happening there.
>>
>> It seems to me we could do things like:
>> - Validate/extend the list of requirements
>> - List what is missing/sub-optimal in the web platform to fulfill the 
>> requirements.
>>
>> Looking at section 3.3 funny hats for instance:
>> - Web Audio can be used to get access to the raw audio data and 
>> modify it (N20, N21, N22).
>> - Web Audio somehow supports N24 if we rewrite N24 to something like: 
>> "It must be possible to do efficient media manipulation out of the 
>> main thread."
>> - N23 supports seems to be missing but it is unclear whether this is 
>> a mandatory requirement that derives from the use cases.
>> The question that arises is whether Web Audio has any shortcoming 
>> with regards to WebRTC media processing.
>>
>> If we look at video for the same section:
>> - Video element + Canvas + requestAnimationFrame allows supporting 
>> N20, N21, N22
>> - OffscreenCanvas supports N24
>> - N23 support seems to be missing
>> There might be a potential additional requirement: hardware 
>> acceleration (say WebGL/WebGPU) should be available to process video 
>> content.
>>
>> Also, while N20, N21 and N22 are fulfilled, I believe the current web 
>> platform support has some shortcomings:
>> - requestAnimationFrame kicks in no matter whether the track has a 
>> new frame or not.
>> - Canvas is ARGB while track frames might be YUV which creates a 
>> conversion cost.
>> These shortcomings might limit the overall efficiency, thus limiting 
>> the processing in terms of frame rate and/or frame resolution.
>>
>> Hope this helps,
>> Y
>>
>>> On Sep 4, 2018, at 11:44 AM, Bernard Aboba 
>>> <Bernard.Aboba@microsoft.com <mailto:Bernard.Aboba@microsoft.com>> 
>>> wrote:
>>>
>>> Actually, a better link to view it is here: 
>>> https://w3c.github.io/webrtc-nv-use-cases/ 
>>> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw3c.github.io%2Fwebrtc-nv-use-cases%2F&data=02%7C01%7CBernard.Aboba%40microsoft.com%7C7838b8800b32481833aa08d62b07b571%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636743709447322819&sdata=TgryuqswpHoqRpt93Y%2FMxqDVS3GSf873MCZXclk5tN8%3D&reserved=0>
>>> ------------------------------------------------------------------------
>>> *From:*Bernard Aboba
>>> *Sent:*Tuesday, September 4, 2018 11:39:00 AM
>>> *To:*public-webrtc@w3.org <mailto:public-webrtc@w3.org>
>>> *Subject:*WebRTC Next Version Use Cases: Now available!
>>> An initial draft of one of the world's least remarkable documents has become available. By clicking on the links below, your eyes could be among the first to see it! Parental discretion advised!
>>> To obtain your complementary copy of "WebRTC Next Version Use Cases", click here:
>>> https://cdn.rawgit.com/w3c/webrtc-nv-use-cases/6643c1b4/index.html  <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcdn.rawgit.com%2Fw3c%2Fwebrtc-nv-use-cases%2F6643c1b4%2Findex.html&data=02%7C01%7CBernard.Aboba%40microsoft.com%7C7838b8800b32481833aa08d62b07b571%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636743709447332882&sdata=xUtzcPVX1y3npF5NdWwSHA2fDifyjij9EyZo0b%2BJRXI%3D&reserved=0>
>>> To enable readers to express their outrage and dismay, Issues can be filed here:
>>> https://github.com/w3c/webrtc-nv-use-cases/issues  <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fw3c%2Fwebrtc-nv-use-cases%2Fissues&data=02%7C01%7CBernard.Aboba%40microsoft.com%7C7838b8800b32481833aa08d62b07b571%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636743709447342834&sdata=rpXwVHt8t%2ByalKpNwCTv5MS87sEgyCs73xhi%2FJ5hDAs%3D&reserved=0>
>>