Re: API proposals for raw data - 1 - Introduction

Thanks Harald for doing this!

I think grasping your proposals ("Raw data API - n" where n is 2,3,4) 
would be even easier if simple example code on how to do the use cases
spelled out in 
https://lists.w3.org/Archives/Public/public-webrtc/2018May/0037.html 
could be provided.

Is that something you could do (or perhaps present at the f2f)?

//Stefan


On 2018-05-29 08:32, Harald Alvestrand wrote:
> **
> 
> *
> 
> This is part 1 of a multipart posting*, *laying out various API ideas in
> response to the need for "raw data access".
> 
> Problem:
> 
> 
> Many creative usages of WebRTC audio and video data require direct access to the
> data - either in raw form or in encoded form.
> 
> 
> All of these things are possible today, but they are not simple.
> 
> 
> The current interfaces to get raw samples are:
> 
>    *
> 
>      Use WebAudio to generate samples as arrays of floats in WebAudio’s clock
>      (example <https://webrtc.github.io/samples/src/content/getusermedia/volume/>)
> 
>    *
> 
>      Use Canvas to paint from a <video> tag (example
>      <https://webrtc.github.io/samples/src/content/getusermedia/canvas/>)
> 
> For injecting raw data we have:
> 
>    *
> 
>      Use WebAudio to generate a MediaStreamTrack from raw audio data
> 
>    *
> 
>      Use canvas.captureStream() to capture frames from a Canvas (example
>      <https://webrtc.github.io/samples/src/content/capture/canvas-video/>)
> 
> For getting encoded data we have:
> 
>    *
> 
>      MediaRecorder (example
>      <https://webrtc.github.io/samples/src/content/getusermedia/record/>)
> 
> For injecting encoded data we have:
> 
>    *
> 
>      MSE feeding a <video> tag, and then generating a MediaStreamTrack from the
>      <video> tag
> 
> 
> 
> All of these mechanisms are somewhat convoluted, they impose restrictions on the
> form of data that can be delivered (often imposing transcoding costs), and
> require the application writer to be familiar with multiple shapes of API surface.
> 
> 
> A simpler method is desirable.
> 
> *
> 
> -- 
> Surveillance is pervasive. Go Dark.
>

Received on Wednesday, 30 May 2018 11:05:35 UTC