Re: [mediacapture-transform] Add AudioTrackGenerator or equivalent (#124)

I'm open to seeing a raw source proposal for RTCPeerConnection, but I see the case for AudioTrackGenerator as very strong. Note that encoded source is not a real replacement for ATG (although it can help for applications that want to take care of encoding on their own).

1. I agree that AudioTrackGenerator is generally a poor choice to use as input for local playback or Web Audio, but that is no reason not to have it. Audio processing is the core AudioWorklet use case, but for many instances of that use case, AudioWorklet is a lot worse than AudioTrackGenerator with these same sinks. We're not discussing taking AW out of the Web Platform because it is bad if the developer misuses it (and it is pretty easy to misuse). I think it makes sense to apply the same criteria to ATG. 
3. For MediaRecorder, the WebKit issue seems to be a matter of implementation and not a problem with the model. It should be fixable without any changes to either API. The Chromium implementation does not have this problem, for example.
4. We already have a well-defined spec for ATG, we just have to say audio is supported and we're done. We have a single place to evolve features for both cases. For a peer-connection raw source, we have to introduce extra complexity so that PC works with both tracks and raw sources (the solution might be to define an ATG-like object internally in webrtc-pc). We might end up doing a lot of duplicate work to basically define a track that only works with peer connections.  
5. Chrome's experience is that the ATG model works well in production for audio processing. A lot of the implementation is shared with video, so it is also not difficult to implement.
6. We could work around the sink issue by having ATG deliver audio only to peer connections or MSTPs by default, but with an explicit option in the constructor to allow delivery for other sinks. This allows applications that know what they're doing to use it as a regular track for any sink. 


-- 
GitHub Notification of comment by guidou
Please view or discuss this issue at https://github.com/w3c/mediacapture-transform/issues/124#issuecomment-4206783098 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Wednesday, 8 April 2026 13:56:39 UTC