Re: [mediacapture-transform] Track transferability requirement adds complexity to some common use cases (#116)

> > This is a common use case,
> 
> Seems more common _not_ to have a "before" preview: None of the big sites with video effects have it.
> 

Are you insinuating this use case shouldn't be supported?


> > ... and there are many others that require or prefer the track on window and don't need or want a track in the worker.
> 
> Which websites require the "before" video on window? How do you know what they "prefer"?
> 

All VC sites I know of allow you to switch between the processed and unprocessed track when configuring which effects you want to apply. They don't necessarily show both at the same time, but they allow you to switch between effects/no effects interactively and both tracks are on Window at the same time. Demos and tools for developing effects are more likely to show the before and after views simultaneously.

Also, all Web sites that do video processing for MediaStreamTracks do it with the track on Window. They keep both the processed and unprocessed tracks on Window, regardless of whether they show the unprocessed one or not.

Can you mention one that is doing it with the track on the worker, without at least having a clone on Window?
Or at least one that has expressed their preference to do it that way?


> As #117 clarified, having a track tucked away in the worker using it is neither a burden nor a liability. It's quite clean: it lets the worker stop the source once piping ends. This cleanly reacts to downstream JS closing the "after" track, regardless of what thread it's on.
> 
> > The current API shape forces a clone in such cases, which shouldn't be necessary.
> 
> A "before" preview element is a separate sink with potentially separate concerns. Using clone to separate those concerns should be encouraged. For instance, tying their lifespans together seems like a mistake.
> 

That's at best an opinion unsupported by any evidence.


> In contrast, your alternative API seems flawed:
> 
> 1. poor encapsulation: the worker has no way to stop the gUM track it's consuming

For the use case when the application wants to stop the gUM track from the Window, this is a better design. This covers all applications I'm aware of. Also, the proposal does not prevent a hypothetical application that wants to stop the gUM track from the Worker from transferring the track to the Worker and managing its lifetime here. I am just not aware of any such application. 

> 2. redundancy: the window has to manage two tracks, even when no "before" preview is needed

Why is managing one track on Window and one on Worker better, especially if all the application logic is on Window?

> 3. tight coupling: if the website forgets to clone its "before" track then stopping it stops the "after" track as well
> 
If that's what the application wants I don't see why it should be forbidden.  My proposal does not prevent cloning and transferring. It just doesn't force it. Your argument seems to be that forcing it is a good idea.


> IOW leaving the window to manage the source discourages encapsulation and encourages repurposing without cloning, creating tight coupling that may surprise developers.

That contradicts real world applications, where all the track lifetime management and other logic is on Window and the worker is used only for processing logic.


-- 
GitHub Notification of comment by guidou
Please view or discuss this issue at https://github.com/w3c/mediacapture-transform/issues/116#issuecomment-2536360574 using your GitHub account


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Wednesday, 11 December 2024 15:45:50 UTC