- From: henbos via GitHub <sysbot+gh@w3.org>
- Date: Thu, 22 Oct 2020 08:48:39 +0000
- To: public-webrtc-logs@w3.org
Replying to @youennf > I am not sure adding another constraint is the best approach, constraints are difficult to use when they are not fully orthogonal one with each other. > > Let's say we do not care about device selection and we already have a video capture track. > UAs could expose the native presets of the track camera. > Developers could then ask for a particular preset. I don't like non-ortohonal constraints either, but I don't think you get around the "non-orthogonality" by making it a second, separate step. Because the relevant variables (which device, resoluiton/fps and pixel format) are all intertwined. On one hand, if you care about performance more than which device to pick, by the time you have your track already capturing from a device it would be "too late" to pick a camera with better pixel formats at a higher resolution. By removing the pixel format from the device-picking equation you might end up with the wrong device. On the other hand, if you know which device to pick already and you just want to configure the best pixel format for an already known device, then 1) I'm not sure you gained much by splitting this up into a separate step, and 2) in order to change pixel format you have to close and re-open the camera, which makes the "start your camera" phase of a website appear to be glitchy. That said, if the app knows a device's pixel formats, it could do the tradeoff between resolution and frame rate. For example, maybe MJPEG is "bad", but the device's support for other formats like YUY2 is so bad that you would end up capturing in VGA, in which case you could say "okay, I'll pick MJPEG even though it is bad". Though maybe you could get around this with constraints and penalties instead of making it impossible to pick MJPEG. Replying to @alvestrand > In the camera above, ... Yes but note the "caveat" above the table: NV12 and YUY2 are from the built-in MacBook Pro camera and MJPEG is from an exernal webcam. This external webcam does also support YUY2 (not NV12) but the set of resolutions and frame rates in this format are different than the table's YUY2. To get a full picture on the impact I need to compare ext webcam's MJPEG with ext webcam's YUY2, for example there may also be overhead due to the fact that the camera is external, but I have not done those measurements yet! > (I assume that the measurements above were done with a conversion to RGBA). They're all conversions to I420, which is supported by all Chromium's encoders and decoders. Some encoders support NV12, but not all of them, so it's not guaranteed that NV12 is better than YUY2. > I see that it can capture in NV12 or YUV2; if the pixel format can be preserved down the chain, we might save even more power. [...] TL;DR: perhaps it should be a "pixelFormat=" constraint? We might but we might also not, depending on encoders, rendering capabilities, etc. Which format is or isn't a good choice would be very much dependant on implementation details. For the time being, we can say "MJPEG is bad because it is compressed" but having the application decide whether to use YUY2 or NV12 doesn't make much sense without exposing other browser and system capabilities. And the tradeoff between YUY2 and NV12 is much, much smaller than the difference between any of those and MJPEG. -- GitHub Notification of comment by henbos Please view or discuss this issue at https://github.com/w3c/mediacapture-main/issues/739#issuecomment-714337344 using your GitHub account -- Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config
Received on Thursday, 22 October 2020 08:48:41 UTC