Concluding on "Proposal: align video scaling with JSEP"

I got two responses [1][2], and both seemed to like the proposal, but 
[2] proposed we should add some more info about how black borders are 
applied.

My reading of that is that the black borders (if needed due to aspect 
ratio mismatch) would be present when rendering in a video element, but 
that JSEP specifies well enough what happens at the encoder/sender 
stage. Therefore I added a note about what happens when rendering (which 
is specced in HTML5.1) to the PR [3].

Based on this I'll now recommend to the other chairs (since I made the 
PR I won't be part of making the decision) that we merge [3] and close 
[4] (which by the way happens to be a CR blocking Issue).

Stefan

[1] https://lists.w3.org/Archives/Public/public-webrtc/2017Sep/0080.html
[2] https://lists.w3.org/Archives/Public/public-webrtc/2017Oct/0002.html
[3] https://github.com/w3c/webrtc-pc/pull/1570
[4] https://github.com/w3c/webrtc-pc/issues/1283

On 05/10/17 12:16, Stefan Håkansson LK wrote:
> On 02/10/17 18:44, Stefan Håkansson LK wrote:
>> On 02/10/17 18:30, Cullen Jennings (fluffy) wrote:
>>>
>>>> On Sep 26, 2017, at 5:56 AM, Stefan Håkansson LK
>>>> <stefan.lk.hakansson@ericsson.com <mailto:stefan.lk.hakansson@ericsson.com>>
>>>> wrote:
>>>>
>>>> Therefore I propose we remove the current text on 'centering, scaling,
>>>> cropping', and instead reference JSEP section 3.6.2.
>>>
>>> That is what we originally had and Martin and others complained that more advice
>>> was needed. I don't think we can change the current advice but it still seems we
>>> need to say something to have consistent behaviour between browsers. I do think
>>> consistent behaviour is important because removing this going to result in
>>> letter boxing and when one thing does it one way and something else does it
>>> another, you can end up with the situation with a black border all the way
>>> around the outside of image. Apps will need to know what the browser are going
>>> to do so that the application can compensate to achieve the desired outcome for
>>> the application.
>>>
>>> I think EKR and Justin have argued for one of the principals being don't loose
>>> information by cropping as this is really bad in cases like screen share. They
>>> are also arguing for don't change aspect ration (other than minor bit caused by
>>> rounding error in scaling ).  So I think that means this needs to say something
>>> along the lines of black border is added to image to get to correct aspect
>>> ratio, then it is scaled to get to correct size.
>>
>> Part of my reasoning is that the sender side does not (usually) know the
>> rendering size on the remote end. All the sender knows is what the
>> receiver is able to receive/decode, it does not know the rendering
>> dimensions so it would be impossible to know if there would be a black
>> border anywhere when this is eventually rendered. "Correct" size is
>> unknown as I have understood it.
> 
> I added a note trying to explain that letterheading/pillarheading is the
> result if the track aspec ratio is different from the video element
> aspect ratio (regardless of whether video has been rescaled or not):
> https://github.com/w3c/webrtc-pc/pull/1570/commits/b7cab1df0df8a7c05459dc0079b94662a2bde2e9
> 
> 
>

Received on Friday, 6 October 2017 08:43:03 UTC