Re: [css-writing-modes-3] text-orientation-upright-directionality-001 test and reference file

On 07/01/2016 04:57 PM, Gérard Talbot wrote:
> Jonathan,
> 
> [test]
> http://test.csswg.org/suites/css-writing-modes-3_dev/nightly-unstable/html/text-orientation-upright-directionality-001.htm
> 
> [reference file]
> http://test.csswg.org/suites/css-writing-modes-3_dev/nightly-unstable/html/reference/text-orientation-upright-directionality-001-ref.htm 
> 
> 
> 
> I just entered the test result for Firefox 50 and Chrome 53 for your test. I suggest 2 improvements.
> 
> 1-
> Right now, only Firefox supports 'writing-mode: sideways-lr' and 'writing-mode: sideways-rl'. If the reference file also uses 
> <div style="writing-mode:sideways-rl"> and <div style="writing-mode:sideways-lr"> (which is the tested feature of the test), 
> then browsers which do not support sideways-* writing-modes will "pass" (false positive) your test. (This is the case with 
> Chrome 53 and with Edge 13 too.) In order to work around this, I suggest to create an image of the expected results and use a 
> textual description for the pass/fail conditions: we have done this in about 16 tests.
> 
> Eg
> http://test.csswg.org/suites/css-writing-modes-3_dev/nightly-unstable/html/text-orientation-010.htm
> 
> Sometimes, we also indicate that inter-character spacing can vary.
> 
> http://test.csswg.org/suites/css-writing-modes-3_dev/nightly-unstable/html/text-orientation-upright-vrl-002.htm

We have a pretty strong preference for reftests rather than manual tests,
so it's better not to do this when we can avoid it. I think the test is
acceptable as-is; if we really want to, we could add 'writing-mode: vertical-rl'
in either the test or the reference, to make sure it fails on UAs that support
vertical-rl but not sideways-lr. But, looking at the test, I think it's probably
OK as-is. (We have a general issue with tests that depend on other tests passing
in order to be valid, and although in most cases they fail correctly, sometimes
like this, the dependent test will pass. It's not ideal, but it's OK.)

> 2-
> I would split your test in 2 distinct, separate tests..
> 
> I can do this for you if you want...

I don't think this is quite necessary, as long as the items all fit within the reftest screenshot area.

> Addendum
> --------
> 
> I just noticed that we already had 2 tests doing what your test was aiming at checking:
> 
> http://test.csswg.org/suites/css-writing-modes-3_dev/nightly-unstable/html/text-orientation-upright-slr-017.htm
> 
> http://test.csswg.org/suites/css-writing-modes-3_dev/nightly-unstable/html/text-orientation-upright-srl-018.htm
> 
> ... although these 2 tests did not use Hebrew and Arabic characters like you did with your 
> text-orientation-upright-directionality-001 test. Interestingly, Firefox 47 passes text-orientation-upright-slr-017 and 
> text-orientation-upright-srl-018 but fails your text-orientation-upright-directionality-001 test. So, this is a good addition.

These don't check the same thing as Jonathan's test: he's making sure that the RTL characters don't get reordered, which they 
would if 'text-orientation: upright' took effect at the bidi level.

~fantasai

Received on Friday, 9 March 2018 05:44:24 UTC