W3C home > Mailing lists > Public > public-css-testsuite@w3.org > December 2012

Re: [RC6] rgb(50%, ..., ...) or rgb(..., 50%, ...) or rgb(..., ..., 50%): fractional value!

From: Rebecca Hauck <rhauck@adobe.com>
Date: Fri, 7 Dec 2012 09:43:36 -0800
To: "brian@jankonet.com" <brian@jankonet.com>, Gérard Talbot <css21testsuite@gtalbot.org>
CC: Arron Eicholz <Arron.Eicholz@microsoft.com>, Public CSS test suite mailing list <public-css-testsuite@w3.org>
Message-ID: <CCE76AA3.23F6A%rhauck@adobe.com>
Hi Brian,

I believe all you have to do is click the unsubscribe link here:

http://lists.w3.org/Archives/Public/public-css-testsuite/


-Rebecca

On 12/7/12 8:21 AM, "Brian Janko" <brian@jankonet.com> wrote:

>how can I stop getting this list via email?  (I'm asking "reply all"
>since "replying to various senders" has not helped me get an answer, so
>sorry about that, everybody.)
>
>Thanks,
>Brian
>brian@jankonet.com
>
>
>
>
>On 12/6/2012 2:55 PM, "Gérard Talbot" wrote:
>> Le Jeu 6 décembre 2012 14:09, Arron Eicholz a écrit :
>>> On Wednesday, December 05, 2012 5:50 PM Gérard Talbot wrote:
>>>>> I encountered many of these tests when scrubbing the list of things
>>>> we
>>>>> want to fix for the CSS2.1 Test Suite 2.0 release. In fact, there
>>>> are
>>>>> ~100
>>>>> tests with this issue  so coming up with a solution to this have a
>>>> big
>>>>> impact on the overall todo list.
>>>>>
>>>>> After looking at them, I noticed that all of them except for those
>>>>> named
>>>>> background-color-* use the same rgb() syntax for both the test and
>>>>> reference elements.
>>>>
>>>> Rebecca,
>>>>
>>>> Correct. All of them except those named background-color-* use the
>>>> same
>>>> rgb() syntax for both the test and reference elements.
>>>>
>>>> If the test is about color (and border-[bottom|left|right|top]-color,
>>>> background-color, outline-color are tests about color), then its
>>>> associated
>>>> reftest should be using another method, a different method for
>>>> expressing/rendering such color. So, if both the test and reftest are
>>>> using
>>>> rgb(%, %, %) and comparing a color with a background-color, then it is
>>>> *not*
>>>> using a different method, another method of expressing colors, of
>>>> rendering
>>>> colors and for comparing colors.
>>>>
>>> I disagree with what you think these tests are testing. The tests are
>>> testing if border-*-color supports the rgb() values correctly. If that
>>> is the case they should match the reference that is verified against
>>>the
>>> image cases. This would mean in anyone's implementation that you must
>>> have 3 things working correctly at the very least. 1. Image rendering
>>> must work correctly. 2. Background-color must work and match the image
>>> rendering. 3. Border-*-color must match the background reference.
>>
>> If A matches B and B matches C, then A must match C also: transitive
>> relation. All these *-color-* tests rely on background-color tests
>> accurately matching an image rendering to begin with.
>>
>>
>>> If you implemented RGB separately for each property you may have made a
>>> mistake in your copy of the code path. Who knows how people write
>>> implementations. At the very least these need to be thought of as more
>>> or less independent implementations of RGB on a particular property and
>>> not as a common function for RGB as smarter implementers implement
>>> things.
>>>
>>>> So,
>>>>
>>>>
>>>>http://test.csswg.org/suites/css2.1/nightly-unstable/html4/color-049.ht
>>>>m
>>>>
>>>> can never actually fail.
>>>>
>>> Actually it can fail if a developer implanted RGB for color and then
>>> some other developer went and implemented RGC for background-color and
>>> they did not implement things correctly or sue a common function for
>>> RGB. We are making an assumption that there is a common RGB function
>>>and
>>> in some implementations they might not have gotten far enough into
>>> coding to need a common function yet but they want to test what they
>>> have. This test can absolutely fail.
>>>
>>
>> Arron, what you describe is always possible. But it was not what I had
>> in mind. Let's assume/postulate a common rgb() function for
>> background-color, border-*-color, color, outline-color exists: then such
>> test
>> http://test.csswg.org/suites/css2.1/nightly-unstable/html4/color-049.htm
>> will not fail and can not fail.
>>
>>
>>>>> Example: border-bottom-color-049 has:
>>>>>
>>>>>       #test
>>>>>       {
>>>>>    border-bottom-style: solid;
>>>>>    border-bottom-width: 1in;
>>>>>          border-bottom-color: rgb(1%, 1%, 1%);
>>>>>          height: 0;
>>>>>       }
>>>>>
>>>>>       #reference
>>>>>       {
>>>>>          background-color: rgb(1%, 1%, 1%);
>>>>>          margin-top: 10px;
>>>>>       }
>>>>>
>>>>>
>>>>>
>>>>> First point:
>>>>> The way this is written, is the fractional color value really a
>>>>> precision issue? In other words, whether the UA rounds up or down,
>>>>> isn't it fair to assume that it'd do the same thing for both the
>>>> test
>>>>> and reference elements so they'd always match?
>>>>
>>>> It is fair to assume that it will do the same rounding (up or down)
>>>> for
>>>> both the test and reference and so they should *always* match: in
>>>> fact,
>>>> this is already verifiable and verified too. A consequence of this is
>>>> that, eg.
>>>>
>>> No you can't make that assumption unless you know exactly how the code
>>> was written.
>>
>> Empirical results then:
>>
>>
>>http://www.gtalbot.org/BrowserBugsSection/css21testsuite/color-rgb-testin
>>g.html
>>
>>
>>> Our typical implementations seem consistent but what if I
>>> am some new implementer and have never written a browser before? Then
>>> what? We can't make assumptions based on our own browser experience. We
>>> need to think as if we are new implementers that do things piece meal
>>> and may not make the best coding choices starting from scratch. The
>>> assumption I made was that at the very least background-color can be
>>> compared against a real baseline and then from there we can compare
>>> other against background-color. IF we don't like that choice
>>
>> >From the beginning, I was not aware of that testing design choice.
>>
>>
>>> then we
>>> need to use the image in all the *-color property cases.
>>
>> I thought we should use the image in all the *-color property cases but
>> now that you have explained the pivoting transitivity with
>> background-color tests, I think we can save/exempt ourselves of all this
>> updating of *-color tests.
>>
>>
>>>>
>>>>http://test.csswg.org/suites/css2.1/nightly-unstable/html4/color-049.ht
>>>>m
>>>>
>>>> can never actually fail.
>>>>
>>> Yes this case can fail as I stated above.
>>>
>>>
>>>>> Or am I missing something here?
>>>>>
>>>>>
>>>>> Second point:
>>>>> However, does this expose a different weakness in the test?
>>>>
>>>> I think it does expose a weakness in the test. An automated checking
>>>> of
>>>> background-color-* test (software comparing screenshots) would
>>>> eventually report failures that no human eyes would be able to
>>>> see/notice to begin with.
>>>>
>>>> rgb(1%, 1%, 1%) is *not* #020202 in some browsers.
>>>>
>>> This is a weakness in the spec not really a weakness in the test.
>>
>> We want tests to work elegantly around difficulties (grey areas, spec
>> silence, user agent stylesheet default values) or known weaknesses of
>> the spec and only reveal the result about a well targeted feature or
>> code situation. I could provide several examples of doing so.
>>
>>
>>
>>> The
>>> spec is not clear enough on how to handle these fractional values. I
>>> assumed that the rounding occurred based on common practices in all the
>>> painting apps I have ever used. I think for these specific cases we may
>>> need to do one of the solutions below:
>>>
>>> 1. Remove percentage from the spec since it is untestable and cannot be
>>> interoperable because we do not define rounding correctly or how
>>> scenarios like these case be verified.
>>> 2. Define rounding of color values explicitly. Right now the spec only
>>> says that values can be approximate, section 6.1.4. In my opinion
>>> rounding is the way we approximate things for this case. That is at
>>> least how every photo editing program works that I have ever tested.
>>> 3. Update the background-color cases to have 2 references one ref that
>>> is on either side of the value being defined.
>>
>> In the past, for situations involving fractional values, you have always
>> opted for solution 3.
>>
>>
>>>> rgb(1%, 1%, 1%) is equivalent to #020202 in some other browsers.
>>>>
>>>> Therefore, there is a weakness in the test which can be explained.
>>>>
>>> Actually the weakness is not in the test the weakness is in the person
>>> verifying the test.
>>
>> I presume that we also want tests to be formalized to a point where even
>> a robot would eventually be able to verify tests, especially
>> "boring"/tedious tests about colors and comparing colors. Or any
>> automated process like comparing screenshots.
>>
>>
>>> Though I am not saying that you have to have
>>> superhuman eyes, what I am saying is that the verification method may
>>> not be someone looking at the test either. And in some cases it might
>>>be
>>> best if the verification were not a normal person looking at the test.
>> You'd be surprised how much "nitpicking" or thorough or
>> strict-minded-accurate some people can be with colors and color names.
>>
>> Some people (or a software or a robot) will say that
>> http://test.csswg.org/suites/css2.1/20110323/html4/inlines-020.htm
>> is failed by all browsers. You see, lime is not green and green is not
>> lime.
>>
>> Compare
>> http://test.csswg.org/suites/css2.1/20110323/html4/blocks-011.htm
>> with
>>
>>http://test.csswg.org/suites/css2.1/20110323/html4/background-image-cover
>>-001.htm
>> and you will get some people (or even a software or a robot) to make one
>> test fail because navy is not blue and blue is not navy. If the 2 tests
>> are taken consecutively, a minority of people will understandably
>> hesitate for sure.
>>
>> Some people will not be able to make a call/verdict for these 2 tests:
>>
>>http://test.csswg.org/suites/css2.1/20110323/html4/border-top-style-009.h
>>tm
>> or
>>
>>http://test.csswg.org/suites/css2.1/20110323/html4/border-left-style-009.
>>htm
>>
>> Etc.
>>
>>
>>>> The thing is:
>>>>
>>>> background-color-049 would eventually be reported as a failure by
>>>> some
>>>> browsers while color-049 would never be reported as a failure by
>>>> any/all
>>>> browsers.
>>>>
>>> Background-color-049 if that is the case then how do we fix it? We
>>>can't
>>> unless we do:
>>>
>>> 1. Remove percentage from the spec since it is untestable and cannot be
>>> interoperable because we do not define rounding correctly or how
>>> scenarios like these case be verified.
>>> 2. Define rounding of color values explicitly. Right now the spec only
>>> says that values can be approximate, section 6.1.4. In my opinion
>>> rounding is the way we approximate things for this case. That is at
>>> least how every photo editing program works that I have ever tested.
>>> 3. Update the background-color cases to have 2 references one ref that
>>> is on either side of the value being defined.
>>>
>>
>> We do 3.
>>
>>
>>
>>
>>>> So, we're not even consistent in doing tests and we're not really
>>>> doing
>>>> good testing either.
>>>>
>>> I disagree with this statement completely. I believe you are looking at
>>> things from a known browser perspective. I am looking at these tests
>>> from a spec perspective like there were no browsers to even test this
>>>in
>>> to begin with. In fact I write almost all my test suite cases without
>>> ever opening them in a browser, that way I am never influenced by
>>> browser behavior. It is not until I am verifying my browser do I
>>> actually run them.
>>>
>>
>> If we were face-to-face or on the phone, it would be interesting to
>> discuss this. I don't necessarly disagree with you on everything.
>>
>>
>> Gérard
>>
>>>>> Since this
>>>>> is
>>>>> testing rgb() with % args with a particular property in focus, is it
>>>> an
>>>>> accepted practice to use the same input for the reference rendering
>>>>> using
>>>>> a different property?
>>>>
>>>> A different property: yes, in a very wide majority of tests and
>>>> situations. But here, what annoyed me from the beginning is that we
>>>> are
>>>> comparing 2 colors with the same feature, with the same method of
>>>> "creating" such color code, which is rgb(%, %, %).
>>>>
>>> We are testing property supporting the color here. The verification of
>>> the color is actually tested by the image which is done in
>>> background-color cases.  Again it seems like you are mixing up what is
>>> actually being tested with browser testing.
>>>
>>>>> I understand you must make assumptions about the
>>>>> behavior or stability of everything you use in a test file. But if
>>>> this
>>>>> test failed, it would be difficult to tell right away where the
>>>> point of
>>>>> failure is - the test property, the ref property or the rgb() value.
>>>>   If
>>>>> it is indeed acceptable to construct a test this way, then my first
>>>>> point
>>>>> still stands.
>>>>>
>>>>> Third point:
>>>>> Specifically for the tests with 50% values - Nothing about those
>>>> tests
>>>>> is
>>>>> special to 50%. I think these can be changed to 20%,40%,60% or 80%
>>>> and
>>>>> compared to non-fractional rgb values (that is, if the %'s need to
>>>> be
>>>>> removed at all)
>>>>
>>>> Those percentage numbers are good numbers:
>>>>
>>>> rgb(20%, 20%, 20%) == #333333  (20% of 255 == 51)
>>>>
>>>> rgb(40%, 40%, 40%) == #666666  (40% of 255 == 102)
>>>>
>>>> rgb(60%, 60%, 60%) == #999999  (60% of 255 == 153)
>>>>
>>>> rgb(80%, 80%, 80%) == #CCCCCC  (80% of 255 == 204)
>>>>
>>>>
>>>>> The tests named background-color-* all use pngs as a reference, so
>>>> those
>>>>> are definitely problematic. Changing the 50% tests to 40% would fix
>>>> some
>>>>> of them, but I don't have a solution for those testing 1% and 99%.
>>>>>
>>>>> And, speaking briefly about it with Arron this morning, I
>>>> understand
>>>>> there
>>>>> are several hundred more that have this issue - basically all
>>>>> color-related tests, so this extends wider than what we currently
>>>> have
>>>>> identified for the 2.0 release.
>>>>>
>>>>> Thoughts?
>>>> Proposal
>>>>
>>>> a) remove all the tests with rgb(1%, , ) or with rgb( , 1%, ) or with
>>>> rgb( , , 1%) and with rgb(99%, , ) or with rgb( , 99%, ) or with rgb(
>>>> ,
>>>> , 99%)
>>> We cannot remove those tests we are testing the boundary cases and
>>> standard testing practices always test boundary scenarios. This would
>>>be
>>> a bad practice to go against standard testing practices for testing.
>>> Also if we remove those cases we just proved that percentage values
>>> cannot be tested or be interoperable this means that the feature should
>>> be removed from the specification since we just proved that it can't
>>> work interoperably.
>>>
>>>> b) convert the ones using rgb(50%, , ) or with rgb( , 50%, ) or with
>>>> rgb( , , 50%) to be using 40% instead
>>>> c) create support files with #660000, #006600, #000066 and #666666
>>>>
>>> I'm ok with this the 50% values are nominal values and they can be any
>>> value between 2% and 98% so it really doesn't matter what those values
>>> are. I just picked 50% because it was half way between 0 and 100. This
>>> is the case for all nominal value cases if there are other scenarios
>>>for
>>> nominal cases that fall into the fractional problem we should probably
>>> address those in a similar way.
>>>
>>> --
>>> Thanks,
>>> Arron Eicholz
>>>
>>>
>>>
>>
>
Received on Friday, 7 December 2012 17:43:42 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Friday, 7 December 2012 17:43:47 GMT