RE: [RC6] rgb(50%, ..., ...) or rgb(..., 50%, ...) or rgb(..., ..., 50%): fractional value!

On Wednesday, December 05, 2012 5:50 PM Gérard Talbot wrote:
> >
> > I encountered many of these tests when scrubbing the list of things we
> > want to fix for the CSS2.1 Test Suite 2.0 release. In fact, there are
> > ~100
> > tests with this issue  so coming up with a solution to this have a big
> > impact on the overall todo list.
> >
> > After looking at them, I noticed that all of them except for those
> > named
> > background-color-* use the same rgb() syntax for both the test and
> > reference elements.
> 
> 
> Rebecca,
> 
> Correct. All of them except those named background-color-* use the same
> rgb() syntax for both the test and reference elements.
> 
> If the test is about color (and border-[bottom|left|right|top]-color,
> background-color, outline-color are tests about color), then its associated
> reftest should be using another method, a different method for
> expressing/rendering such color. So, if both the test and reftest are using
> rgb(%, %, %) and comparing a color with a background-color, then it is *not*
> using a different method, another method of expressing colors, of rendering
> colors and for comparing colors.
> 

I disagree with what you think these tests are testing. The tests are testing if border-*-color supports the rgb() values correctly. If that is the case they should match the reference that is verified against the image cases. This would mean in anyone's implementation that you must have 3 things working correctly at the very least. 1. Image rendering must work correctly. 2. Background-color must work and match the image rendering. 3. Border-*-color must match the background reference.

If you implemented RGB separately for each property you may have made a mistake in your copy of the code path. Who knows how people write implementations. At the very least these need to be thought of as more or less independent implementations of RGB on a particular property and not as a common function for RGB as smarter implementers implement things.

> So,
> 
> http://test.csswg.org/suites/css2.1/nightly-unstable/html4/color-049.htm

> 
> can never actually fail.
> 

Actually it can fail if a developer implanted RGB for color and then some other developer went and implemented RGC for background-color and they did not implement things correctly or sue a common function for RGB. We are making an assumption that there is a common RGB function and in some implementations they might not have gotten far enough into coding to need a common function yet but they want to test what they have. This test can absolutely fail. 

> 
> >
> > Example: border-bottom-color-049 has:
> >
> >      #test
> >      {
> >  border-bottom-style: solid;
> >  border-bottom-width: 1in;
> >         border-bottom-color: rgb(1%, 1%, 1%);
> >         height: 0;
> >      }
> >
> >      #reference
> >      {
> >         background-color: rgb(1%, 1%, 1%);
> >         margin-top: 10px;
> >      }
> >
> >
> >
> > First point:
> > The way this is written, is the fractional color value really a
> > precision issue? In other words, whether the UA rounds up or down,
> > isn't it fair to assume that it'd do the same thing for both the test
> > and reference elements so they'd always match?
> 
> 
> It is fair to assume that it will do the same rounding (up or down) for
> both the test and reference and so they should *always* match: in fact,
> this is already verifiable and verified too. A consequence of this is
> that, eg.
> 

No you can't make that assumption unless you know exactly how the code was written. Our typical implementations seem consistent but what if I am some new implementer and have never written a browser before? Then what? We can't make assumptions based on our own browser experience. We need to think as if we are new implementers that do things piece meal and may not make the best coding choices starting from scratch. The assumption I made was that at the very least background-color can be compared against a real baseline and then from there we can compare other against background-color. IF we don't like that choice then we need to use the image in all the *-color property cases.

> http://test.csswg.org/suites/css2.1/nightly-unstable/html4/color-049.htm

> 
> can never actually fail.
> 

Yes this case can fail as I stated above.


> 
> > Or am I missing something here?
> >
> >
> > Second point:
> > However, does this expose a different weakness in the test?
> 
> 
> I think it does expose a weakness in the test. An automated checking of
> background-color-* test (software comparing screenshots) would
> eventually report failures that no human eyes would be able to
> see/notice to begin with.
> 
> rgb(1%, 1%, 1%) is *not* #020202 in some browsers.
> 

This is a weakness in the spec not really a weakness in the test. The spec is not clear enough on how to handle these fractional values. I assumed that the rounding occurred based on common practices in all the painting apps I have ever used. I think for these specific cases we may need to do one of the solutions below:

1. Remove percentage from the spec since it is untestable and cannot be interoperable because we do not define rounding correctly or how scenarios like these case be verified.
2. Define rounding of color values explicitly. Right now the spec only says that values can be approximate, section 6.1.4. In my opinion rounding is the way we approximate things for this case. That is at least how every photo editing program works that I have ever tested.
3. Update the background-color cases to have 2 references one ref that is on either side of the value being defined.

> 
> rgb(1%, 1%, 1%) is equivalent to #020202 in some other browsers.
> 
> Therefore, there is a weakness in the test which can be explained.
> 

Actually the weakness is not in the test the weakness is in the person verifying the test. Though I am not saying that you have to have superhuman eyes, what I am saying is that the verification method may not be someone looking at the test either. And in some cases it might be best if the verification were not a normal person looking at the test.

> The thing is:
> 
> background-color-049 would eventually be reported as a failure by some
> browsers while color-049 would never be reported as a failure by any/all
> browsers.
> 

Background-color-049 if that is the case then how do we fix it? We can't unless we do:

1. Remove percentage from the spec since it is untestable and cannot be interoperable because we do not define rounding correctly or how scenarios like these case be verified.
2. Define rounding of color values explicitly. Right now the spec only says that values can be approximate, section 6.1.4. In my opinion rounding is the way we approximate things for this case. That is at least how every photo editing program works that I have ever tested.
3. Update the background-color cases to have 2 references one ref that is on either side of the value being defined.

> So, we're not even consistent in doing tests and we're not really doing
> good testing either.
> 

I disagree with this statement completely. I believe you are looking at things from a known browser perspective. I am looking at these tests from a spec perspective like there were no browsers to even test this in to begin with. In fact I write almost all my test suite cases without ever opening them in a browser, that way I am never influenced by browser behavior. It is not until I am verifying my browser do I actually run them.

> 
> > Since this
> > is
> > testing rgb() with % args with a particular property in focus, is it an
> > accepted practice to use the same input for the reference rendering
> > using
> > a different property?
> 
> 
> A different property: yes, in a very wide majority of tests and
> situations. But here, what annoyed me from the beginning is that we are
> comparing 2 colors with the same feature, with the same method of
> "creating" such color code, which is rgb(%, %, %).
> 

We are testing property supporting the color here. The verification of the color is actually tested by the image which is done in background-color cases.  Again it seems like you are mixing up what is actually being tested with browser testing.

> 
> > I understand you must make assumptions about the
> > behavior or stability of everything you use in a test file. But if this
> > test failed, it would be difficult to tell right away where the point of
> > failure is - the test property, the ref property or the rgb() value.  If
> > it is indeed acceptable to construct a test this way, then my first
> > point
> > still stands.
> >
> > Third point:
> > Specifically for the tests with 50% values - Nothing about those tests
> > is
> > special to 50%. I think these can be changed to 20%,40%,60% or 80% and
> > compared to non-fractional rgb values (that is, if the %'s need to be
> > removed at all)
> 
> 
> Those percentage numbers are good numbers:
> 
> rgb(20%, 20%, 20%) == #333333  (20% of 255 == 51)
> 
> rgb(40%, 40%, 40%) == #666666  (40% of 255 == 102)
> 
> rgb(60%, 60%, 60%) == #999999  (60% of 255 == 153)
> 
> rgb(80%, 80%, 80%) == #CCCCCC  (80% of 255 == 204)
> 
> 
> >
> > The tests named background-color-* all use pngs as a reference, so those
> > are definitely problematic. Changing the 50% tests to 40% would fix some
> > of them, but I don't have a solution for those testing 1% and 99%.
> >
> > And, speaking briefly about it with Arron this morning, I understand
> > there
> > are several hundred more that have this issue - basically all
> > color-related tests, so this extends wider than what we currently have
> > identified for the 2.0 release.
> >
> > Thoughts?
> 
> Proposal
> 
> a) remove all the tests with rgb(1%, , ) or with rgb( , 1%, ) or with
> rgb( , , 1%) and with rgb(99%, , ) or with rgb( , 99%, ) or with rgb( ,
> , 99%)

We cannot remove those tests we are testing the boundary cases and standard testing practices always test boundary scenarios. This would be a bad practice to go against standard testing practices for testing. Also if we remove those cases we just proved that percentage values cannot be tested or be interoperable this means that the feature should be removed from the specification since we just proved that it can't work interoperably.

> b) convert the ones using rgb(50%, , ) or with rgb( , 50%, ) or with
> rgb( , , 50%) to be using 40% instead
> c) create support files with #660000, #006600, #000066 and #666666
> 

I'm ok with this the 50% values are nominal values and they can be any value between 2% and 98% so it really doesn't matter what those values are. I just picked 50% because it was half way between 0 and 100. This is the case for all nominal value cases if there are other scenarios for nominal cases that fall into the fractional problem we should probably address those in a similar way.

--
Thanks,
Arron Eicholz

Received on Thursday, 6 December 2012 19:12:51 UTC