RE: CSS 2.1 @charset tests invalidated by Level 3

On Wednesday, February 11, 2015 6:48 AM Simon Pieters [mailto:simonp@opera.com] wrote:
> On Wed, 11 Feb 2015 19:35:32 +0400, Arron Eicholz <arronei@microsoft.com>
> wrote:
> 
> > On Wednesday, February 11, 2015 1:00 AM Simon Pieters
> > [mailto:simonp@opera.com] wrote:
> >> On Tue, 10 Feb 2015 14:27:01 +0400,  wrote:
> >>
> >> > On Mon, Dec 3, 2012 at 7:10 PM, Henri Sivonen <hsivonen@iki.fi>
> wrote:
> >> >> CSS Syntax Level 3 invalidates some @charset tests in the CSS 2.1
> >> >> test suite. It’s not good for anyone when people think that a
> >> >> Level 3 implementation is buggy when it correctly implements Level
> >> >> 3 but the Level 2.1 test suite shows red. Therefore, please
> >> >> rescind the following tests per
> >> >> http://w3cmemes.tumblr.com/post/31865121758/the-joker-shares-

> his-
> >> appr
> >> >> oach-on-css2-1-issues
> >> >>
> >> >> Tests that are invalid per Level 3 but still pass (i.e. green for
> >> >> a different reason than the reason why they are supposed to be
> green):
> >> >> http://test.csswg.org/suites/css2.1/20110323/html4/at-charset-001.

> >> >> htm
> >> >> http://test.csswg.org/suites/css2.1/20110323/html4/at-charset-044.

> >> >> htm
> >>
> >> Fixed the tests.
> >
> > You created new tests for CSS-syntax. Please move these tests there
> > (CSS-Syntax) and revert your changes to CSS 2.1. Those CSS 2.1 tests
> > were correct for 2.1. And will remain correct unless we do some major
> > reworking of the spec in errata which I seriously doubt.
> 
> We decided in Sydney to errata 2.1 and fix the tests for this issue. Peter Linss
> suggested that I just fix the tests in place, so that's what I did.
> The errata is not done yet, though. I think character encoding detection can
> be fixed without major reworking on the spec since it's a preprocessing step.
> 
> Does your request to move/revert stand with that in mind? Or do you
> disagree with 2.1 errata for this?

I was not part of the discussion in Sydney but if the CSS 2.1 errata matches what is in CSS-Syntax then I am fine with it. If we are not changing CSS 2.1 errata to match CSS-Syntax then I have the issue. 

The dated copy of 20110323 should never change. It’s a dated "stable" copy and it is what it is, good, bad or otherwise. The dated copies are of the state of the world at that time, they should never advance.

I hope any changes we make are in the current copy of the test cases, the cases that are published to the nightly location.

> 
> > It may be a good idea to put a note in the CSS 2.1 test cases that
> > they have be superseded by new tests. Maybe even link to the new tests.
> >
> > <p>Note: This test has been superseded by definitions from a newer
> > level of CSS (<a href="">at-charset-0xx</a>).</p>
> >
> > Think of it this way... As an implementer I am not required to
> > implement in any particular order. I can go ahead and implement CSS
> > 2.1 first and then at some later date implement CSS-Syntax. The test
> > suites should remain independent for that reason. The only order I
> > have to follow is newer levels of specs override older but I am not
> > required to implement the latest and greatest right from the start.
> 
> Sure. But you still want the latest 2.1.

The latest 2.1 right now, does not match CSS-Syntax. If we write the errata to match then of course it would.

Maybe this happened in Sydney but I don't see and changes to the CSS 2.1 spec since Sept. 25th and I see no errata that covers the changes that the tests require in CSS 2.1.

Right now if I were to review tests against 2.1 they are all invalid and incorrect test cases. I need 2.1 updated, errata to 2.1 or the cases moved to CSS-syntax in order to approve the tests.

> >> >> Tests whose validity was questionable to begin with:
> >> >> http://test.csswg.org/suites/css2.1/20110323/html4/at-charset-013.

> >> >> htm
> >>
> >> Would you like this test to be removed? Or annotated as "MAY"?
> >> Something else? (We didn't discuss this test in particular at the
> >> f2f.)
> >
> > Annotated would be good. I am thinking we might want to add a flag to
> > the test case (replaced??). Though I am not sure what yet. For now
> > provide a note:
> >
> > <p>Note: This test has been superseded by definitions from a newer
> > level of CSS (<a href="">at-charset-0xx</a>).</p>
> 
> I don't think this one has changed in css-syntax -- you still use the document's
> encoding if there's no other encoding information, and what the document's
> encoding is is not up to CSS.
> 

Sure no problem. I was just trying to show an example of what needed to be done, not specifically calling out test case at-charset-013.

> > You should also correct the help link to point to CSS-Syntax these new
> > tests aren't part of CSS 2.1.
> >
> >> >
> >> > It looks like these tests are still the latest. What should my
> >> > expectations be regarding getting CSS 2.1 tests rescinded when they
> >> > are invalidated by newer levels?
> >>
> >
> > The test are valid tests still for CSS 2.1. Though they may have been
> > superseded by CSS-Syntax doesn't make them any less valid for CSS 2.1.
> > The tests need a note in them stating that they are superseded.
> >
> > On a brighter note the tests seem perfectly ok for CSS-Syntax I will
> > have no problem approving them once they are tweaked per my notes
> above.
> 
> Great, thanks for reviewing.
> 

--
Thanks,
Arron Eicholz

Received on Wednesday, 11 February 2015 20:01:17 UTC