RE: CSS 2.1 @charset tests invalidated by Level 3

On Wednesday, February 11, 2015 2:45 PM Tab Atkins Jr. [mailto:jackalmage@gmail.com] wrote:
> On Thu, Feb 12, 2015 at 3:15 AM, Arron Eicholz <arronei@microsoft.com>
> wrote:
> > On Wednesday, February 11, 2015 7:46 AM Ms2ger
> [mailto:ms2ger@gmail.com] wrote:
> >> On 02/11/2015 04:35 PM, Arron Eicholz wrote:
> >> > You created new tests for CSS-syntax. Please move these tests there
> >> > (CSS-Syntax) and revert your changes to CSS 2.1. Those CSS 2.1
> >> > tests were correct for 2.1. And will remain correct unless we do
> >> > some major reworking of the spec in errata which I seriously doubt.
> >> >
> >> > It may be a good idea to put a note in the CSS 2.1 test cases that
> >> > they have be superseded by new tests. Maybe even link to the new
> >> > tests.
> >>
> >> I'm not sure what goal you're pursuing here, but mine, and I hope the
> >> CSSWG's as well, is improving interoperability. Your suggestion hurts
> >> that goal, so I find it objectionable.
> >>
> >
> > I understand your goal but that isn't the goal of the test suites. The test
> suites are created to test specifications not interoperability or browsers
> compliance. The interop and compliance is only a bonus of the test suites and
> is not a goal of the test suites.
> 
> No it is not. The goal of the test suites is to improve interop.  

Interop is the result of running test cases and comparing the results. I haven't even got to that stage yet I am still talking about making sure the test case actually has some backing from the specification. This is my main point. Any philosophical argument you would like to make about what is and what isn't a test suite let's take that offline and debate it there, not here in this thread. This thread is supposed to be about, "Is the test case correct per what is publicly available in the specification the test case help link points me to?".

> To help
> encourage this, tests are required for various W3C publishing milestones, but
> that is a secondary and less important part of their purpose.  Attempting to
> place W3C publishing requirements over helping interop is a complete
> inversion of priorities which is not acceptable.
> 
> When necessary, we can obtain exceptions to W3C publishing requirements;
> we can't obtain exceptions to interoperability.  Please don't suggest things
> that stand a good chance of hurting interop; making tests require invalid
> behavior is one of those things.
> 
> > This goal is important when you need to verify that specifications are ready
> to move to REC or provide the correct details for errata. Changing these 2.1
> tests to match CSS-Syntax makes sections of CSS 2.1 unverified by tests.
> >
> > The test suites are checkpoints. They are locked to a particular spec
> version/level. If we were rewriting the @charset section of CSS 2.1 I would
> agree with the changes. However, we are not, we are working on CSS-Syntax
> and those tests need to be in CSS-Syntax and the CSS 2.1 tests need to be
> left alone as much as possible.
> >
> > As a brief history note the CSS 1.0 test cases have not been deleted they
> still stand as they were but there are many that have been moved to CSS 2.1
> and superseded by newer CSS 2.1 tests. The only unfortunate this is we have
> not gone back and put notes in those CSS 1.0 tests. Here we have a chance to
> make things better for CSS 2.1 by adding in a note and link to the newer test.
> 
> "Making things better for CSS2.1" is a non-goal, except insofar as some
> sections of 2.1 haven't yet been superceded, and so 2.1 is still the definitive
> spec for those features.
> 

I was hopeful we would be making this better for the future of the test suite as a whole. There are multiple test suites we manage each has its definitions, CSS 1.0 test suite, CSS 2.1 test suite, even the individual test suites of each new spec we create have their own test suites and definitions within their respective specs. As an implementer (and I mean as an implementer of an entirely new product) I am free to implement and comply to which ever level of CSS I want. If I need to verify my compliance I need a test suite to do it and that test suite should be locked to the definitions within the spec.

--
Thanks,
Arron Eicholz

Received on Thursday, 12 February 2015 15:58:45 UTC