Re: Consolidating css-wg and web-platform-tests repositories (Was: test suite meta data)

On 2013-08-06 14:39, Linss, Peter wrote:
> On Aug 6, 2013, at 12:40 PM, James Graham wrote:
> 
>> On 2013-08-06 11:39, Peter Linss wrote:
>>> On Aug 6, 2013, at 11:14 AM, James Graham wrote:
>> 
>>>> Where does the requirement to have the full suite in multiple 
>>>> formats come from? It seems unlikely that the CSS layer in browsers 
>>>> would depend on the parser that was originally used. Do you have 
>>>> examples of tests that found bugs when run in XML but not in HTML?
>>> It's not so much to test a browser's behavior in both input formats,
>>> but to make the suite available for clients that don't support one
>>> format or the other. Clients which we needed to exit CR for CSS2.1 
>>> and
>>> will likely need again for other specs. For example, some of the
>>> implementations are offline XHTML to PDF converters or XHTML-Print
>>> renderers embedded in printers. This is particularly true for
>>> paged-media CSS features that are generally poorly supported in
>>> browsers (but I'd be more than happy if we could rely on browsers to
>>> pass those tests).
>> 
>> So essentially all this complexity is to support non-web use cases? 
>> That seems unfortunate.
> 
> Sorry, but web technologies are used in places outside browsers, that
> doesn't make it "non-web".

Well that's a matter of definitions, but I would consider things like 
consumer electronics device UIs to be "non-web", even if they happen to 
be implemented in a web browser.

> It's also not unfortunate that this
> happens, it's a good thing. Pretending those use cases don't exist is
> unfortunate.

I'm not pretending that those use cases don't exist. But I don't think 
that we should optimise for the non-web cases at the expense of the 
simplicity for the web cases, or even edge cases of the web at expense 
of the more common cases. For example I don't think we should require 
that DOM tests also work for Java implementations of the DOM, and I 
don't think that we should require all tests work in monochrome just so 
that eink-based ebooks can be tested.

> But that's all beside the point, we're not actually going out of our
> way to support "non-web" use cases, we're supporting getting our specs
> to REC.

It's unfortunate that you still see the testing primarily as a tool to 
get specs to move along the Rec. track. I think we should see testing as 
a way of improving interop. and strengthening the platform. After all 
the Process is a means to an end and non an end in itself.

> The fact that it takes implementations other than browsers to
> do that _is_ unfortunate, but it's what we have had to do, and will
> continue to need to do for some time yet. If you can guarantee
> implementations of all of CSS in two browsers, I'd be happy to drop
> these extra requirements. The day we can do that will be a good day.

Perhaps the build system could be dropped for modules that are actually 
implemented in browsers then? I guess that will be the vast majority.

Received on Tuesday, 6 August 2013 22:38:50 UTC