Re: Testing CSS tokenization

I'm not sure why you would need to see the lexer tokens to test "Parse a list of component values" since the only exit condition is an EOF. Seems straightforward to return a known list of component values must match for all whitespace delimiters except EOF. 

FWIW, I think exposing lexer tokens to web content is not desirable.

--Jet

----- Original Message -----
From: "Simon Sapin" <simon.sapin@exyr.org>
To: "public-css-testsuite@w3.org mailing list" <public-css-testsuite@w3.org>
Cc: "Tab Atkins Jr." <jackalmage@gmail.com>
Sent: Monday, June 17, 2013 1:40:02 PM
Subject: Testing CSS tokenization

Hi,

So, I’ve been implementing the new CSS Syntax draft:

http://dev.w3.org/csswg/css-syntax/
https://github.com/SimonSapin/tinycss2

And of course, I’m writing tests. This time around, I’m writing them in 
JSON in the hope to share them in multiple implementations. (I’m working 
on at least two of them myself.)

https://github.com/SimonSapin/tinycss2/tree/master/tinycss2/tests
(This is very much "in progress", but I hope you get the idea.)


In order to test tokenization, I really want to look at the output of 
the tokenizer. (Actually the output of "Parse a list of component 
values", which is close enough.) Unfortunately, the tokenizer is not 
exposed on the web. I can not take these tests and run them in any 
browsers; this would require writing custom code against the browser’s 
source code. I understand this is not acceptable for a W3C spec’s test 
suite, but anything else has to be much higher level and I don’t find it 
very satisfying.

For example, many corner cases of the tokenizer are not tested AFAICT by 
the 2.1:

http://hg.csswg.org/test/file/63d1b51beff2/approved/css2.1/src/syntax


What do you think? What is the way to test a spec such as CSS Syntax 
whose product is not directly exposed to the web platform?

Cheers,
-- 
Simon Sapin

Received on Tuesday, 18 June 2013 00:28:08 UTC