Re: [css3-syntax] Preserved vs. non-preserved tokens.

On Sat, Jan 19, 2013 at 5:06 AM, Simon Sapin <simon.sapin@kozea.fr> wrote:
> Le 19/01/2013 13:54, Simon Sapin a écrit :
>> In §3.5. Tree Construction:
>>
>>> preserved tokens
>>>
>>> The tokens that can appear in the tree are: ident, hash, string, url,
>>> delim, number, percentage, dimension, unicode-range, whitespace,
>>> comment, cdo, cdc, colon, semicolon, at-keyword, ], ), and }.
>>>
>>> Note: This means that the following tokens emitted by the tokenizer
>>> stage will not appear in the stylesheet object: function, bad-string,
>>> bad-url, [, (, {.
>>
>>
>> However, §3.5.14. Consume a primitive happily returns bad-string or
>> bad-url tokens. They should be parse errors instead.
>>
>> What about ], ), and }? Shouldn’t they be parse errors too?
>
>
> By this I mean that maybe these tokens should be non-preserved, and "Consume
> a primitive" should trigger a parse error when encountering them.
>
> The same applies with cdo and cdc tokens.
>
>
> These tokens are never valid after tree construction. We could either have
> tree construction take care of them (make them non-preserved) or leave that
> to more specific parsers (selector, prelude of a given at-rule, value of a
> given property, etc.)
>
> Either way, I don’t see a reason for treating cdo or ] differently from
> bad-string.

I'm not sure yet.  I've added an issue below the relevant paragraph to
track it.  It would complexify the parser a bit for something that's
just going to get caught by the validity checker anyway.  I may just
move bad-string and bad-url to the "preserved tokens" list.

~TJ

Received on Wednesday, 30 January 2013 05:02:27 UTC