Re: [css3-syntax] Preserved vs. non-preserved tokens.

On Tue, Jan 29, 2013 at 9:01 PM, Tab Atkins Jr. <jackalmage@gmail.com> wrote:
> On Sat, Jan 19, 2013 at 5:06 AM, Simon Sapin <simon.sapin@kozea.fr> wrote:
>> Le 19/01/2013 13:54, Simon Sapin a écrit :
>>> In §3.5. Tree Construction:
>>>
>>>> preserved tokens
>>>>
>>>> The tokens that can appear in the tree are: ident, hash, string, url,
>>>> delim, number, percentage, dimension, unicode-range, whitespace,
>>>> comment, cdo, cdc, colon, semicolon, at-keyword, ], ), and }.
>>>>
>>>> Note: This means that the following tokens emitted by the tokenizer
>>>> stage will not appear in the stylesheet object: function, bad-string,
>>>> bad-url, [, (, {.
>>>
>>>
>>> However, §3.5.14. Consume a primitive happily returns bad-string or
>>> bad-url tokens. They should be parse errors instead.
>>>
>>> What about ], ), and }? Shouldn’t they be parse errors too?
>>
>>
>> By this I mean that maybe these tokens should be non-preserved, and "Consume
>> a primitive" should trigger a parse error when encountering them.
>>
>> The same applies with cdo and cdc tokens.
>>
>>
>> These tokens are never valid after tree construction. We could either have
>> tree construction take care of them (make them non-preserved) or leave that
>> to more specific parsers (selector, prelude of a given at-rule, value of a
>> given property, etc.)
>>
>> Either way, I don’t see a reason for treating cdo or ] differently from
>> bad-string.
>
> I'm not sure yet.  I've added an issue below the relevant paragraph to
> track it.  It would complexify the parser a bit for something that's
> just going to get caught by the validity checker anyway.  I may just
> move bad-string and bad-url to the "preserved tokens" list.

Never mind, I've made "consume a primitive" sometimes trigger a parse
error and return nothing, and adjusted all the states to account for
that.  The bad tokens are ), ], }, cdo, cdc, bad-url, and bad-string.

~TJ

Received on Thursday, 31 January 2013 21:29:07 UTC