- From: Kang-Hao (Kenny) Lu <kanghaol@oupeng.com>
- Date: Fri, 05 Apr 2013 22:01:02 +0800
- To: Zack Weinberg <zackw@panix.com>
- CC: Simon Sapin <simon.sapin@exyr.org>, "Tab Atkins Jr." <jackalmage@gmail.com>, www-style list <www-style@w3.org>
(13/04/05 21:27), Zack Weinberg wrote:
> I tend to think that the tokenizer should be considered mostly frozen,
> but I don't see any harm in adding new "punctuators" (to borrow a term
> from C) as necessary.
Our favorite scripting language, ECMAScript, use this term too :)
> An alternative (a la Smalltalk) would be to declare that any
> two-character sequence of DELIM characters -- that is, ASCII
> punctuation excluding ,;:()[]{} -- is a single token. That
> would be future-proof, but we'd have to audit the existing grammar
> carefully to make sure it doesn't do anything it shouldn't.
You'll have to exclude * from DELIM then. Because there are *|* and
*.class. Otherwise, it sounds good. Are there other situations like this?
Cheers,
Kenny
--
Web Specialist, Opera Sphinx Game Force, Oupeng Browser, Beijing
Try Oupeng: http://www.oupeng.com/
Received on Friday, 5 April 2013 14:01:37 UTC