Re: delimiters between tokens in turtle

On Thu, 2012-02-23 at 15:47 -0500, Alex Hall wrote:
> On Thu, Feb 23, 2012 at 12:01 PM, Henry Story
> <henry.story@bblfish.net> wrote:
>         I can't quite work out what the delimiters between tokens are.
> 
> 
> Well, that's because there are no delimiters between tokens, per se.
> You can always use whitespace to separate tokens that would otherwise
> be parsed as a single token, but this isn't necessary if there's only
> one way to parse a given character sequence.
>  
>         
>         The following seems to be correct N3 (cwm parses it)
>         
>          @prefix : <>.
>          :me</knows>:her,:him.
> 
> 
> Yes, this is correct N3 (and Turtle). Without getting into the gory
> details of parsing theory, what the parser is doing is:
> 
> 
> 1. Starting with the first character in the input sequence, determine
> what type of token it's looking at

If only this were always possible...

> , by looking ahead as many characters as necessary to eliminate any
> alternatives.

... without this

> 2. Once it determines the token type, it then goes back to the
> beginning of the input and advances character by character

Thankfully since the number of cases where it isn't in Turtle is small
and they are compatible (e.g. true and false clashing with prefixed
names) you don't actually have to do this.

-dr

Received on Saturday, 25 February 2012 21:23:02 UTC