RE: ACTION 614-12: Smart Quotes

I agree that tokenization should be possible before evaluating the query prolog.  I'm also concerned that we simply don't have the mechanisms in our grammar to specify the kind of construct that Mike Kay wants, and no precedent for doing anything like this in the language.  That seems like a risky thing to do at this point in our cycle.


Mike Kay wrote:
> Currently the only way of getting “${“ into the string is to write ${“${“} which is fairly horrible.

> If we allowed any character, we would not need the separate construct ~~!…!~~.

Good points. 

And I think it's worse if with the ~~$ $-- convention, you can get expressions like this:

~~$ ${“${“} $--

I prefer that our start delimiter and end delimiter not contain $, which will be used in every variable reference in the XQuery expressions.  I also prefer that the start and end delimiter be different so it's obvious which is which, and I don't really see why we need two different kinds of smart quotes. I also prefer that all four of these delimiters have some visual similarity that makes it obvious that they belong together.

Suppose we did this:

ExtStringConstructor ::= "~~|" ExtStringText "|~~"
ExtStringText := ((Char* - "--{") | "--{" Expr "}--")*

That makes our earlier example look like this:

~~| ~~{ “${“ }~~ |--

Here is a longer example, modified from Liam's proposal:

    declare variable $json := ~~| {"menu": {
      "id": "file",
      "value": "File",
      "popup": {
        "menuitem": [
          {"value": "New", "onclick": "CreateNewDoc()"},
          {"value": "Open", "onclick": "OpenDoc()"},
          {"value": "Close", "onclick": "CloseDoc()"},
          {"callback": null },
          {"session-id":  --{ get-session-id() }-- }
        ]
      }
    }} |~~ ;

Since you can embed any arbitrary expression, you can effectively do an escape by assigning a sequence to a string variable.

declare variable $double-squiggle := "~~";
declare variable $pipe := "|";

~~|  --{ $double-squiggle, $pipe  }--  |~~

Do we need an escape mechanism beyond that?

Jonathan

P.S. The minutes Josh pointed to aren't terribly clear to me, but they do seem to imply a declaration like what Mike Kay wants.

> Modified proposal:
>
> - use ~~~ for starting, ending
> - only ${ } is special, it indicates an XQuery expression
> - result is a string, which must be parsed using parse-json() to
>    create JSON
> - can use option declarations to change start/end parameters and
>    characters for start/end of an expression
>
> declare option foreign-region-start "";
> declare option foreign-region-end "";
> declare option foreign-region-expression-start "";
>
> declare variable $x :=
> ~~~
>    {
>      ${$x} : ${3*$y}
>    }
> ~~~
>
> Action A-554-06: Liam to write a modified proposal for foreign
> regions based on Shenzhen f2f minutes.



________________________________________
From: Josh Spiegel [josh.spiegel@oracle.com]
Sent: Thursday, September 10, 2015 12:48 PM
To: Michael Kay
Cc: Robie, Jonathan; Public Joint XSLT XQuery XPath
Subject: Re: ACTION 614-12: Smart Quotes

> I would prefer to allow users to choose any character they like in place of the “$”

This was debated and decided at Face-To-Face Meeting #554:
https://lists.w3.org/Archives/Member/w3c-xsl-query/2013Nov/0074.html

Some implementations may separate lexical processing [1] from parsing so the tokenization of a query shouldn’t, for example, depend on evaluating a declaration in the prolog.  There were also concerns that third-party parser generators may not be able to support this kind of thing.

Thanks,
Josh

[1] http://www.w3.org/TR/xquery-xpath-parsing/

> On Sep 10, 2015, at 9:01 AM, Michael Kay <mike@saxonica.com> wrote:
>
> Some comments on this proposal.
>
> [201] ExtStringConstructor ::= "~~$" . ExtStringText "$~~"
> [203] ExtStringText := ((Char* - "${") | "${" Expr "}”)*
>
> In the first production, what does the “.” signify? Is it just a typo?
>
> In the second production, there seems to be nothing to exclude “$~~” from the body of the string.
>
> As I mentioned before in discussion, I would prefer to allow users to choose any character they like in place of the “$”. We might find that the grammar notation is stretched to specify this, but I don’t see any serious difficulty in parsing it. There are things users might want to generate in which “${“ occurs frequently. Currently the only way of getting “${“ into the string is to write ${“${“} which is fairly horrible.
>
> If we allowed any character, we would not need the separate construct ~~!…!~~.
>
> It’s not stated where this construct can be used. The obvious answer is “anywhere you can use a string literal”. But there are places in XQuery where we have things that look like string literals, but are not. Notably DirAttributeValue. There are also places we use StringLiteral where we might not want to allow this syntax, for example within a VersionDecl; and I believe that URILiteral is defined syntactically to be a StringLiteral, but has some different semantic constraints.
>
> I assume this is XQuery only, not XPath?
>
> Does the setting of “boundary space” have any impact?
>
> Michael Kay
> Saxonica
>
>
>
>
>> On 8 Sep 2015, at 15:49, Robie, Jonathan <jonathan.robie@emc.com> wrote:
>>
>> ACTION 614-12 on JRobie to prepare a concrete proposal on smart quotes
>> for 2015-09-08.
>>
>> After putting in time on this proposal, I followed the chain of reference on Liam's action item, and found this proposal.  I like it as is, and suggest that we simply incorporate it directly into the document.  Note that it looks quite different from the proposal we discussed at the F2F, but I think it is a major improvement of that proposal.
>>
>> https://lists.w3.org/Archives/Member/w3c-xsl-query/2014Jan/0044.html
>>
>> Jonathan
>>
>
>

Received on Thursday, 10 September 2015 19:04:50 UTC