Re: pre-binding as used in SHACL document

The specific problem what I identified in the email that started this thread
has nothing to do with subqueries.  None of the examples that I provided
involve subqueries at all.  What did happen here was that I pointed out that
the definition of pre-binding as of 9 May 2017 produces unsuitable results for
quite a few queries that are in the SHACL document.

At its meeting of 10 May 2017 the working group decided to change the
definition of pre-binding in the SHACL document.  This change appears to
affect the results of pre-binding for many queries that do not contain
subqueries, MINUS, SERVICE, VALUES, or AS.

This is a new definition of pre-binding with significant differences from the
previous definition.  It thus needs substantive internal and external review,
neither of which have happened, if it is to remain part of SHACL.

At-risk features in Candidate Recommendations are not features that can be
modified without a requirement to publish a new Candidate Recommendation.
Instead,  at-risk features "may be *removed* [emphasis added] before
advancement to Proposed Recommendation without a requirement to publish a new
Candidate Recommendation." [World Wide Web Consortium Process Document,
https://www.w3.org/2017/Process-20170301/#candidate-rec]

Peter F. Patel-Schneider
Nuance Communications

PS:  The basis of the definition of pre-binding in the SHACL document is a
proposal *to* the SPARQL EXISTS Community Group not a proposal *by* the SPARQL
EXISTS Community Group.


On 05/12/2017 09:27 AM, Irene Polikoff wrote:
> Peter,
> 
> Let me first clarify what issue we are discussing here. I feel a need to do so because I found your recent e-mails on this topic to be rather obtuse and unhelpful. So much so that it made me wonder whether this was intentional.
> 
> Over the lifespan of the WG and, more specifically, in the last 6-9 months there was a lot of discussions about possible issues with pre-binding in the context of certain query patterns. Many of these discussions were either initiated by you or you have participated in them actively.
> 
> A specific issue we are discussing here has to do with what happens when a subquery binds a variable, but doesn’t return it to the top query. 
> 
> If a subquery does not return such bound variable to the top query, there is a chance that a SPARQL engine may re-use that same variable for something else. This would be an issue for SHACL and, thus, needed a solution. This is the full extent of the issue. It was discussed in many e-mails. It was also discussed at several meetings. It was always discussed in the context of sub or inner queries.
> 
> A solution to this issue was proposed by the SPARQL EXISTS Community Group. This solution involved projection and variable remapping. The solution was then copied from the CG Note into Appendix A of the SHACL document. 
> 
> All along, it was clear that this was addressing a problem with subqueries that do not return pre-bound variables. There was never a need nor a reason to apply this solution in any other context.
> 
> The contextual nature of the solution was, thus, well understood. Its applicability to sub-queries was enforced not only through numerous (granted, non normative) examples throughout the entire specification, but also through tests, which are normative. There is not a single shred of evidence that it was misunderstood by anyone.  There is plenty of evidence that the solution was understood to be for subqueries only - by the WG, by the implementers and by the members of general public which took interest in this topic, including yourself, until you started to interpret it differently a few days ago. 
> 
> After CR was published, review identified that there were still some narrow cases where the proposed solution may not guarantee the intended result. The WG then decided that the cleanest approach was to require that subqueries return all pre-bound variables. With this, the original issue went away and projection and variable remapping was no longer needed. Thus, it was removed from the document. The WG was able to make the change without issuing a second CR because SHACL-SPARQL was indicated to be “at risk” when CR was published. 
> 
> Therefore, I feel it is rather disingenuous to say that some fundamental aspect of SHACL was changed. The only relevant change is that the subqueries that do not return all pre-bound variables are no longer allowed. And this change made it possible to simplify definition of pre-binding.
> 
> Irene
> 
> 
>> On May 12, 2017, at 8:45 AM, Peter F. Patel-Schneider <pfpschneider@gmail.com> wrote:
>>
>> The definition of pre-binding in SHACL is contained in Appendix A of the
>> SHACL document.  Appendix A is very clear that that appendix is the
>> definition of pre-binding.  The definition of pre-binding there in the
>> Candidate Recommendation version of the document at
>> https://www.w3.org/TR/2017/CR-shacl-20170411/ consists only of four
>> definitions.  There is no other description of pre-binding to be found
>> anywhere else in the SHACL document.  Previous post-CR changes to
>> pre-binding resulted the removal from the document of an example that was no
>> longer conformant to the changed definition, emphasizing that this
>> definition is the normative definition for pre-binding in SHACL.
>>
>> The appendix in the Candidate Recommendation version of the document is
>> absolutely clear that the variable remapping operation, PrjMap, is to be
>> applied to all projections.  Removing this aspect of the definition of
>> pre-binding is a major change to pre-binding, changing the results of
>> pre-binding for very many queries, including several that are in the SHACL
>> document.
>>
>> I see that the RDF Data Shapes Working Group decided in its meeting on 10
>> May to significantly change the definition of pre-binding.  This change is
>> labelled as editorial although it is decidedly not.  As far as I can tell
>> this change was first proposed at the meeting itself.  At this meeting the
>> working group also decided to request transition to proposed recommendation.
>> There is no way that the new definition of pre-binding could have received
>> the amount of internal review that is required for a significant change to
>> such an important part of SHACL.  There was no opportunity at all for the
>> new definition of pre-binding to have external review.  This new definition
>> of pre-binding may be suitable.  However, there is no way to tell whether
>> this is the case because of the lack of review.
>>
>> Making such a change to a fundamental aspect of SHACL requires full review,
>> both internal and external.  The working group should not be requestion
>> transition to proposed recommendation without such review.  At the very
>> minimum SHACL needs to go through a second candidate recommendation phase,
>> with a special request for review of the new definition of pre-binding.
>>
>> Peter F. Patel-Schneider
>> Nuance Communications
>>
>>
>> On 05/11/2017 07:52 PM, Irene Polikoff wrote:
>>> Peter,
>>>
>>> Examples, tests and implementations are correct.
>>>
>>> I believe you are referring to the projection and re-mapping definitions in the appendix A and interpreting them broader than intended. These have always been intended to apply only to sub-queries, not to the top level queries. There is no need to do the remapping for the variables in the top level queries.  Implementors did understand it correctly  - as per the tests and examples.
>>>
>>> Further, given that the SHACL WG decided a week ago to always require for the sub-queries to return all pre-bound variables,  there is now no need to stipulate this for the subqueries either. These definitions are no longer needed by SHACL and have been removed from the document. 
>>>
>>> I believe that in the context of the EXIST CG Note, they are still needed and the note may need to further clarify their applicability.
>>>
>>> Regards,
>>>
>>> Irene
>>>
>>>
>>>> On May 9, 2017, at 6:53 AM, Peter F. Patel-Schneider <pfpschneider@gmail.com> wrote:
>>>>
>>>> I took a quick look at the use of pre-binding in the SHACL document, version
>>>> 07 May 2017.  Previously I had been looking at the definition of pre-binding
>>>> and only looking at its use when I found problems.
>>>>
>>>> I found that many uses of pre-binding in the document produce unsuitable or
>>>> unexpected results.  This shows that there has been no effective review of
>>>> the current definition of pre-binding to check whether it is suitable for
>>>> use in SHACL, a very surprising situation for such a central part of
>>>> SHACL-SPARQL.
>>>>
>>>>
>>>> Here are a few of the uses of pre-binding where it produces unsuitable or
>>>> unexpected results.
>>>>
>>>> The very first use of pre-binding in the document produces completely
>>>> unsuitable results.
>>>> SELECT DISTINCT ?this WHERE { BIND ($targetNode AS ?this) }
>>>> with the variable targetNode pre-bound to some RDF term will produce a
>>>> solution sequence containing a single solution.  That solution will have
>>>> the variable this unbound.
>>>>
>>>> The second use of pre-binding also produces completely unsuitable results.
>>>> SELECT DISTINCT ?this WHERE {
>>>>   ?this rdf:type/rdfs:subClassOf* $targetClass .
>>>>   }
>>>> with the variable targetClass pre-bound to some RDF term will produce a
>>>> solution sequence containing solutions binding the variable this to each
>>>> node in the graph that is the subject of a triple in the graph with rdf:type
>>>> as predicate.
>>>>
>>>> The second-last use of pre-binding produces unexpected results.  The
>>>> results of
>>>> SELECT DISTINCT $this ?value WHERE {
>>>>   $this $PATH ?value .
>>>>   FILTER (!isLiteral(?value) || !langMatches(lang(?value), $lang))
>>>>   }
>>>> are independent of the pre-binding of the variable lang.  This query forms
>>>> the basis of one of the SHACL-SPARQL tests, with expected results
>>>> dramatically different from the ones actually produced by the current
>>>> definition of pre-binding.  SHACL-SPARQL implementations thus are not
>>>> implementing pre-binding as currently defined.
>>>>
>>>>
>>>> Peter F. Patel-Schneider
>>>> Nuance Communications
>>>>
>>>
>

Received on Friday, 12 May 2017 19:51:44 UTC