Re: Shapes Constraint Language (SHACL) Working Draft of 2017-02-02

> Peter,
>
> I believe most of the comments from
> https://lists.w3.org/Archives/Public/public-rdf-shapes/2016Aug/0005.html
> e-mail have been resolved and responded to in some of the previous
> communications. Many have been recorded as issues, but a couple of the
> comments may not have been responded to yet. Since this e-mail contained
> many different comments which were discussed and addressed individually and
> in different communication threads over the last 6 months, I think it would
> be a complex undertaking for an interested third party to comb through and
> track each one.  For this reason, I went through the text of the original
> e-mail and, for each individual comment, provided below an in-line response
> that corresponds to the current status of the document.
>
> In four cases, it was not clear to me what issue was being hinted at and
> whether it was still relevant. I have indicated this in the response to them
> with “please clarify”.
>
> To facilitate better clarity and transparency in tracking individual
> comments to their resolution, it would be appreciated if you responded to
> each clarification request (assuming the original comment is still relevant)
> in a separate e-mail titled with the topic of the comment e.g., "status of
> $shapesGraph is unclear”.

Several of the issues brought up in this reply have resulted in their own
email threads.   Several others have already been satisfactorily handled.
It is useful to keep the few remaining ones associated with this interaction
so I will not be creating new threads for continuing issues at this time.

> Thanks,
>
> Irene Polikoff

> > Here are a few of the problems with the current public working draft I found
> > during a quick scan of it.
> >
> > * pre-binding
> >
> > SPARQL does not evaluate variables that occur in basic graph patterns.  This
> > means that the definition of pre-binding has unusual behaviour.  For
> > example, the normative SPARQL definition of sh:class will return validation
> > results for every pair of nodes in the graph such that there is an
> > rdf:type/rdfs:subClass* path from the first to the second.
> >
> > This problem affects many parts of the definition of SHACL.  It means that
> > the normative definition of many SHACL constructs is counter to intuitions.
> > This problem is not ameliorated by the caution box in Appendix B.
>
> RESPONSE: The definition of pre-binding has been significantly changed since
> the question was asked. ISSUE-222 has been opened to track your more recent
> comments on this topic. Note that the normative definitions of SHACL Core no
> longer rely on SPARQL.

This is the first substantive response to this part of the message.

It is true that the definition of pre-binding has substantially changed.
However, the new definition is also unsuitable for use in SHACL.  As there
are subsequent comments on this issue this problem does not need further
tracking from this message.

> > * syntax of SPARQL variables
> >
> > SPARQL treats $ and ? as equivalent so $PATH and ?PATH both refer to the
> > PATH variable.  SHACL uses $ as a special marker and includes $ and ? as
> > part of the variable.
> >
> > Would ?PATH be substituted as $PATH is?  If a SPARQL query for a SHACL
> > constraint only used ?this would the variable this be pre-bound?

> RESPONSE: Yes both are treated uniformly, and in 6.2.3.1 the spec only
>  refers to <code>PATH</code> by its variable name.

This problem has been the subject of a previous substantive reply and has
now been satisfactorily resolved.

> > * pre-binding optional?
> >
> > "SPARQL variables using the $ marker represent external values that must be
> > pre-bound or substituted in the SPARQL query before execution."
> > "When SPARQL constraints are executed, the validation engine should pre-bind
> > values for these variables."
> > Are some $-marked variables not necessarily pre-bound, counter to the
> > earlier requirement?

> RESPONSE: All $-marked variables must be prebound with the exception of
> $PATH which is substituted - as explicitly stated in section 1.6 (see
> response below).

This problem has been the subject of a previous substantive reply and has
now been satisfactorily resolved.

> > * $PATH vs other $-prefixed variables
> >
> > The variable PATH is treated specially in SHACL.  However, the general
> > description of $ does not specially call out PATH:
> > "SPARQL variables using the $ marker represent external values that must be
> > pre-bound or substituted in the SPARQL query before execution.”

> RESPONSE: Section 1.6 now states:
>
> SPARQL variables using the $ marker represent external bindings that are
> pre-bound or, in the case of $PATH, substituted in the SPARQL query before
> execution. Also note that section 1.6 is not normative.

This problem has been the subject of a previous substantive reply and has
now been satisfactorily resolved.

> > * $value
> >
> > $value is used in many ASK queries.  However the definition of ASK
> > validators does not appear to pre-bind value.

> RESPONSE: Section 6.3 has just been updated to clarify that $value is
> pre-bound to the value node.

This problem has been the subject of a previous substantive reply and is
satisfactorily resolved.

> > * aggregation
> >
> > The prohibition "Furthermore, any query that uses the variable $this in an
> > aggregation is invalid." is vague.  It appears to disallow the use of $this
> > in any part of the SPARQL 1.1 aggregation machinery, as the pointer in the
> > sentence is to Section 11 of the SPARQL specification.  This would rule out
> > all of the examples of aggregation in the SHACL document.

> RESPONSE: Aggregation is no longer mentioned anywhere in the SHACL document

This problem was the subject of several emails and gave rise to ISSUE-208.
There was an email suggesting how to close the issue but no response
indicating that the problem had been resolved and the issue closed.  The
next step should be a message from the working group in the thread for this
problem providing a description of how the resolution of ISSUE-208 addresses
this problem.

> > * ASK validators syntax
> >
> > The syntax for ASK queries in SPARQL 1.1 is
> >   "ASK" DatasetClause* WhereClause SolutionModifier
> > The syntax for WhereClause is
> >   'WHERE'? GroupGraphPattern
> > The syntax for EXISTS constructs SPARQL 1.1 is
> >   'EXISTS' GroupGraphPattern
> > Stripping the ASK from the beginning of an ASK query does not generally end
> > up with a GroupGraphPattern that can be used as the argument for EXISTS.
> >
> > It appears that the values of sh:ask are never used as ASK queries by SHACL
> > processors.  Why then are these of the form of ASK queries?

> RESPONSE: This section has been completely changed since you wrote your
> comment. Please revisit if your concern still applies.

This problem has been the subject of a previous substantive reply.  That
reply was not satisfactory, but I had not pursued the issue as I expected
further changes in this area.  These changes have happened and the current
version resolves this problem.   However there is newly-introduced problem
related to ASK queries.  I have sent in a separate message on this new
problem.

> > * different levels of SHACL implementation
> >
> > There are several different kinds of SHACL implementations that are hinted
> > at in the document.
> >
> > "SHACL implementations may, but are not required to, support entailment
> > regimes."
> > "Access to the shapes graph is not a requirement for supporting the SHACL
> > Core language."
> > "This sections [sic] defines the built-in SHACL constraint components that
> > MUST be supported by all SHACL validation engines."
> > "Not all SHACL validation engines need to support this variable."
> > "The same support policies as for $shapesGraph apply for this variable."
> > "SPARQL engines with full SHACL support can install a new SPARQL function
> > based on the SPARQL 1.1 Extensible Value Testing mechanism."
> > "SHACL validation engines are not required to support any entailment regimes."
> > "SHACL implementations with full support of the SHACL SPARQL extension
> > mechanism must implement a function sh:hasShape, ...."
> > "A SHACL validation engine MUST implement all constructs in the Core of SHACL
> > (Sections 2, 3, 4). A SHACL engine MAY not implement the other parts of
> > SHACL."
> > "Implementations that cover only the the SHACL Core features are not
> > required to implement these mechanisms or the sh:hasShape function."
> > "SHACL validation engines MAY pre-bind the variable $shapesGraph to provide
> > access to the shapes graph."
> > "A SHACL validation engine MAY use such suggestions to determine which shapes
> > graph to use for validating a data graph."
> > "A SHACL validation engine MAY take this information into account to
> > determine which shapes graph to use for validating a data graph that uses
> > that ontology or vocabulary."
> >  
> > There needs to be a section that explicitly defines the different levels of
> > implementation.

> RESPONSE: Majority of the quoted sentences are no longer present in the
> document. The document explicitly talks about two types of processors
> (SHACL CORE and SHACL SPARQL) and describes criteria for their
> conformance.

There is an email thread on this problem that lead to some changes to the
document.  However, there are still problems with levels of SHACL
implementation.  First, some of the governing senteces are in non-normmative
parts of the document, such as at the end of Section 1.6.  Second, there is
still wording indicating that there can be differing levels of support for
SHACL langauge features beyond just SHACL Core and SHACL-SPARQL related to
how shapesGraph is handled.  I have pushed back on the changes to SHACL in
this area.  The thread has messages with title "Different levels of SHACL"
and should be used to finish interaction on this problem.

> > * order of processing for filters
> >
> > The discussion of how filters are processed appears to be contradictory.
> > First there is:
> > "SHACL validation engines MAY alter the order of the depicted steps as long
> > as the returned validation results are correct."
> > Later there is:
> > "Filter shapes MUST be evaluated before validating the associated shapes or
> > constraints.”

> RESPONSE: Filter shapes are no longer supported

There was back-and-forth on this problem, but no final resolution that I can
see.  However, as you say, filters are no longer in SHACL, so this problem
has been removed.

> > * $shapesGraph
> >
> > The status of $shapesGraph is unclear:
> > "SPARQL variables using the $ marker represent external values that must be
> > pre-bound or substituted in the SPARQL query before execution."
> > "SHACL validation engines MAY pre-bind the variable $shapesGraph to provide
> > access to the shapes graph.”

> RESPONSE: Please clarify the issue. What is unclear?

The first sentence says that $-marked variables must be pre-bound or
substituted.  The second contracts that by using may.  The wording here has
changed somewhat, but it is still a tension between the two wordings.

> > * circular filters
> >
> > What happens if a shape is one of its own filters?
> RESPONSE: Filter shapes are no longer supported

There was a substantive response on this issue.  As filters are no longer
part of SHACL the previous substantive response is no longer correct, but as
filters are no longer part of SHACL the problem has been removed.

> > * EXISTS and blank nodes
> >
> > The definition of ASK binds the value variable and then uses it inside an
> > EXISTS.  The definition of SPARQL provides a counter-intuitive result if
> > this variable is bound to a blank node, resulting in, for example, a
> > sh:class constraint with class ex:C returning no violation for _:d in any
> > data graph containing the triple
> >   ex:c rdf:type ex:C .
> >
> RESPONSE: The spec no longer relies on SPARQL EXISTS.

This is the first substantive response to this message on this problem.  As
EXISTS no longer occurs in the document the problems with EXISTS are less
important and the particular one here has been removed.  However, EXISTS
still produces a distinct problem for SHACL.  I have sent in a new comment on
this remaining problem.

> > * union operations on data graphs and shapes graphs
> >
> > It is unclear just what the data graph and the shapes graph are.  There is
> > wording that both of these cannot be changed.  However, there is also
> > wording that various kinds of union operations are to be performed on shapes
> > and data graphs.
> RESPONSE: Please clarify the issue.

There was an email thread on this issue.  I have closed out that email thread
saying that this problem is no longer active.

> > * It is unclear what is meant by:  "The variable $targetNode is assumed to
> >   be pre-bound to the given value of sh:targetNode."  Is this something that
> >   SHACL implementations have to do?  There are several occurences of this
> >   kind of wording.
> RESPONSE: Please clarify the issue. What is unclear?

It is unclear as to what force comes from the use of "assumed".  Does the
"assume" mean that "because implementation have to pre-bind certain
variables therefore it is the case that ..."  or just "if it happens that
..."?  There were a couple of messages that included comments on this
problem.  The last one was from me, stating that I did not accept the
rationale given for keeping the word.  The next step should be a respose
from the working group indicating whether the wording will be kept or
modified.

> > * MAY is used in 1.5 but defined in 1.6
> RESPONSE: May is defined in section 1.3 as follows:
>
> The key words may, must, must not, and should are to be interpreted as
described in [RFC2119].
> > * "A SHACL engine MAY not implement the other parts of SHACL." reads as if
> >   no SHACL engine is allowed to implement any non-core part of SHACL.
> RESPONSE: Quoted text is not in the document

Both of these were addressed in previous emails.  I had stated that I was
satisfied in both cases.

> > * "The data graph SHOULD include all the ontology axioms related to the data
> >   and especially all the rdfs:subClassOf triples in order for SHACL to
> >   correctly identify class targets and validate Core SHACL constraints."
> >   Data graphs are just graphs.  How thus can SHOULD be applied to them?
> >
> > * "A SHACL validation engine MAY use such suggestions to determine which
> >   shapes graph to use for validating a data graph."  Can this be done even
> >   when an explicit shapes graph is provided to the engine?
> RESPONSE: Quoted text is not in the document

This was addressed in previous emails.  I had stated that the new wording
was better but still had some problems.

> > * "The same mechanism applies for ontologies or vocabularies included in the
> >   shapes graph. The ontology or the vocabulary IRI can point to one or more
> >   shapes graphs with the predicate sh:shapesGraph. A SHACL validation engine
> >   MAY take this information into account to determine which shapes graph to
> >   use for validating a data graph that uses that ontology or vocabulary."
> >   If there already is a shapes graph in play, why is there any need for a
> >   different shapes graph to be used?
> RESPONSE: Quoted text is not the document

This was addressed in previous emails.  I had stated that I felt that there
were problems with the revised wording.

> > * "a deep copy of sh:path as its sh:path"  What is "deep copy" in this
> >   context?
> RESPONSE: ‘deep copy’ is not mentioned in the document

There was an email thread on this problem resulting in multiple changes
to the definitions.   That thread did not lead to a resolution.  As copying
is no longer part of SHACL, this problem has been removed.

> > * "A filter is a shape in a shapes graph that can be used to limit the nodes
> >   that are validated against a given constraint or shape."   Are there some
> >   filters that cannot be used in this way?  Which ones?
> RESPONSE: Filter shapes are no longer supported

There was a response on this problem.  As filters are no longer part of
SHACL this problem has been removed.

> > * "The following table enumerates variables that have special meaning in
> >   SPARQL constraints. When SPARQL constraints are executed, the validation
> >   engine should pre-bind values for these variables."  However, many other
> >   variables also need to be pre-bound, such as the variables corresponding
> >   to parameters.
> RESPONSE: Please clarify the issue.

There was a response on this problem.  A table that enumerates some category
(here variables that have special meaning to SHACL) generally is taken to
mention all elements of the category.  This is not the case here.  There are
more variables that have special meaning to SHACL than just this,
shapesGraph, and currentShape, particularly the variables derived from
parameters.

Peter F. Patel-Schneider
Nuance Communications

Received on Tuesday, 7 February 2017 23:19:04 UTC