Re: central problem with test suite

On 03/12/2017 04:48 PM, Holger Knublauch wrote:
> The test suit document is work in progress and I have basically just started
> to take a deeper look. I welcome any help on this and really don't want to
> "own" this document.
>
> On 12/03/2017 6:02, Peter F. Patel-Schneider wrote:
>> It's going to be hard.  It's not possible to just remove the parts of the
>> validation report that can vary because some of these parts have conditions on
>> them.  For example, removing type and subclass triples will prevent checking
>> the SHACL instance requirements.
>
> Ok, the fact that reports allow for instances of subclasses of
> sh:ValidationReport and sh:ValidationResult indeed requires an extra
> pre-processing step. I have now added this step, normalizing these to their
> direct rdf:type.
>
>>   There is also the problem that there are
>> different RDF literals with the same value.
>
> Why is this a problem? I believe RDF ismorphism relies on term equality:
>
> https://www.w3.org/TR/rdf11-concepts/#graph-isomorphism
> https://www.w3.org/TR/rdf11-concepts/#dfn-literal-term-equality

Just following the second link here shows that RDF term equality looks at
the syntactic form of RDF literals, not their value.  There is even a very
illustrative example provided.

https://www.w3.org/TR/rdf11-concepts/#dfn-literal-term-equality
*******************
Literal term equality: Two literals are term-equal (the same RDF literal) if
and only if the two lexical forms, the two datatype IRIs, and the two
language tags (if any) compare equal, character by character. Thus, two
literals can have the same value without being the same RDF term. For
example:
      "1"^^xs:integer
      "01"^^xs:integer
denote the same value, but are not the same literal RDF terms and are not
term-equal because their lexical form differs.
*******************

>>   Probably the biggest problem is
>> that the number of values for sh:result can vary between SHACL Core
>> implementations for the same validation.
>
> This is not the intention of the spec. The spec states that each validator
> must have a mode in which it always produces all results.
>
> SHACL-compliant processors /must/ be capable of returning a validation report
> with all required validation results
> <http://w3c.github.io/data-shapes/shacl/#dfn-validation-results>described in
> this specification.

Consider the validating the data graph
  ex:i ex:p ex:j ; ex:q ex:j .
  ex:j ex:p ex:j .
against the shapes graph
  ex:s1 rdf:type sh:PropertyShape ;
    sh:targetNode ex:i ;
    sh:property [ sh:path ex:p ; sh:property ex:s2 ] ;
    sh:property [ sh:path ex:q ; sh:property ex:s2 ] .
  ex:s2 sh:path ex:p ; sh:class ex:C .
It is reasonable and acceptable to have one top-level validation result
here.  It seems to me that there is an argument that it is also reasonable
and acceptable to have two top-level validation results here.

> Holger

peter


>> On 03/11/2017 09:30 AM, Irene Polikoff wrote:
>>> Do you have any suggestions?
>>>
>>>> On Mar 11, 2017, at 7:56 AM, Peter F. Patel-Schneider
<pfpschneider@gmail.com> wrote:
>>>>
>>>> The testing process is not adequate to check for compliance of validation
>>>> reports.  All that it takes to see this is to notice that there are multiple
>>>> compliant non-isomorphic validation reports for when the data graph conforms
>>>> to the shapes graph.  Something more flexible than RDF graph isomorphism is
>>>> needed as the central operation of determining whether the actual validation
>>>> report is complaint with the requirements in the SHACL document.
>>>>
>>>> Peter F. Patel-Schneider
>>>> Nuance Communications
>>>>
>>>>
>>>> On 03/08/2017 11:52 PM, Holger Knublauch wrote:
>>>>> I made some progress defining the test suite.
>>>>>
>>>>>    http://w3c.github.io/data-shapes/data-shapes-test-suite/
>>>>>
>>>>> Please take a look esp at the lower sections on how the comparison of
expected
>>>>> and actual results happens.
>>>>>
>>>>> This document was started by Jose who has since left the WG and since I have
>>>>> no idea whether this still reflects his thinking I have inserted my name to
>>>>> the list of editors. This does not mean that I am particularly keen on
>>>>> "owning" this work or this document. It may not look too good if the primary
>>>>> editor and implementer also defines the test cases. If anyone more neutral
>>>>> wants to step up owning this task, I'd be more than happy to take a back
seat.
>>>>> Of course I will submit and edit tests though
>>>>>
>>>>> If this process looks reasonable, I suggest we start working on the actual
>>>>> tests "soon", e.g. after reaching CR.
>>>>>
>>>>> Holger

Received on Monday, 13 March 2017 11:22:54 UTC