Re: central problem with test suite

The test suit document is work in progress and I have basically just 
started to take a deeper look. I welcome any help on this and really 
don't want to "own" this document.

On 12/03/2017 6:02, Peter F. Patel-Schneider wrote:
> It's going to be hard.  It's not possible to just remove the parts of the
> validation report that can vary because some of these parts have conditions on
> them.  For example, removing type and subclass triples will prevent checking
> the SHACL instance requirements.

Ok, the fact that reports allow for instances of subclasses of 
sh:ValidationReport and sh:ValidationResult indeed requires an extra 
pre-processing step. I have now added this step, normalizing these to 
their direct rdf:type.

>    There is also the problem that there are
> different RDF literals with the same value.

Why is this a problem? I believe RDF ismorphism relies on term equality:

https://www.w3.org/TR/rdf11-concepts/#graph-isomorphism
https://www.w3.org/TR/rdf11-concepts/#dfn-literal-term-equality

>    Probably the biggest problem is
> that the number of values for sh:result can vary between SHACL Core
> implementations for the same validation.

This is not the intention of the spec. The spec states that each 
validator must have a mode in which it always produces all results.

SHACL-compliant processors/must/be capable of returning a validation 
report with all requiredvalidation results 
<http://w3c.github.io/data-shapes/shacl/#dfn-validation-results>described 
in this specification.

Holger


>
> peter
>
>
> On 03/11/2017 09:30 AM, Irene Polikoff wrote:
>> Do you have any suggestions?
>>
>>> On Mar 11, 2017, at 7:56 AM, Peter F. Patel-Schneider <pfpschneider@gmail.com> wrote:
>>>
>>> The testing process is not adequate to check for compliance of validation
>>> reports.  All that it takes to see this is to notice that there are multiple
>>> compliant non-isomorphic validation reports for when the data graph conforms
>>> to the shapes graph.  Something more flexible than RDF graph isomorphism is
>>> needed as the central operation of determining whether the actual validation
>>> report is complaint with the requirements in the SHACL document.
>>>
>>> Peter F. Patel-Schneider
>>> Nuance Communications
>>>
>>>
>>> On 03/08/2017 11:52 PM, Holger Knublauch wrote:
>>>> I made some progress defining the test suite.
>>>>
>>>>     http://w3c.github.io/data-shapes/data-shapes-test-suite/
>>>>
>>>> Please take a look esp at the lower sections on how the comparison of expected
>>>> and actual results happens.
>>>>
>>>> This document was started by Jose who has since left the WG and since I have
>>>> no idea whether this still reflects his thinking I have inserted my name to
>>>> the list of editors. This does not mean that I am particularly keen on
>>>> "owning" this work or this document. It may not look too good if the primary
>>>> editor and implementer also defines the test cases. If anyone more neutral
>>>> wants to step up owning this task, I'd be more than happy to take a back seat.
>>>> Of course I will submit and edit tests though
>>>>
>>>> If this process looks reasonable, I suggest we start working on the actual
>>>> tests "soon", e.g. after reaching CR.
>>>>
>>>> Holger
>>>>
>>>>