Re: Test cases format

Replied before I saw your answer Jose

On Thu, Apr 2, 2015 at 8:57 AM, Jose Emilio Labra Gayo <jelabra@gmail.com>
wrote:

> On Thu, Apr 2, 2015 at 7:18 AM, Holger Knublauch <holger@topquadrant.com>
> wrote:
>
>>  Ok thanks for the pointer. As far as I understand it, the results so
>> far are only booleans of whether a given node matches a shape. I would hope
>> we extend this with the full result of the validation (once this vocabulary
>> gets agreed upon) so that implementers can verify that their engine returns
>> meaningful results.
>>
>
> I agree with that. I was waiting until we have more consensus on the
> validation results format. Maybe, we should try to resolve that issue
> first.
>

As I said in my reply, I suggest we start we a simple pass/fail approach
until we finalize the reporting format, it could take a long time to
resolve this.


> I also think we need a way to validate a complete graph with all nodes
>> using the built-in node selection properties sh:nodeShape and rdf:type.
>> This would then also include global constraints.
>>
>
>> Regarding the two Syntax tests, I can see why they created them for
>> SPARQL - to test the SPARQL string parsers. However, for SHACL it sounds
>> like we'd mainly need structural tests of the syntax. Currently the draft
>> of the SHACL schema is self-validating, i.e. it is possible to validate
>> SHACL shape definitions using SHACL itself.
>>
> As a result, we may not need the Syntax tests.
>>
>
> Not sure about that. If the syntax is for example RDF, we could at least
> have some tests with bad RDF syntax to check that the SHACL implementation
> handles them appropriately.
>

I think this comes down to the closed shapes req and we'd need test cases
for that first :)
We can use it for bad SPARQL queries / select variables and maybe the
compact syntax later on


> Later on, if we accommodate other syntactic possibilities similar to ShExC
> (or whatever is called), those tests could also be handled.
>
>>
>> Finally, if we only rely on the mf: namespace, how could we add new
>> features if we need them?
>>
>
> In the case of RDF, they used rdft as:
>
> @prefix rdft:   <http://www.w3.org/ns/rdftest#> .
>
> We may have something similar and use sht as:
>
> @prefix sht:   <http://www.w3.org/ns/shacltest#> .
>
> But I am not sure if it is possible to add that namespace.
>
> Best regards, Jose Labra
>
>  Holger
>>
>>
>>
>> On 4/2/2015 15:04, Jose Emilio Labra Gayo wrote:
>>
>>  In the last F2F, it was already resolved to employ a format similar to
>> what the W3c has employed for other specifications like RDF. In fact,
>> Dimitris and me were assigned the task to create the test-suite.
>>
>> We already started this web page that explains the format:
>>
>> http://w3c.github.io/data-shapes/data-shapes-test-suite/
>>
>>  However, we didn't add more tests because we were waiting until there
>> were more consensus on the language constructs and the error messages of
>> the validator.
>>
>>  I would propose to start with simple tests for the more basic language
>> constructs and to add gradually more tests.
>>
>>  In any case, as you can see in the web page, the format of the manifest
>> file allow us to signal the test status as proposed, accepted, etc.
>>
>>  Best regards, Jose Labra
>>
>> On Thu, Apr 2, 2015 at 4:39 AM, Holger Knublauch <holger@topquadrant.com>
>> wrote:
>>
>>> Could we start defining a format for our test cases? I have attached two
>>> files illustrating a format that I used for a few test cases while writing
>>> my prototype. Each test case consists of two parts:
>>>
>>> 1) A SHACL file including instances
>>> 2) A manifest file declaring the tests to run and the expected output
>>>
>>> Both files are in Turtle, and the manifest file uses a simple ontology
>>> that can be found at the end of
>>>
>>> http://w3c.github.io/data-shapes/shacl/shacl.shacl.ttl
>>>
>>> Of course this is just one possible format among many others, I welcome
>>> alternatives. Yet I would like to point out that it may be beneficial to
>>> have an RDF based exchange format of such test cases, because people may
>>> maintain and publish the test cases together with their data models, as
>>> linked data.
>>>
>>> Regards,
>>> Holger
>>>
>>>
>>
>>
>> --
>>  -- Jose Labra
>>
>>
>>
>
>
> --
> -- Jose Labra
>
>


-- 
Dimitris Kontokostas
Department of Computer Science, University of Leipzig
Research Group: http://aksw.org
Homepage:http://aksw.org/DimitrisKontokostas