Re: Test cases format

On Thu, Apr 2, 2015 at 8:18 AM, Holger Knublauch <holger@topquadrant.com>
wrote:

>  Ok thanks for the pointer. As far as I understand it, the results so far
> are only booleans of whether a given node matches a shape. I would hope we
> extend this with the full result of the validation (once this vocabulary
> gets agreed upon) so that implementers can verify that their engine returns
> meaningful results.
>

We'll do that as soon as things get more stable, for now we can easily
check TRUE/FALSE where FALSE is at least one violation. Later we should
extend this for result lists & severity levels as well.


> I also think we need a way to validate a complete graph with all nodes
> using the built-in node selection properties sh:nodeShape and rdf:type.
> This would then also include global constraints.
>

This can be covered with the TRUE/FALSE approach again. Later we can either
adjust the existing tests to perform more advanced result checking or
create new test cases for that.


> Regarding the two Syntax tests, I can see why they created them for SPARQL
> - to test the SPARQL string parsers. However, for SHACL it sounds like we'd
> mainly need structural tests of the syntax. Currently the draft of the
> SHACL schema is self-validating, i.e. it is possible to validate SHACL
> shape definitions using SHACL itself. As a result, we may not need the
> Syntax tests.
>

I thought that too at the beginning but we need them (at least) for the
SPARQL test cases to check valid SPARQL syntax and possibly the correctness
of variables. This might be also needed if we deliver a compact syntax.


> Finally, if we only rely on the mf: namespace, how could we add new
> features if we need them?
>

True, we'll need to change that. So far Jose did all the work and we
decided to minimize the effort until we actually started creating tests and
see the actual needs.

Q: do you think it is a good idea to move tests in a separate repo so that
we can import them easier as a git submodule?

Dimitris

>
>
> Holger
>
>
>
> On 4/2/2015 15:04, Jose Emilio Labra Gayo wrote:
>
>  In the last F2F, it was already resolved to employ a format similar to
> what the W3c has employed for other specifications like RDF. In fact,
> Dimitris and me were assigned the task to create the test-suite.
>
> We already started this web page that explains the format:
>
> http://w3c.github.io/data-shapes/data-shapes-test-suite/
>
>  However, we didn't add more tests because we were waiting until there
> were more consensus on the language constructs and the error messages of
> the validator.
>
>  I would propose to start with simple tests for the more basic language
> constructs and to add gradually more tests.
>
>  In any case, as you can see in the web page, the format of the manifest
> file allow us to signal the test status as proposed, accepted, etc.
>
>  Best regards, Jose Labra
>
> On Thu, Apr 2, 2015 at 4:39 AM, Holger Knublauch <holger@topquadrant.com>
> wrote:
>
>> Could we start defining a format for our test cases? I have attached two
>> files illustrating a format that I used for a few test cases while writing
>> my prototype. Each test case consists of two parts:
>>
>> 1) A SHACL file including instances
>> 2) A manifest file declaring the tests to run and the expected output
>>
>> Both files are in Turtle, and the manifest file uses a simple ontology
>> that can be found at the end of
>>
>> http://w3c.github.io/data-shapes/shacl/shacl.shacl.ttl
>>
>> Of course this is just one possible format among many others, I welcome
>> alternatives. Yet I would like to point out that it may be beneficial to
>> have an RDF based exchange format of such test cases, because people may
>> maintain and publish the test cases together with their data models, as
>> linked data.
>>
>> Regards,
>> Holger
>>
>>
>
>
> --
>  -- Jose Labra
>
>
>


-- 
Dimitris Kontokostas
Department of Computer Science, University of Leipzig
Research Group: http://aksw.org
Homepage:http://aksw.org/DimitrisKontokostas