- From: Holger Knublauch <holger@topquadrant.com>
- Date: Thu, 29 Sep 2016 20:02:08 +1000
- To: Jose Emilio Labra Gayo <jelabra@gmail.com>
- Cc: "public-data-shapes-wg@w3.org" <public-data-shapes-wg@w3.org>
- Message-ID: <1c2027ec-14f1-3fb1-2b1b-e8c9fb4a421a@topquadrant.com>
On 29/09/2016 19:47, Jose Emilio Labra Gayo wrote: > > On Thu, Sep 29, 2016 at 2:10 AM, Holger Knublauch > <holger@topquadrant.com <mailto:holger@topquadrant.com>> wrote: > > (Moved thread back to mailing list, I hope that's OK) > > > I was hoping that we could try to solve this without adding more noise > to the WG mailing list. > > By your email, can I assume that you want to change the test-suite > format that has already been accepted by the WG? The work on our test case framework received very little attention for the last year or so. So I believe it's a valid question to at least take another fresh look. > > If that is the case, you should probably raise a issue proposing that > change and justifying the rationale for it. > > You are proposing to remove manifest files because they are difficult > to maintain. > > However, although I partially agree that they may be difficult to > maintain, I think manifest files help separate concerns and reuse > shapes graphs and instance data. Also, manifest files are a common > practice in most of the W3c specifications. For example, both RDF and > SPARQL specifications define their test-suites using manifest files > [1], [2]. Why should we reinvent something new when there is an > accepted practice on this? > > Your example: http://datashapes.org/testcases.html#ValidationTestCase > lacks information about which nodes are going to be validated and > against which shapes they are going to be validated. Where is that > information? The tests simply validate all target nodes based on the usual SHACL target mechanism. In this particular case, ex:TestShape is both a class and a sh:Shape, which means that all its instances will be validated against that shape. In other tests, I am using things like sh:targetNode and sh:targetClass. > > In that example, it appears that two violation erros must be reported > for "ex:InvalidResource1", but where is it declared that the SHACL > processor must validate "ex:invalidResource1" against "ex:TestShape"? > > Also, in that example, "ex:ValidResource" seems to be a valid > resource...where is it declared that it must be validated against > "ex:TestShape"? > > I would assume that the information about those validations should be > declared in the manifest file. See above, no manifest file necessary. > > In fact, I would have defined the shapes graph in a different file as > well as the instance data so they could be reused for other tests. Yes, this can be achieved via owl:imports, again using standard mechanisms. Admittedly, a downside of my current architecture is that the assumption is that shapes graph == data graph. This is not always the case and would need to be refined by introducing another property that would be attached to the TestCase instance. Look, I don't want to start longish debates. I am fine either way. It's more important that we actually have test cases. Holger > > Best regards, Jose Labra > > > > > > > > [1] https://www.w3.org/TR/rdf11-testcases/ > [2] http://www.w3.org/2009/sparql/docs/tests/README.html > > > On 29/09/2016 6:28, Jose Emilio Labra Gayo wrote: >> As I said yesterday during the WG call, I have been working on a >> translator that could map part of the shex test-suite to SHACL >> with the goal to add the resulting tests to the test-suite. >> >> One problem that I am finding is that I am not sure if all of you >> agree on the expected result of what is a test-suite. >> >> My expectation was to have a battery of tests following the >> format described here: >> >> http://w3c.github.io/data-shapes/data-shapes-test-suite/ >> <http://w3c.github.io/data-shapes/data-shapes-test-suite/> >> >> That format was proposed and approved in a WG meeting some time >> ago. Do you agree with it now? >> >> I am asking because in the meantime, I have seen several tests >> added to the repository: >> >> https://github.com/w3c/data-shapes/tree/gh-pages/data-shapes-test-suite/tests >> <https://github.com/w3c/data-shapes/tree/gh-pages/data-shapes-test-suite/tests> >> >> which don't follow that format. >> >> I have also seen this page: >> >> http://datashapes.org/testcases.html >> <http://datashapes.org/testcases.html> >> >> but I am not sure how does it fit here...Holger, are you >> proposing that we use that tool to generate the tests? What would >> be the role of that tool? (I am asking before installing it). > > The link above shows a tool but is primarily about an RDF data > model/ontology to represent test cases. An example of this format is > > http://datashapes.org/testcases.html#ValidationTestCase > <http://datashapes.org/testcases.html#ValidationTestCase> > > This format is more compact than the one that we had proposed in > the WG last year. In particular it no longer requires the overhead > of the manifest metadata - it will simply execute all test cases > in a given folder. I am rather negative about the design to have > the expected validation results in the separate manifest file - > this will lead to a maintenance nightmare. > > But otherwise it is very similar. If your starting point is your > own format (from ShEx) anyway, then potentially it shouldn't > matter too much to which format you write your converter for? > > Overall I would appreciate other opinions on whether the format > from datashapes.org <http://datashapes.org> could be an option or > not. As said before, I am too biased because I wrote it myself. > > Holger > > > >> >> In principle, if everyone agrees with the proposed format or >> something similar, I volunteer to add more tests and to >> edit/clean the contents of the repository, but I would like to >> know, if we are all on the same page here. >> >> Dimitris, are you also still interested in working on the >> test-suite? If not, maybe Mark could work with me here (I would >> prefer not to be alone in this task) >> >> Best regards, Jose Labra >> PS. I prefer to send this email without using the mailing list to >> avoid adding more noise. >> >> >> >> >> >> On Wed, Sep 28, 2016 at 1:29 AM, Holger Knublauch >> <holger@topquadrant.com <mailto:holger@topquadrant.com>> wrote: >> >> Answering Mark, the process that I am using is already >> functional for everyone with tool support from TopBraid >> Composer (Free Edition) 5.2. Just place the test files into >> the workspace, open the Test Cases view and press the run >> button as described in http://datashapes.org/testcases.html >> <http://datashapes.org/testcases.html> >> >> While I personally find this tool support and format very >> convenient (including the ability to create new tests with a >> single button click), I am of course too biased to promote or >> push this format further. However, it should be easy to write >> translators between the officially proposed test format and >> the one in the dash namespace. So if the WG officially >> decides on the format, I will provide such a translator >> because I'll need it myself. >> >> Holger >> >> >> >> On 28/09/2016 6:49, Jose Emilio Labra Gayo wrote: >>> During today's meeting we were talking about the test-suite. >>> >>> My goal is to work on the test-suite once the spec seems to >>> be more stable. I am now working in a translator that will >>> convert part of the ShEx test-suite to SHACL. I hope to be >>> able to provide this new battery of tests in the next 2/3 weeks >>> >>> In principle, I was considering to maintain the structure of >>> the test-suite that had been approved by the WG some time ago. >>> >>> I think it would be better to keep that structure and to add >>> more tests following it...although if the WG considers that >>> we should use a different approach, maybe we should discuss >>> it before doing more work on it. >>> >>> Best regards, Jose Labra >>> >>> >>> On Tue, Sep 27, 2016 at 5:51 PM, mark <markh@metarelate.net >>> <mailto:markh@metarelate.net>> wrote: >>> >>> I haven't seen any further discussion on this topic. >>> >>> I think it is a valuable thing to have a w3 maintained >>> set of >>> validation cases >>> >>> If no-one is actively using >>> https://github.com/w3c/data-shapes/tree/gh-pages/data-shapes-test-suite >>> <https://github.com/w3c/data-shapes/tree/gh-pages/data-shapes-test-suite> >>> I would support retiring this (onto a branch, so it's >>> not lost) and >>> putting the current set of test cases from TopQuadrant >>> into the W3 space >>> >>> I am happy to help administer this if it is a useful >>> approach. >>> >>> Holger: If I stepped in to help, would you be happy to >>> start from your >>> current test cases and link your development flow into >>> W3 test cases >>> resources once it is functional? >>> >>> Are there other implementations that we should consider >>> contacting >>> directly to encourage them to use a new shared set of >>> test resources >>> who would help drive the development and maintenance of >>> these? >>> >>> What would working group members make of this approach? >>> >>> all the best >>> mark >>> >>> >>> On Mon, 5 Sep 2016 15:33:52 +1000 >>> Holger Knublauch <holger@topquadrant.com >>> <mailto:holger@topquadrant.com>> wrote: >>> >>> > Just to point out some work related to test cases for >>> SHACL. >>> > >>> > The current library of (53) test cases that I am using >>> to validate >>> > the TopBraid SHACL API can be found at >>> > >>> > >>> https://github.com/TopQuadrant/shacl/tree/master/src/test/resources/sh/tests >>> <https://github.com/TopQuadrant/shacl/tree/master/src/test/resources/sh/tests> >>> > >>> > The vocabulary, including tool support, is described here: >>> > >>> > http://datashapes.org/testcases.html >>> <http://datashapes.org/testcases.html> >>> > >>> > Who else has any test cases that are tracking the spec? >>> > >>> > The work on an official test cases format has been >>> dormant: >>> > >>> > >>> https://github.com/w3c/data-shapes/tree/gh-pages/data-shapes-test-suite >>> <https://github.com/w3c/data-shapes/tree/gh-pages/data-shapes-test-suite> >>> > >>> > Is anyone still "driving" that format and the >>> corresponding tests? >>> > >>> > Thanks, >>> > Holger >>> > >>> > >>> >>> >>> >>> >>> >>> -- >>> -- Jose Labra >>> >> >> >> >> >> -- >> -- Jose Labra >> > > > > > -- > -- Jose Labra >
Received on Thursday, 29 September 2016 10:02:51 UTC