Re: follow up on TOOLS - what is your opinion?

Hello David

I have to finally break my silence, don’t I? :)

I’ve just created an issue in the implementation report repo to continue that discussion there [1].

I think that one more missing piece in the tooling space is also a consistent testing platform for current and future. I can only talk about the R2RML spec and my observation is that the implementors of the tools are a bit on their own in terms of actually creating the report. The official test harness only provides the means to build the EARL documents but running the transformation first, preferable on different daily should be simplified.

For this purpose I had earlier created a test runner template repository [2] which I would propose extending with dockerised setup of various RDBMS so that any tool can be tested in a controlled, repeatable environment. Individual implementors would only have to fork and provide their "run command”. The shared setup does all the heavy lifting required to bootstrap the database, insert seed data for each test case and run the test harness in the end.

Best,
Tom

[1]: https://github.com/kg-construct/r2rml-implementation-report/issues/2
[2]: https://github.com/r2rml4net/test-runner 


On 29 October 2020 at 21:58:55, David Chaves (dchaves@fi.upm.es) wrote:
> Dear all,
> Last but not least, our ideas about the next steps about tools and evaluation methods.  
>  
> We suggest
> • waiting for results from use cases discussion before proceeding further with discussions  
> on how the use cases may reflect on the data
> • organizing a smaller group to discuss on methods for evaluating and comparing tools  
> • organizing a smaller group on a set of new test cases that could be considered for heterogeneous  
> data beyond what was already proposed for R2RML and was translated for heterogeneous  
> data (https://github.com/RMLio/rml-test-cases). The idea is then, similarly to  
> mapping languages, that people can write down their approach of tackling the challenge  
> as a scientific paper.
> • organizing a smaller group to revive and keep it alive a new R2RML implementation report  
> (https://github.com/kg-construct/r2rml-implementation-report)
>  
> The different options do not exclude each other!
>  
> We welcome comments on the suggestions, proposals on how to proceed and of course intend  
> of participation in any of the two above.
>  
>  
> Best regards,
> David
>  
> David Chaves
> PhD Student
> Ontology Engineering Group (OEG)
> Universidad Politécnica de Madrid
>  
>

Received on Wednesday, 4 November 2020 19:42:52 UTC