Re: CSVW Test Harness

On Dec 1, 2014, at 1:20 AM, Ivan Herman <ivan@w3.org> wrote:
> 
> Gregg,
> 
>> On 30 Nov 2014, at 15:09 , Gregg Kellogg <gregg@greggkellogg.com> wrote:
>> 
>> On Nov 29, 2014, at 11:49 PM, Ivan Herman <ivan@w3.org> wrote:
>>> 
>>> As a former user of the RDFa test site: great:-)
>>> 
>>> On the RDFa site a test implementation could be used by providing a web service of some sort, taking an HTML file and producing a turtle output. What will be the requirement this time?
>> 
>> Basically, a service will take a URL of a test input and perform a transformation to return a result. The type of transformation could be provided either via Accept ,header, or additional query parameter TBD. For RDF, any standard format should do.
>> 
> 
> We may also keep the door open for people sending their own EARL files. I am not sure my implementation will be good enough for a thorough testing, as I said at the TPAC, but it is basically providing a jQuery extension, ie, does not run through a web service. I could of course make a variant running with node.js, but I simply cannot promise I would do that...

Sure, GreenTurtle had that same issue with RDFa, nothing says that the test harness is the only way to run tests. However, a node.js implementation of your service would also be useful.

Gregg

> Ivan
> 
>> In this case, the test input could be either a CSV, or a metadata file, possibly with link header for describedby.
>> 
>> We likely also need to test validators, which could be described using a different format or query parameter. Once we flesh out more tests, this should become more apparent.
>> 
>> Gregg
>> 
>>> Ivan
>>> 
>>>> On 29 Nov 2014, at 20:52 , Gregg Kellogg <gregg@greggkellogg.net> wrote:
>>>> 
>>>> I've created an initial version of the CSVW test harness [1]. This is similar to the online test runner used for RDFa [2], but uses AngularJS for the web application, and is substantially simpler.
>>>> 
>>>> Right now, the only configured processor is "Reflector", which simply loads the expected results and compares it with itself. However, it should work with an arbitrary processor. Note that the test manifest loaded is one Andy and I created some time ago, and the actual tests will change as we move forward.
>>>> 
>>>> Tests and manifest are loaded from GitHub dynamically, but we will likely need to change this to run from the harness itself, so that we can control HTTP headers.
>>>> 
>>>> For now, the repository is at [3]. It could simply be folded into the existing CSVW site on GitHub, and a new instance run from there.
>>>> 
>>>> Note that both RDFa and JSON-LD groups used such a site for more than just running test suites, including general information, links to documents, and a playground for testing formats. We could do the same, but for right now, the site is limited to just the test harness.
>>>> 
>>>> Also, it could use a better name! This is simply a randomly-generated name provided by heroku. A W3C subdomain could potentially point here, or we could do something else.
>>>> 
>>>> The site is built using Ruby/Sinatra and AngularJS. It is pretty generic, and could be used for similar tests (such as the existing RDF test suites) with fairly minor modifications.
>>>> 
>>>> Suggestions, comments, bug reports are welcome.
>>>> 
>>>> Gregg Kellogg
>>>> gregg@greggkellogg.net
>>>> 
>>>> [1] https://floating-sands-3222.herokuapp.com
>>>> [2] http://rdfa.info/test-suite/
>>>> [3] https://github.com/gkellogg/csvw-test
>>> 
>>> 
>>> ----
>>> Ivan Herman, W3C
>>> Digital Publishing Activity Lead
>>> Home: http://www.w3.org/People/Ivan/
>>> mobile: +31-641044153
>>> ORCID ID: http://orcid.org/0000-0003-0782-2704
>>> 
>>> 
>>> 
>>> 
> 
> 
> ----
> Ivan Herman, W3C
> Digital Publishing Activity Lead
> Home: http://www.w3.org/People/Ivan/
> mobile: +31-641044153
> ORCID ID: http://orcid.org/0000-0003-0782-2704
> 
> 
> 
>

Received on Monday, 1 December 2014 15:31:07 UTC