json-ld test suite

On 04/04/2013 06:40 PM, Markus Lanthaler wrote:
> On Friday, April 05, 2013 12:08 AM, Sandro Hawke wrote:
>
>>> Yeah.. The only other thing I'm aware of is the link to the test
>>> suite. Currently it points to http://json-ld.org/test-suite/. Shall
>>> I change that to http://www.w3.org/2013/json-ld-tests/ as we've
>>> discussed? I just checked. It's not setup yet.
>> I think so.    Before setting it up, I was waiting to hear feedback on
>> the URL (eg json-ld/test-suite).  If I proxy that to
>> https://github.com/json-ld/json-ld.org/tree/master/test-suite we get
>> individual test URLs that look like:
>>
>> http://www.w3.org/2013/json-ld-tests/tests/compact-0001-context.jsonld
>>
>> I'm wondering about getting that "/tests" out of there.   Hard to do,
>> though.   Whatever.
> I assumed that we will need to freeze the test suite anyway at some point.
> So it shouldn't be a problem to eliminate that unnecessary sub-directory.
>
> We can change that redirection/proxy at any point, right? So probably we'll
> just create another directory official-w3c-tests or something and move
> everything that was approved there. For the time being, using what we
> currently have should be enough and allows me to change the link.

Turns out I can't really proxy for github because it does redirects, or 
something.   So, instead I made a landing page which explains the 
situation (for now).   I put it on the wiki, so feel free to edit.

>
>> Also, I see the test suite says it's maintained as a free-for-all.
>> That's probably not appropriate, at least while we're going through CR
>> (or an LC that might also be a CR).
> Right. I think it makes sense to have an official, frozen test suite (under
> the req. W3C license) and a place where we collect new tests even after
> going to REC.
>
>
>> I suggest that we start to add to the manifests entries like
>>       "approved": "http://www.w3.org/2013/04/04-rdf-wg-irc#T21-59-26"
>>
>> ... for any test which has been approved by the RDF WG.    (which might
>> be a whole bunch at once.)
> That's another option but it might become problematic when tests are
> changed/extended etc. I prefer to freeze the official test and to not touch
> them anymore :-)

Well, history suggests we will have to keep touching them.

There's no need to ever modify a test in place though; just 
approve/reject them.

Some test suites also have EXTRACREDIT tests, for optional features.

>> This way we don't really need much access control -- anyone can see if
>> the test has been modified since being approved, and check whether it
>> was actually approved by following that link.
>>
>> Then, to exit CR, we'll need people to be passing "approved" tests.
>> This should be explained in the README.
>>
>> We should also tell people how to submit their test results, and where
>> the table of everyone's test results is.
> Gregg is already working on that:
>
> https://github.com/json-ld/json-ld.org/issues/229

I assumed he would.   :-)

Other things I observe about the test suite that might be problems:

    - there's no way to test for extra triples being generated by JSON->RDF
    - there's no way to test RDF->JSON
    - there are frame and normalize tests, but I have no idea what those 
are (since that's not in the spec any more)

Yes?

          -- Sandro

>
>> Yeah, let's add that text and see how it goes.
> OK, cool.
>
>
> Cheers,
> Markus
>
>
>
> --
> Markus Lanthaler
> @markuslanthaler
>
>

Received on Friday, 5 April 2013 12:06:39 UTC