Re: Call for Concensus: Update SPARQL 1.1 test suite to align with changes introduced in RDF 1.1

> On Nov 5, 2015, at 2:20 PM, Andy Seaborne <andy@apache.org> wrote:
> 
> 
> 
>> Also, following this, the tests at www.w3.org/2009/sparql/docs/tests/
>> should be either synchronized from, or redirected to
>> https://w3c.github.io/rdf-tests/sparql11/.
> 
> Ok - I've lost the plot here.
> 
> If links www.w3.org/2009/sparql/docs/tests/ gets to go to (the content of) https://w3c.github.io/rdf-tests/sparql11/
> 
> then
> 
> 1/ Where are the original tests the WG implementation report going to be?
> 
> There are links in the implementation report back to the tests under www.w3.org/2009/sparql/docs/tests/ so those would not be stable.

If we don’t change the semantics of any given test, that shouldn’t be a problem; the proposed changes should either simply mark the old tests as Obsolete, or create new tests, which are Proposed; this requires a change to the existing PR. An updated implementation report would either report values for obsoleted tests (if the implementation’s report wasn’t updated), or not have an entry for a new test.

We need to weigh the plus/minus of keeping separate frozen snapshots of everything. That said, we can certainly create a frozen branch for the state of the test suite at any given point, and refer to that for any history that needs to be shown.

> Or are all the links in the implementation report going to be updated to point to a frozen copy?

The implementation report used for transition is static and shouldn’t change. It should, however, contain a link to the most current implementation report, which can simply be the gh-pages version.

> Suggestion:
> 
> Like the yellow "New Version Available" boxes at the top of old RDF and OWL specs, put one of those at the top of www.w3.org/2009/sparql/docs/tests/ pointing to the new area.
> 
> 2/ Aren't the w3c.github.io/rdf-tests/sparql11/ evolving at the moment and may be new tests?  Specifically, they are not formally frozen but www.w3.org/2009/sparql/docs/tests/ is to a frozen set?
> 
> (I found 2 tests that depend on a specific execution order which is not a spec compliance assumption)

Keeping the tests in a non-standard place may be a problem for some, where there is an assumption about the test location (mostly, Turtle IIRC). I think that if we keep a link from the current, to the historic test/report locations that serves much the same purpose. The most useful thing for implementors is to have a predictable place to find the most useful tests and/or reports.

I would say the submitted Implementation Report is updated to point to the gh-pages version for newer results, or some W3C-maintained reflection of this. The tests point to or otherwise reflect the latest tests, which keep a link to the historic tests used for the implementation report.

Gregg

>  Andy
> 
>

Received on Thursday, 5 November 2015 22:40:53 UTC