Re: Status of lc2 comments - outstanding primer-1, testcases-5

On Tue, 11 Nov 2003, Brian McBride wrote:

> However, that still leaves 5 comments on TestCases and one on primer
> needing a response.

One (more than one?) of those comments is Dave R's question on the test
cases whose correctness depends on Pat's Semantics draft - that JJC has
also raised with the list. I can't divine exactly what the situation is
with that, so I was holding off from commenting.

Sandro's raised a couple of points which might be boiled down to one
comment, that the test case manifest format enshrines a "closed world"
view. This comment would apply to the declarations of: entailmentRules,
supportedDatatype (I think, WebOnt's test cases have the same issue
with datatypes), and the premise documents for entialment tests.

The options are basically to

- either fix this with a small change to the manifest syntax and the
test case document describing that syntax. However, that small change
would mean that the manifest format changes at quite a late stage. (Jos
amongst others is in favour of doing it "right" here.)

This change is substantive but does not affect any of the actual test
cases, only their descriptions.

- introduce the entailmentRules marker for simple entailment. This does
not address all of Sandro's comments.

- reject the closed world comment. Such a rejection would be on the
basis that it's just too late in the game to alter the test format. In
support of this, the test case document has always claimed:

[[
The test cases have been created to illustrate the resolution of
particular issues on the RDF Issue Tracking list. Consequently, test
case results should always be interpreted in conjunction with the
resolution of the associated issue and the description (if any) attached
to the test case in the manifest.
]]

In other words, although we make effort to enable running the test cases
automatically, if your harness "fails" a test then inspection "by hand"
may be required. This would include the failure of a test case due to
the open/closed world issue.


So "where we're at" is that there are several options. Were time not an
issue I'd fix the test case format - I believe the number of test
harness authors is small and "doing it right" is worthwhile. However I
appreciate that this is effectively a substantive change to the test
case document (since harnesses will be broken) so I'm unprepared to make
that call without the approval of the chairs and/or WG.

Of course, if semantics has had substantive changes (which I think are
to the good) post-LC2 then making a substantive change to the test
format seems like less of an issue.

-- 
jan grant, ILRT, University of Bristol. http://www.ilrt.bris.ac.uk/
Tel +44(0)117 9287088 Fax +44 (0)117 9287112 http://ioctl.org/jan/
I shave with Occam's Razor.

Received on Wednesday, 12 November 2003 05:16:04 UTC