RE: tests and inference?

Hum.  I think that testing for protocol conformance would be part of a
data access test suite ... but, let's focus on the other question:

  "Can DAWG deliver a test suite for the query language?"

It seems that unless there is a means to constrain what inference may
be performed in response to a query, that the behavior of the query
processor can not be tested.

I think that a standard that can not be tested would be likely to show
lots of interoperability problems in the field.

-bryan

-----Original Message-----
From: Seaborne, Andy [mailto:andy.seaborne@hp.com] 
Sent: Thursday, December 16, 2004 8:43 AM
To: Thompson, Bryan B.
Cc: 'Dan Connolly '; 'RDF Data Access Working Group '; Bebee, Bradley R.
Subject: Re: tests and inference?




Thompson, Bryan B. wrote:
> Andy,
> 
> So, would you agree that, as things stand, we can not have test cases 
> for the functional performance of a SPAQL server as part of the spec?

I was talking about the query language, not the protocol.  Testing servers
is 
about protocol.  What is not covered in testing functional performance that
is 
not covered by issuing queries and checking the results?  Example?

I'll defer on what we are doing about testing the protocol - it's harder
(not 
impossible) than testing just the query language as it is built on several 
different components which interact (HTTP, networking, the query processor).

> 
> If so, then I think that this is an issue that needs to be addressed.  
> The simplest path would be to specify conformance for servers with no 
> inference, and then leave clients on their own when the server was 
> performing inference.
> 
> I also think that this issue is sufficiently important that DAWG might 
> re- consider a means in the protocol for the client to demand NO 
> inference,

Coudln't that be done by asking a different service?  An RDF graph is the
unit 
being queried (or collection of graphs in the case of SOURCE).  If a
publisher 
wishes to provide access to both inference and non-inference versions of
some 
information, then this can be achieved by having two services.  If the
publisher 
does not wish to do that, then let's allow that choice as well.

 > if
> only so the client could establish a baseline against which to 
> understand whether or not the server was performing correctly.

I don't completely understand this - "performing correctly" means returning
the 
right answers.

If the testing is by the information publisher, then they can choose to do
this 
(internally to their deployment) how they want.   No protocol defined
feature 
necessary.

If the testing is by a regular client, then I don't understand what acess to
the 
non-inference version achieves.  The service is query over one of more RDF 
graphs.  How those graphs come about is an implementation matter and a
matter 
for what the publisher chooses to make available.  I don't think we need
define 
("an implementation must provide to be compliant") a feature, leave it to
teh 
publishers to decide what services to make available.

	Andy

> 
> As you say, there is no means to have functional tests in the broader 
> realm of server-based inference.
> 
> -bryan
> 
> -----Original Message-----
> From: Seaborne, Andy [mailto:andy.seaborne@hp.com]
> Sent: Thursday, December 16, 2004 7:59 AM
> To: Thompson, Bryan B.
> Cc: 'Dan Connolly '; 'public-rdf-dawg-request@w3.org '; 'RDF Data Access
> Working Group '; Bebee, Bradley R.
> Subject: Re: tests and inference?
> 
> 
> 
> 
> Thompson, Bryan B. wrote:
> 
>>Based on my current understanding, the client is not able to either be
>>informed concering what inferences the server may draw, nor is able to 
>>constrain what kinds of inference the server may perform.
> 
> 
> Not within the query language itself.
> 
> For the description of capabilities:
> 
> Such information could be recorded in RDF in a service description.  
> This would seem to be the way to get any information relating to the 
> service made available
> - assert some RDF facts.
> 
> Like RDF assertions about any document - we (DAWG) don't need to 
> provide a
> mechanism to find, publish, change, sign, ... such information as it can
be
> a 
> regular RDF graph somewhere.
> 
> For the constraining of queries:
> 
> Different services would offer different capabilities.  I think the 
> spectrum of possibilities is wide enough that there is no fixed set we 
> could define now and
> be useful - implementations don't fall into a neat set of a few classes of

> functionality.
> 
> 	Andy
> 
> 
>> Given this,
>>how can SPARQL tests be written that could be used to demonstrate
>>(non-) conformance by SPARQL processors that perform inference?
>>
>>-bryan
>>
> 
>

Received on Thursday, 16 December 2004 14:11:03 UTC