Re: AW: [RIF-Test] RIF Test Cases

Hi Adrian,

Well, I see no practical problem representing that in metadata see 
example below.

I don't buy the argument that this carries semantics and so shouldn't be 
metadata. From the point of view of a RIF translator or engine the 
metadata is just metadata. The ruleset can be used as a ruleset on its 
own outside of any testing situation (albeit a boring ruleset in this 
case). The test metadata doesn't affect the semantics of normal rule 
execution. It is just that a (separate) test harness would be able to 
extract the test metadata and so execute and validate the test on the 
contained ruleset.

Document(
   (*
     :test1
     And (
       :test1 [
         dc:name -> "Test 1"
         dc:comment -> "Example of test metadata"
         dc:creator -> "mailto:Adrian.Paschke@gmx.de"^^rif:iri
         dc:creator -> "mailto:Dave.Reynolds:hp.com"^^rif:iri

	rif:testType -> rif:positiveEntailmentTest
	rif:testStatus -> rif:pendingDiscussion

         rif:query -> "p"^^rif:local(?X)
	rif:resultSet -> "_1"^^rif:local
       ]

       "_1"^^rif:local [
         rif:resultEntry -> rif:resultTuple( "1"^^xsd:int )
         rif:resultEntry -> rif:resultTuple( "2"^^xsd:int )
       ]
     )
   *)


   Group (
     (Forall ?X ("p"^^rif:local(?X) :- "q"^^rif:local(?X)))
     "q"^^rif:local( "1"^^xsd:int )
     "q"^^rif:local( "2"^^xsd:int )
   )
)

The idea here is that each rif:resultTuple uniterm gives a single tuple 
of variable bindings corresponding to the variables in the query. Not 
sure that it is necessary to represent the set of expected results as a 
subframe, it just seemed more flexible that way.

Note that I switched the test query to be p(?X) rather than q(?X) since 
I guessed that was what you meant.

Dave


Adrian Paschke wrote:
> Here an example 
> 
> Your rule program:
> 
> p(X):-q(X).
> 
> Your (unit) test case for the rule program:
> 
> Your test assertions (test facts)
> 
> q(1).
> q(2).
> 
> Your test query
> 
> q(X)?
> 
> Your intended results
> 
> true 
> X=1, X=2
> 
> 
> For the query we could reuse the RIF condition language. For the variable bindings we could use equality  lists. But we would need to distinguish from the use of the condition language e.g. to represent facts. In principle, we could use meta data properties. But, there is a general language design problem. Several of the meta data properties would carry semantics which is important for the execution / processing of a test case, e.g. test case type "positiveEntailmentTest" or "negativeEntailmentTest". So, if meta data is something we can be omitted from the main rule document without loosing important information for the execution, it would be better to define it as part of the main rule language, i.e. a query construct, an answer construct. 
> 
> Other, properties like author, status, date, description of the test case a clearly meta data and could be part of the manifest of the test case.
> 
> 
> - Adrian
>  
> 
> 
> -------- Original-Nachricht --------
>> Datum: Fri, 15 Aug 2008 12:16:15 +0100
>> Von: Dave Reynolds <der@hplb.hpl.hp.com>
>> An: Stella Mitchell <cleo@us.ibm.com>
>> CC: public-rif-wg@w3.org
>> Betreff: Re: AW: [RIF-Test] RIF Test Cases
> 
>> Stella Mitchell wrote:
>>>  > What we currently have is a format to describe test cases on the  
>>>  > WIKI which is mostly intended for human readers.
>>>
>>> I think it's more accurate to say that the format on the wiki (really,
>>> the files that the wiki display will be translated into) are intended
>>> to be machine readable and are intended to be executed (to aid in
>>> conformance evaluation). But in order to run them in an automated way,
>>> the testers will need to design and implement their own test harnesses.
>>> Also, to express additional test types we don't currently have, we will
>>> need to define more metadata properties and values (though you say
>> metadata
>>> may not be enough to express all additional test types we might want)
>>> This clarification doesn't contradict the rest of what you say though.
>> Seems reasonable to me. I see no particular problem in defining a 
>> metadata vocabulary to express the test cases as RIF metadata.
>>
>> Is there a specific example test which illustrates the sort of 
>> information needed to drive a test harness but is not expressible via a 
>> metadata vocabulary?
>>
>> Dave
>>
>>> Stella
>>>
>>>
>>>
>>> *"Adrian Paschke" <Adrian.Paschke@gmx.de>*
>>>
>>> 08/14/2008 07:05 PM
>>>
>>> 	
>>> To
>>> 	Chris Welty <cawelty@gmail.com>, adrian.paschke@biotec.tu-dresden.de
>>> cc
>>> 	public-rif-wg@w3.org, der@hplb.hpl.hp.com, Stella 
>>> Mitchell/Watson/IBM@IBMUS
>>> Subject
>>> 	Re: AW: [RIF-Test] RIF Test Cases
>>>
>>>
>>> 	
>>>
>>>
>>>
>>>
>>>
>>>  
>>>  > I see what you mean now.  On the telecon I thought you were talking 
>>> about
>>>  > designing a test-cases dialect for a rule language that is more 
>>> expressive
>>>  > than
>>>  > BLD. Really you mean, I think, a set of standard meta-data 
>>> "properties" to
>>>  > hold
>>>  > the test-case manifest.
>>>  >
>>>
>>> Yes and no. What we currently have is a format to describe test cases on
>>> the WIKI which is mostly intended for human readers. What I would 
>>> additionally envision is to represent test cases directly in RIF so that
>>> they can be interchanged together with a RIF rule program/rule set. Test
>>> cases can then be used by automated tools to e.g.:
>>>
>>> 1. verify the syntax of the interchange RIF program (producer side test)
>>> 2. verify the execution environment, i.e. test that the consumer side 
>>> can correctly translate, interpret and execute the received RIF program
>>> 3. validate that the RIF program covers the unit tests (producer tests, 
>>> e.g. for test-driven engineering of RIF programs, consumer tests to 
>>> validate that the (application specific) semantics of the received 
>>> program and the rule engine comply)
>>>
>>> The test cases described on the WIKI are based on templates such as 
>>> http://www.w3.org/2005/rules/wiki/Template:PositiveEntailmentTest which 
>>> use a set of standard properties defined in 
>>> http://www.w3.org/2005/rules/wiki/Test_Case_Format#Format. Some of these
>>> properties like author, discussion etc. are more informative (could be 
>>> part of the manifest file), others like e.g. test type, dialect, 
>>> premise, conclusion contain important semantic information which is need
>>> to correctly execute a test case.  
>>>
>>> So, if we want to represent test cases directly in RIF we need to add 
>>> this information to a RIF document, either as additional meta data, e.g.
>>> to distinguish facts from queries and conclusions or we would need new 
>>> XML constructs or attributes, e.g. for conclusions, queries, variable 
>>> bindings, expected result of a test (positive entailment or negative 
>>> entailment) etc.
>>>
>>> But let's start simple and first collect positive entailment tests which
>>> demonstrate BLD and DTB.
>>> Some example test cases for BLD (positive entailment test cases) can be 
>>> found here
>>>
>>> http://www.w3.org/2005/rules/wiki/Test_Case_Ordered_Relations
>>> http://www.w3.org/2005/rules/wiki/Test_Case_Unordered_Relations
>>> http://www.w3.org/2005/rules/wiki/Test_Case_Frames
>>> http://www.w3.org/2005/rules/wiki/Test_Case_Equality
>>>
>>> - Adrian
>>> -- 
>>> Ist Ihr Browser Vista-kompatibel? Jetzt die neuesten
>>> Browser-Versionen downloaden: http://www.gmx.net/de/go/browser
>>>
> 


-- 
Hewlett-Packard Limited
Registered Office: Cain Road, Bracknell, Berks RG12 1HN
Registered No: 690597 England

Received on Saturday, 16 August 2008 15:03:02 UTC