Entailment as Syntax Test Case Category

I've thought about it some more and still very strongly oppose  
introducing a test case category ostensibly for syntax that works by  
mutual entailment.

Here are my core reasons:

	1) It introduces a new kind of "strange" test case. I think we should  
strongly default to not introducing new categories (we already have  
quite a few) as they increase the complexity and cognitive load of the  
test suite. The barrier to introduction gets much much higher when the  
category is unusual in some respect as that makes it harder to get  
right and clear. The test suite should do what it does well and make  
clear what it doesn't do, esp. if we're not quite sure how to do  
something.
	I trust, for everyone, it's obvious that using mutual entailment to  
test correctness of a syntax conversion is strange (even if there is  
some precedent with RDF...but there, the entailment is very very close  
to a normal syntatic match, i.e., graph isomorphism...and even then,  
it is weird and not as widely used as it could be). Aside from the  
inherent violation of natural separation of concerns, the ME test is  
far too weak. You can easily pass it while not implementing the tested  
function.

	2) It is otiose. If this is the test then there is not need for a new  
category, as one can submit positive entailment tests that cover that  
easily. Similarly for round tripping...as long as we have test that  
cover all the syntax (which we need anyway) there's no need for  
additional categorization.

	3) It is not required. I've seen no evidence that proper syntax tests  
(where the supplied output is associated with an appropriate equality  
function) are impossible. If the WG does not have the energy to  
produce such tests, then we should not produce them.

Cheers,
Bijan.

Received on Sunday, 26 April 2009 17:30:28 UTC