Re: integrating error codes into the test suite and its schema

Norm Tovey-Walsh writes:

> "C. M. Sperberg-McQueen" <cmsmcq@blackmesatech.com> writes:
>> Norm Tovey-Walsh writes:
>>
> […]
>>> I can’t help but feel like you’re making the problem harder by having an
>>> implementation that returns more than one error code. :-)
>>>
>>> Does the following work/is it equivalent:
>>>
>>> If a test is expected to fail and no error code is provided, any failure
>>> code is a pass.
>>
>> I am not sure I understand.  Let me try a paraphrase, and you can
>> correct what I get wrong.
>
> I tried to respond in detail to your paraphrase, but I got confused by
>
>       For any test with an expected result of assert-not-a-sentence (in an
>       instance test), assert-xml, or assert-xml-ref, without an error
>
> because assert-not-a-sentence is an error case in my mind and assert-xml
> is not.

I will try to respond in more detail later, but for now I'll just say we
do seem to have a terminological issue.

I am thinking of "error" as a failure to conform to the ixml spec. As we
explained at some length to an unwilling Dave Pawson a while back, that
means that failure to parse the input string against the grammar is not
in itself an "error".

Clearly both of us periodically want some term to distinguish cases
where the input grammar G is a conforming grammar, the input string S is
in L(G), and the processor produces appropriate XML, from cases where
something else happens.  Since we don't have one handy, I moved down a
level of abstraction and tried to talk just in terms of the assertions
in the test suite.

I think "error" is not the correct term for all cases in the second
class.  If we have to have one, 'failure' might work.  I sometimes refer
to 'abnormal termination', which is probably confusing to some or many
people.

More later.

Michael

-- 
C. M. Sperberg-McQueen
Black Mesa Technologies LLC
http://blackmesatech.com

Received on Thursday, 26 May 2022 14:09:40 UTC