Re: [Zorba-coders] Re: Zorba revision 8827 XQuery/XQueryX Update Test Suite Results

Sorin,

please regenerate the results with "NOT APPLICABLE" instead of fail.
Please resend to Andrew after that.

Andrew, I guess this wasn't clear enough for us.


Thanks
Dana


On Sep 15, 2010, at 1:38 PM, Andrew Eisenberg wrote:

>
> > Hi Andrew,
> >
> > again thanks for your efforts.
> >
> > I looked over the reports and I would have some questions if I may:
> >
> > - for 'Minimal Conformance' Saxon EE is shown as passing all the
> > tests: how is that since for test 'namespace-errors-q16' there is an
> > opened bug #10432 ?
> > Zorba reports the alternate result as proposed by Markos and agreed
> > upon by Michael Kay but since that alternate result is not in  
> XQUTS at
> > the moment, Zorba reports 'namespace-errors-q16' as failing, which  
> is
> > the correct thing, right?
> > Am I missing something here?
>
> This too shall pass. I understand your confusion here. We published  
> XQUTS 1.0.1, began collecting results and publishing reports,  
> accepted bug reports, and modified the test suite. Saxon's results  
> were submitted earlier, yours were submitted later. I'm planning to  
> wait until Bug #10432 is resolved and then ask all vendors to  
> resubmit their results.
>
> > - for the optional feature 'Update Facility Static Typing Feature'
> > Zorba mentions that it does not support this feature (see
> > http://dev.w3.org/2007/xquery-update-10-test-suite/results/
> > XQUTSReport.html#features)
> > yet in the first table of the report Zorba is shown in red (meaning
> > failed) instead of white (which stands for untested).
> > Could it be that there is an error in the report generation  
> process here?
> > I mean shouldn't the tests for a feature that an implementation
> > mentions it does not support, be shown in white ?
>
> The stylesheet that generates the reports uses the pass/fail results  
> that you provide, independent of whether the feature is supported.  
> You could provide a result of  "not applicable" or "not tested"  
> instead of "fail", Only "fail" generates the red shading.  You could  
> add a comment attribute explaining that the feature is not supported  
> if you wish.
>
> > - for the optional feature 'fn:put applied to nodes other than
> > document and element nodes'  there are 4 tests that Zorba 'fails'  
> but
> > these tests should be run only if the XQuery processor does NOT
> > support the feature.
> > Since the tests were created by you, can you please tell us what is
> > the correct way to handle these tests so that we can correctly show
> > them as passing?
>
> As I said above, you could report a result of  "not applicable" or  
> "not tested". You could not generate results for those tests. This  
> looks like what the other products are doing.
> _______________________________________________
> Zorba-coders mailing list
> Zorba-coders@lists.inf.ethz.ch
> https://lists.inf.ethz.ch/mailman/listinfo/zorba-coders