Reporting on QT3 test results for the 3.1 Specs

Dear All,

I have re-generated the reports on the implementation results received for the QT3 test suite. See update: http://dev.w3.org/2011/QT3-test-suite/ReportingResults31/report.html <http://dev.w3.org/2011/QT3-test-suite/ReportingResults31/report.html>
We now have three implementations for XQuery 3.1, with one being anonymous. However we only have one for XPath 3.1.

We have been working on the main page today. Specifically the stats on the optional features. We found that there were some overlap reported in the passes and failures. For example under ‘staticTyping’ it is reported that there are 75 test not being passed by any implementation. Under the 'higherOrderFunction’ feature we have 18 test passed by no implementation. If you inspect the higherOrderFunction tests more closely we will see that 17 of these tests depend on staticTyping therefore the underlying reason for the higherOrderFunction failures is to do with staticTyping. With what I have said in mind we think it helpful to show the combination of optional features for the test case.

I welcome feedback on your preference of presentation on the Optional Features section. I find 2) much more useful:

1) http://dev.w3.org/2011/QT3-test-suite/ReportingResults31/report.html#optFeatures <http://dev.w3.org/2011/QT3-test-suite/ReportingResults31/report.html#optFeatures>
2) http://dev.w3.org/2011/QT3-test-suite/ReportingResults31/report.html#optComboFeatures <http://dev.w3.org/2011/QT3-test-suite/ReportingResults31/report.html#optComboFeatures>


Using 1) or 2) we observe that we have 75 tests with dependency on staticTyping not being passed by any implementation (including those with combinations of other dependencies).  I would like to ask if it still merits having tests on static typing?

kind regards,


-------------------------------
O'Neil Delpratt
Software Developer, Saxonica Limited 
Email: oneil@saxonica.com <mailto:oneil@saxonica.com>
Twitter: https://twitter.com/ond1 <https://twitter.com/ond1>
Tel: +44 118 946 5894
Web: http://www.saxonica.com <http://www.saxonica.com/>
Saxonica Community site: http://dev.saxonica.com <http://dev.saxonica.com/>
Bug tracking site: https://saxonica.plan.io/ <https://saxonica.plan.io/>