- From: Abel Braaksma <abel.braaksma@xs4all.nl>
- Date: Thu, 10 Mar 2016 17:51:00 +0100
- To: "'Michael Kay'" <mike@saxonica.com>, "Public XSLWG" <public-xsl-wg@w3.org>
Thanks, I will add these after the call :) > -----Original Message----- > From: Michael Kay [mailto:mike@saxonica.com] > Sent: Thursday, March 10, 2016 3:52 PM > To: Abel Braaksma > Cc: Public XSLWG > Subject: Re: ACTION-2016-02-17-003: ABr to create a readme file for the > repository based on this email: https://lists.w3.org/Archives/Public/public- > xsl-wg/2016Feb/0001.html > > Suggested addition: > > ### CHALLENGING AND CORRECTING TESTS ### > > If you believe that there is an error in a test, raise a bug report at > https://www.w3.org/Bugs/Public/ - use product = "XPath / XQuery / XSLT", > component = "XSLT 3.0 Test Suite". > > If you have write access to the repository, please obey the following protocol > for changing existing tests: > > (a) Changes that can be made without formality > > * If it's a simple error in a test that you yourself committed very recently, you > can change it without any record other than the commit message > > * You can also make non-substantive changes without formality, for example > expanding the description of the purpose of a test or improving the layout, > or fixing the catalog to be schema-valid. > > (b) Changes that should be marked by adding <modified by="your name" > on="date" change="reason for change"/> below the <created/> element. > > * If you are confident that a test needs to change because the specification > has changed, you don't need to raise a bug report: fix the test, but always > include a record of your change > > * If someone else has recently committed a new test and it contains obvious > errors, you can correct the test with a simple record of the change you have > made > > * If you are confident that the dependencies of the test have been > incorrectly documented, for example if the test requires a streamable > processor, then again, you can fix the test with a simple record of the change > that you made. > > * Similarly, if you find that the test makes assumptions about the result that > are unwarranted (for example if the result assertions assume the order of > items in the result), and these assumptions are not central to the purpose of > the test, you can replace them with assertions that improve the > interoperability of the test. > > (c) Changes that require consensus > > * Any change that goes beyond those listed above requires consensus. This > doesn't mean it necessarily needs to be debated by the Working Group. It > may be enough just to check with the author of the test that you haven't > overlooked anything. You should ask yourself "is there any possibility that > someone will disagree with this change?", and if the answer is yes, you need > to raise a bug report. > > * If the test raises questions about the correct interpretation of the spec, it's > probably better to raise the bug against the spec rather than against the test. > But cite the test case in your bug report. > > * Once a bug report is raised, the test suite coordinator will decide whether a > WG decision is needed, or whether it is possible to reach informal consensus > through discussion on the bugzilla forum. > > (d) Housekeeping > > * For all changes, include a pertinent commit message when committing. For > all non-trivial changes, also include a record of the change in the form of a > <modified by=.../> element. > > * Always check that your changes to test catalogs are schema-valid. > Remember that the order of elements in a test-case is significant. > > Michael Kay > Saxonica >
Received on Thursday, 10 March 2016 16:51:34 UTC