- From: David A. Lee <dlee@calldei.com>
- Date: Tue, 30 Dec 2008 07:57:30 -0500
- To: "Norman Walsh" <ndw@nwalsh.com>, "XProc Dev" <xproc-dev@w3.org>
Quote NW: > I have discovered that some tests have changed (I think recently) Not very recently. ---------- Dec 4, I guess recent is relevent. Its just yesterday I refreshed and discovered it. > I liked it the way it was before where there were no such hidden > requirements. > > Making the input URL's relative is great, but only if you can > somehow manage comparing the outputs so they dont depend on the > previous assumptions. I guess I'd be open to suggestions on that point, but for the purposes of comparison, it was expedient to simply run the tests from the website. For my own purposes, I sometimes run the tests locally while I'm debugging and I just ignore the failures that I know are caused by that. ------------- Suggestion: Put it back the way it was. Prior: Tests were independant of where they were run. Atleast this test, I havent checked them all. But with this set of tests they worked and were deterministic reguardless of where they were run. The test body itself was explicit and the results were correct without assuming anything. Now: Tests (atleast this one) are non-deterministic. They produce different results depending on where they were run. In order to "pass" tests MUST be run with the base URI set to the http://tests site reguardelss of where they were run. Tests now have an undocumented hidden assumption which is completely unnecessary and adds considerible complexity and confusion. IMHO, the change was not an improvement, it was a regression. Compromise: If for some overwhelming reason you think using the relative URI's in the tests are necessary, or better, then I suggest you put the base URI in the test as well so that programatically a test process can figure out what the relative URI is relative to to produce to "correct" results. Simply "ignoring failures" because you "know" its OK ... sometimes ... is not a good strategy ... especially for us poor fools who are using the tests to figure out how xproc is supposed to work. But I suggest this is unnecessary complexity and work when they worked fine as full URL's before. Tests that are more sensitive, rather then less , to initial conditions and environemnt make the implementors job very frustrating. Especially when they used to work then dont, I assume its my code. In this particular case, I assert the test is actually broken now until you add something to the test structure and documentation to make explicit the implicit assumputions which are now coded into the test. Or just put it back and avoid all that work. ----------------------------------------------------------- David A. Lee dlee@calldei.com http://www.calldei.com http://www.xmlsh.org
Received on Tuesday, 30 December 2008 12:58:07 UTC