RE: XProc test suite

Mio,

See for my answers below:

> here are some comments about the official XProc test suite i
> have collected during development of my own XProc engine
> implementation. I would greatly appreciate some feedback.
>  
> 1. First, some comments about individual tests:
>  
> data-002, data-006 - expected attribute value 'text/plain;
> charset="utf-8"' but was 'text/plain;charset=UTF-8' - the
> test should be modified so that it accepts both forms, AFAIK
> both are ok according to respective specifications

You are correct that both 'text/plain; charset="utf-8"' and 'text/plain;charset=UTF-8' are allowed. That the tests expect 'text/plain; charset="utf-8"' is simply because Norm Walsh (implementor of Calabash) and I sort of agreed to return the former to make our life easier when writing the tests. I agree, changing the tests to accept either 'text/plain; charset="utf-8"' or 'text/plain;charset=UTF-8' (or any other possible variant) would be more correct, but we sort of based the tests around the results of our implementations, implicitly assuming that when there are more implementations, we will update the tests where necessary.

> err-d0006-002 - should throw xd0007 instead, because the
> sequence appears on a non-sequence output rather than an
> input

Indeed, err-d0006-002 should raise err:XD0007. It is a bug in Calumet, and I have already fixed both our implementation and the test. Thanks for catching this.

> err-d0020-001 - should be moved into serialization
> directory, not relevant for engines not supporting
> serialization

Regarding err-d0020-*, moving the tests to the serialization directory might be a good idea. As it is now, the serialization directory contains tests that test support for the XSLT 2.0 and XQuery 1.0 Serialization functionality, and not really for the p:serialization element itself. But that may be just my view, and maybe moving all p:serialization-related tests there would be better...

> err-primary-001 - i cannot find a reason for xs0008 in the
> pipeline of this test, is it supposed to be the primary
> attribute on the p:identity input? isn't this attribute
> allowed by the spec?

The @primary attribute is not allowed on p:input (it is not a declaration, but a binding).

> err-s0007-004 - why is the difference by sequence an error?
> according to the spec (4.4) "As a convenience to authors, it
> is not an error if some subpipelines declare outputs that
> can produce sequences and some do not."

Bug. I have fixed the test (and made it choose-007) as well as our implementation.

> err-s0009-003 - why is the difference by sequence an error?
> according to the spec (4.6) "As a convenience to authors, it
> is not an error if an output port can produce a sequence in
> the initial subpipeline but not in the recovery subpipeline,
> or vice versa."

Bug. I have fixed the test (and made it try-006) as well as our implementation.

> err-s0027-001 - in this test, the default namespace is
> xmlns="http://xproc.org/ns/testsuite", thus it is used in
> determination of the option name in p:with-option, so
> {}:select (short form) and
> {http://xproc.org/ns/testsuite}:select (long form) are not
> the same option on the step, are they?

I think the test is correct. The default namespace does not apply to attribute values in XProc. If you want to specify an option name in a namespace, you have to prefix it.

> err-s0039-00* - all this steps should be moved into
> serialization directory, not relevant for engines not
> supporting serialization

See my answer for err-d0020-* above.

> http-request-003, http-request-009, http-request-010 - see
> comment for data-002 and data-006

See my answer for data-002 above.

> load-002 - throws xc0027 when the engine uses a
> non-validating parser, since this test is required, does it
> mean any xproc engine implementation MUST use a validating
> parser?

The specification (7.1.14) says: "It is a dynamic error (err:XC0027) if the document is not valid or the step doesn't support DTD validation.". So if an implementation is not using a validating parser, I think it must report err:XC0027 (and therefore load-002 will fail). I think this is correct, and implementations that use a non-validating parser will report a failure for load-002.

> xslt-006 - should be moved to optional, because it requires
> xslt 2.0

You are probably right here.

> 2. It would be nice if you could provide the dictionary or
> schema or at least an example of the test suite results XML
> which is used to generate a test results page like
> http://tests.xproc.org/results/calumet/. Also it would be
> great if you could provide the transformation to generate
> the page itself from the results.

The schema for the test result format is described here: http://tests.xproc.org/documentation/testsuite.html

The (semi-public) XSLT 2.0 stylesheet for generating the HTML report is here: http://tests.xproc.org/style/format-report.xsl


> 3. What would I have to do if I wanted to register my own
> XProc implementation so that I can put up a result page like
> mentioned above to the official test suite page?

Contact Norm Walsh. (And welcome aboard!)

Regards,
Vojtech

--
Vojtech Toman
Consultant Software Engineer
EMC | Information Intelligence Group
vojtech.toman@emc.com
http://developer.emc.com/xmltech

Received on Monday, 23 August 2010 14:59:41 UTC