A Call for Rapprochement between W3C XSD and ISO DSDL:
A Non-Intrusive Extension Framework for XSD 1.1 to Support Schematron and Beyond

Rick Jelliffe


This note is a contribution to discussions on adding various kinds of constraint checking to XSD 1.1. The bottom line is a call for a rapprochement between W3C WSD and ISO DSDL: ISO DSDL is not a stalking horse for RELAX NG but should be considered a valuable and primary resource for little languages and approaches for evolving XSD in positive direction.

ISO 19757 Document Schema Description Languages (DSDL) is a multi-part standard for standardizing small, narrow-focus schema languages. It is often portrayed as some kind of attempted competitor to XSD, due to OASIS RELAX NG being one part, but it ain't necessarily so. If you consider ISO DSDL parts 3 and on as a series of small schema languages designed to complement any grammar-based schema language without adding to monolithic complexity, then XSD is clearly the primary potential adopter of ISO DSDL.

The current state of affairs with the W3C XSD WG reminds me of the SGML working group at ISO when we enhancing IS 8879 SGML to encompass XML which we had recently developed through W3C. One group felt that we just needed to parameterize SGML more, to complicate it, in order to cope with the variations required by XML; another group, which I belonged to, instead felt that layering was the answer: at a certain point it becomes positively harmful to add complications to a base specification. So we added an “additional constraints” link (“SEEALSO”), which allowed the SGML constraints to be extended by an external document: SGML validity remained determinate and unitary, and the additional constraints could be validated as a further, different type of validity (i.e. XML well-formedness.)

What is the relevance to XSD? XSD is in the same position as SGML was a decade ago: large, stiflingly difficult to implement, and with a strong requirement not weaken determinate validity. Similarly, this requirement not weaken validity is mistakenly opposed to ideas of layering. In fact the reverse is true: layered systems are easier to test, implement and reason about.

I write not only as the developer of Schematron, and former member of both the W3C XSD WG and the ISO DSDL WG, but also as a commercial developer of schema-related products including XSD. It has long been obvious to me that the XSD-inspired dazedness would eventually clear and that calls for schema capabilities beyond or above that of simple grammars would then have their season: the intent of ISO DSDL has been to collect such little languages for early adopters and for general commmunity benefit as the fog/panic/exploration of XSD clears.

Let me frank, but certainly with no disrespect intended. When XSD was developed, perhaps a majority of the then XSD WG had never actually written a serious DTD or other schema for XML (or SGML.) This perhaps ultimately showed itself in a certain obsessional intricacy in some minor areas (nillibility, the lack of integration of keys and uniqueness into the type system, extension by suffixation only, elementFormDefault, etc.) at the expense of other major areas (patterns on mixed content for example.) I suspect that probably the majority of the current XSD WG may have never written a constraint-based schema for XML, e.g. with Xlinkit, XCSL, Schematron or even XSLT. One cannot after all be a specialist in everything! The tendency of the XSD WG may therefore to be less aware than grassroots users of the advantages, characteristics and opportunities afforded by Schematron, and therefore relegate it to a few convenient categories such as “co-occurrence constraint.” Schematron was developed in 1999, and has continued in its current popularity solely because of its general-purpose utility, not because of any hype or party spirit: XSD users favour it equally with RELAX NG and DTD users. It is used for detecting conflicting flight plans over Belgium, checking software architecture rules in USA, checking local government forms in Japan, and the conformance of documents to house rules by big three publishers including here in Australia.

Executive Summary

  1. Support for simple co-occurrence constraints is better done by allowing attributes as particles in content models rather than by using path expressions. Recommend to adopt the mechanism used successfully by ISO DSDL Part 2 (RELAX NG).

  2. XSD needs an extension mechanism which will allow embedded little languages with constraints required for extended validity. Recommend new PSVI properties such as [extended validity] to support extensibility. Raid and support the ISO DSDL effort for appropriate extension languages.

  3. XSD needs a constraint language, regardless of the support for 1) above. It should use the extension mechanism in 2) above. Recommend ISO DSDL Part 3 (Schematron) as a required (or strongly recommended) extension.

  4. XSLT Keys and Uniqueness probably should be moved out of Part 1 and re-cast as an extension. I don't suppose this is feasible, for reason of scaring the horses. But it is an example of exactly the kind of schema language that should be an extension. Because key and uniqueness constraints are embedded in the Structures specification currently, there is no XSD-compliant way in which developers can experiment and evolve new schema languages. The danger of this is that the XSD WG is forced to do armchair speculative development of enhancements (“yeah that sounds good”): a recipe for perpetual premature standardization, inadequate testing, and a sure way to gather dead wood.

Attributes in Content Models

Support for simple co-occurrence constraints is better done by allowing attributes as particles in content models rather than by using path expressions. The approach used by ISO RELAX NG should be adopted: it has proved to be straightforward to implement, easy for users to understand, is declarative, streamable. This would require, I believe, no changes to the PSVI.

Adopting this does not entail any notion of somehow saying “RELAX NG was right and we were wrong”; on the contrary, interested users would rejoice that XSD was adopting proven technology. RELAX NG adopted this feature late, it was not obvious to James Clark and Murato Makoto etc. that it was the correct feature to adopt. (I believe I was one of the first to suggest it.) But it has proven itself. I hope the XSD WG relentless refuses to take part in any childish NIH-ism in this: XSD's earliest development was guided by the proven experience with various deployed schema languages. I strongly recommend that the XSD WG discipline itself to adopt proven features from existing deployed schema languages when they are available, as in this case.

Even though I am obviously a fan of Xpaths, introducing some reduced Xpath based syntax is, I believe, the wrong approach here: while technically feasible it plays to all XSD's weaknesses. It complicates understanding while only addressing one small area: exactly the kind of “not enough bangs per buck” that XSD is notorious for.

In particular, one of Paul Biron's useful suggestions, to use the streaming subset of Xpaths to identify a node whose presence provides the condition for the occurrence of an attribute or element, is, I think on reflection, the wrong way to go. First because the content model enhancement above is simpler, cleaner and roughly equivalent. Second because it uses paths for what they are OK at, but does not use them for where they shine: with value-based predicates and with random access. Third because the idea that just providing the most basic occurrence constraints will actually satisfy user requirements is wrong-headed: a tokenistic path language will merely temporarily shift the boundary at which user frustration with XSD's power sets in.

Extension Framework

XSD needs an extension mechanism which will allow embedded little languages with constraints required for extended validity.

XSD is notoriously under-layered and complicated to reason about. That even vendors so freely admit the difficulties they have faced in implementing XSD properly should be uttermost in the mind of the XSD WG, I believe; the failure of implementability in XSD must not be quibbled out of, especially given that XSD had a long gestation period that, one would expect, would have made early implementations higher quality than expected.

So the WG needs to adopt a very different mindset: I am not talking about changing the PSVI approach or type derivation, I am talking of the futility of “valid means valid everywhere in all implementations” when combined with a monolithic architecture. The failure of XSD implementations to provide consistent validity results is to some extent attributable to this monolithic architecture. I suggest that the problem is not with “valid means valid everywhere in all implementations” but in the lack of extensibility in XSD: Appinfo is not enough.

My suggestion for an extension mechanism is below. The focus of ISO DSDL (Document Schema Description Languages) is no to provide a standard library of little languages, suitable for XSD to include or allow by reference. These include ISO DSDL Part 3: DTLL (Datatype Language Library) and ISO DSDL Part 7: CRDL (Character Repertoire Description Language). In case there is any feeling that these are somehow “anti W3C” technologies, I perhaps should note that DTLL was developed by Jeni Tennison, invited expert to the W3C XSLT WG, while CRDL was developed by Martin Duerst, long time head of W3C Internationalization. Indeed, CRDL is based on a technical note at the W3C

Extension Framework Details

All places where <xsd:appinfo> are allowed and the top-level should allow a new element <xsd:extension>, allowing any element in a non-XSD namespace



Document Root

[Extended validity] is defined as [validity] plus successful use of all elements in relevant extension elements. Importantly, this allows an on-ramp for implementations to keep their current notations of validity: they can allow but ignore all extensions. However, for extended-validity (which should become the new default for implementations to support) validation fails if either there is an element in an unknown namespace (i.e. One for which the schema implementation does not support) or if validation with those constraints fail, then extended validation fails. This satisfies the important objection to “optional” validation: extended validity always means extended validity.

Note that it is a design requirement in XSD that [Validity] can be assessed in a single-pass streaming fashion. It is not a design requirement that [Extended Validity] can be assessed in this manner. This split commends a layered approach.

Note that the extended validity of the Document Root refers to outcomes of validating extensions defined on the document root, and should not be confused with “the validity of the document”.

[Extended validation attempted] gives a list of the namespaces of the children of the relevant extension elements, which provide keys for different kinds of extended validation.

[Extended validity diagnostics] are lists of [namespace, text] pairs, which provide the namespace of the extension coupled with a human-readable text message, for example as generated dynamically by Schematron. (Note: the PSVI extension do not limit the ability of an API to report other information from schemas for various uses, or to perform different kinds of non-standard validations.)

The presence of these extra PSVI items is the key to extensibility. I don't believe any “required-extension” mechanism is needed or warranted.

Schematron as a Required Extension

XSD 1.1 should define ISO Schematron as a required or strongly recommended extension.

To some extent, attempting to cover all important bases with exhaustive declarative enhanements to XSD becomes an exercise in tail-chasing: even if XSD is extended with a dozen new co-occurrence constraint elements, there will still be the need for a general purpose constraint language. And, indeed, the best way to determine which constraints should be generalized into some first-class property in XSD is to first provide a general purpose constraint language like Schematron to gather information and increase user and WG expertise.

The subset of Schematron used conforms to ISO 19757-3 Information technology — Document Schema Definition Languages (DSDL) — Part 3: Rule-based validation — Schematron (2006) Annex F: Use of Schematron as a Vocabulary. The namespace used is http://purl.oclc.org/dsdl/schematron

The following effective DTD are the required elements and attributes of the subset. ISO Schematron defines other elements: it is an error for them to be present. ISO Schematron defines other attributes: it is not an error for these to be present; they may be ignored.

<!ELEMENT sch:rule (sch:let*, (sch:assert | sch:report)+)>
<!ATTRIBUTE sch:rule
   context (.) #FIXED '.'
<!ELEMENT sch:let EMPTY>
<!ELEMENT sch:let 
   value CDATA #REQUIRED >
<!ELEMENT sch:assert (#PCDATA  | sch:span | sch:emph | sch:dir | sch:name | sch:value-of)*>
<!ATTRIBUTE sch:assert
<!ELEMENT sch:report (#PCDATA | sch:span | sch:emph | sch:dir | sch:name  | sch:value-of)*>
<!ATTRIBUTE sch:pattern
<!ELEMENT sch:span (#PCDATA)>
<!ELEMENT sch:emph (#PCDATA)>
<!ELEMENT sch:dir (#PCDATA)> <!ELEMENT sch:name EMPTY>
<!ATTLIST sch:name
<!ELEMENT sch:value-of EMPTY>
<!ATTLIST sch:name

Note that in this subset: