RE: [XMLVersioning-41] Comments and Suggestions on Draft Extensibility Finding


This is an excellent note that deserves a careful reply.   Unfortunately 
I'm off for two days of travel, and so will try and respond in more detail 
later in the week.  Failing that, we can surely talk during the plenary

In the meantime, I want to thank you for the very careful and thoughtful 
explanation.  Given that we've spoken before, little if any of what you've 
written is a suprise, but it is still very useful to see it set out in 
such an integrated way.  FWIW, I think many of what appear to be 
disagreements between us are ones of emphasis more than substance.  For 
example, given that we agree that extensibility is an interesting 
tradeoff, we seem to have different initial intuitions regarding the best 
emphasis in a TAG finding.  I expect that with some discussion we will 
find common ground on such questions.

Mostly, I wanted to thank you for your note, and apologize for not 
replying in more detail until later. 

Noah Mendelsohn 
IBM Corporation
One Rogers Street
Cambridge, MA 02142

"David Orchard" <>
02/21/05 02:46 PM

        To:     <>, <>
        Subject:        RE: [XMLVersioning-41] Comments and Suggestions on Draft Extensibility 


I think it's great that you are taking such an active interest in the
area of extensibility and versioning, and bringing that to the work of
the TAG.  To try to summarize my lengthy response within 1 sentence: I
somewhat disagree with #1, agree with #2, #3, mostly agree with #5,
partially agree with #4 pending my comments within, and I propose the
next update of the finding should at least focus on the issue brought to
the TAG around namespaces and languages. 

I agree that the architectural issues of language design and e/v are
worthy of TAG work.  This was one of the planks in my campaign platform.
I believe that this includes the relationships between language design,
namespaces, components, etc.  Indeed, this is why I proposed UML models
for the extensibility/versioning finding and for the Web architecture,
though I didn't detect overwhelming TAG interest for these at the time.
You may notice that these models, and the textual description in the
finding, show relationships between languages, terms, namespaces, etc.
I am quite keen to work on a finding that helps describe principles
related to namespaces (your point #2) and I think the architectural
models proposed in the finding can be a starting point.

I am interested in exploring language design beyond "simply" xml and xml
schema, but I retain the worry that the more abstract the discussion,
the smaller the audience or the less useful particular audiences will
find the material.  The finding already is almost too general for my
tastes as I believe that XML Schema is the most popular choice of schema
language for xml design.  Moving from XML Schema to XML in general was
necessary for the finding to proceed previously, but I'm not yet sure
that I can support another layer of abstraction or indirection.  This is
the natural tendency between the abstract and specific that I
articulated in the last TAG telcon.  For example, it would be necessary
to give non-xml examples, which would "bore" any xml schema reader.  I
say this with certainty because a common comment on the current finding
is that it is too abstract and readers would like to see the XML Schema
examples up front.  When I have presented at conferences on the topic,
the audience has very much appreciated the focus on XML Schema.  I have
certainly thought of more generalization than just XML schema.  I have
written up at least one of these in a discussion of Protocol
extensibility and versioning, I remember numerous discussions about URIs
and various rules, and I've also written about XML's design from an e/v
perspective.  I previously decided that more generalization, while
perhaps useful to a wider audience, would take away from the target
audience of XML language authors. 

Further, there is the issue of how much time and energy to devote to
various aspects.  I still believe that focusing on XML Schema and
solving the "problems" of how to extend and version multi-ns languages
in a distributed environment is the single most important aspect of this
work.  I'm always a little concerned about "boiling the ocean" which is
a trite way of talking about scope increases. 

I have expressed my reservations on this part, and I expect that we will
have many more discussions on the scoping side. 

On to Pros and Cons, your first point.  I agree that there are pros and
cons to extensibility and versioning.  That is system
design/architecture 100.  However, I believe that the XML community is
struggling with extensibility and versioning because the "e/v pendulum"
swung too far from the HTML style of extensibility to the "draconian
error-handling" side.  Arguably, we promised the larger community a web
style of loose coupling which we haven't achieved.  I regularly hear
from Web services customers the comment that they were told they should
use Web services rather than older distributed object technologies
because they get benefits of the Web wrt coupling, and they've been less
than happy with what they've found.  To a great extent, my goals have
been to push the e/v pendulum back towards the middle.  Perhaps I have
achieved some of that goal given the interest in e/v that is occurring
now that did not occur 2 years ago.  Yet as it stands, I do not believe
that we are building XML based systems that are more loosely coupled now
than we were 2 years ago. 

In general, there are always trade-offs in the "ilities" of systems.
Related to this is the technology choices in the systems.  Again, I
believe that the underlying designs of XML and XML Schema have pushed
the pendulum away from what I call "distributed touchless
extensibility".  Hence I am also leery of moving the extensibility and
versioning finding away from a "pro-e/v" message until I see XML based
systems being regularly built that are too extensible or too

I agree with providing a more detailed description of general guidelines
and the capability of using these to evaluate various language designs.
I was leery of doing this in the finding because it could have seemed
too much to "critique" various languages - and in fact I removed lengthy
sections on what are desirable characteristics and designs from the

I should supply a bit of history and rationale for you.  I started on
this work a number of years ago - the first internal draft was in April
2003 - because of regular customer and working group requests that are
roughly of the form "best practices for non-brittle XML Schemas".  I
started by working on an article, eventually published in, that
focused on a very common case of extensibility and versioning, which is
where one piece of software changes and another doesn't. 

This work led me to believe that there are implicit constraints in the
web architecture.  These are the constraints of extensibility and
versioning, and avoiding the "myth of single administrator".  The
community has gradually increased it's understanding of REST
constraints, but the e/v issues are not part of REST.  As such, I
believed that they were not as widely understood or deployed as were
needed by customers and specification writers. 

Having started from the point of answering an apparently simple question
and progressing to believing that there are fundamental constraints
embodied in the web architecture, it made sense to have a finding on e/v
in the web architecture.  The finding has always had the delicate task
of how far to discuss XML Schema.  In very early versions, there was
very specific information on how to use XML Schema.  There were also
very specific comments on why the XML Schema constructs were less than
desirable, and it proposed a number of suggestions for what could have
been done in XML schema and XML, or perhaps could be.  Currently, the
older article is no longer available as there is a redirect from
the older article at
to the update.  I attach the relevant text towards the end of my
additions.  As an aside, I'm pleased that the main suggestions were
published in 2003 - multiple namespaces, default extensibility, revised
extensibility model for ns, etc. - are contained within your note [11].

To conclude, I think we have a lot to talk about and you've made some
excellent suggestions.   I would like to be focused on the most pressing
issues from our community and this seems to be both: 1) the relationship
between namespaces and languages and 2) the ongoing XML Schema NG work.


>>> begin 12/03/versioning.html extract >>>>
Why is this hard?
We've shown that using XML and W3C XML Schema to achieve loose coupling
via compatible changes that fully utilize yet do not require new schema
definitions is hard.  Following these extensibility rules leads to W3C
XML Schema documents that are more cumbersome and at the same time less
expressive than one might like.  The structural limitations introduced
by W3C XML Schema's handling of extensibility are a consequence of W3C
XML Schema's design and are not an inherent limitation of schema-based

With respect to W3C XML Schema, it would useful to be able to add
elements into arbitrary places, such as before other elements, but the
determinism constraint constrains this.  A less restrictive type of
deterministic model could be employed, such as the "greedy" algorithm
defined in the URI specification [4].  This would allow optional
elements before wildcards and removing the need for the Extension type
we introduced.  This still does not allow wildcards before elements, as
the wildcard would match the elements instead.  Further, this still does
not allow wildcards and type extension of the type to co-exist.  A
"priority" wildcard model, where an element that could be matched by a
wildcard or an element would match with an element if possible would
allow wildcards before and after element declarations.  Additionally, a
wildcard that only allowed elements that had not been defined -
effectively other namespaces plus anything not defined in the target
namespace - is another useful model.  These changes would also allow
cleaner mixing of inheritance and wildcards.  But that still means that
the author has to sprinkle wildcards throughout their types.  A
type-level any element combined with the aforementioned wildcard changes
is needed.  One potential solution is that the sequence declaration
could have an attribute specifying that extensions be allowed in any
place, then a commensurate attributes specifying namespaces, elements,
and validation rules. 

The problem with even this last approach is that with a specific schema
it is sometimes necessary to apply the same schema in a strict or
relaxed fashion in different parts of a system.  A long-standing rule
for the Internet is the Robustness Principle, articulated in the
Internet Protocol [3], as "In general, an implementation must be
conservative in its sending behavior, and liberal in its receiving
behavior".  In schema validation terms, a sender can apply a schema in a
strict way while a receiver can apply a schema in a relaxed way.  In
this case, the degree of strictness is not an attribute of the schema,
but of how it is used. A solution that appears to solve these problems
is defining a form of schema validation that permits an open content
model that is used when schemas are versioned.  We call this model
validation 'by projection', and it works by ignoring, rather than
rejecting, component names that appear in a message that are not
explicitly defined by the schema.  We plan to explore this relaxed
validation model in the future.

A final comment on XML Schema extensibility is that there is still the
unmet need for the ability to define schemas that validate known
extensions while retaining extensibility.  An author will want to create
a schema based upon an extensible schema but mix in other known schemas
in particular wildcards while retaining the wildcard extensibility.  We
encounter this difficulty in areas like describing SOAP header blocks.
The topic of composing schemas from many schemas is difficult yet

Leaving the topic of wildcard extensibility, the use of type extension
over the web might be more palatable if the instance document could
express a base type if the receiver does not understand the extension
type, as in xsi:basetype="". The receiver could then fallback to using
the basetype if it did not understand the base type's extension.

Another area for architectural improvement is that XML - or even XML
Schema - could have provided a mustUnderstand model.  As things stand,
each vocabulary that provides a mustUnderstand model re-invents the mU
wheel.  XML could have provided an xml:mustUnderstand attribute and
model that each language could use.  Tim Berners-Lee articulated the
need for this in XML in his design note on mandatory extensions in Feb
2000[18], but neither XML 1.0 nor 1.1 included this model. 

<<< End insert <<<

> -----Original Message-----
> From: []
> Sent: Sunday, February 20, 2005 9:31 AM
> To: David Orchard;
> Subject: [XMLVersioning-41] Comments and Suggestions on Draft
> Extensibility Finding
> Background
> ----------
> Dave Orchard is leading the TAG's effort on extensibility and
> and with help from co-editor Norm Walsh, Dave has been writing an
> extensive two part draft finding.  Copies of a revised draft were
> to this list in November, just before the TAG's Cambridge F2F  [1].
> TAG members read the revisions in time for the meeting, but Dave
> did walk us through them.  Dan Connolly submitted some comments later
> which generated a bit of discussion [3,4].
> At the meeting, I indicated that I thought the drafts would benefit
> more focus on framing the broader issues relating to versioning, XML
> the Web, perhaps at the expense of some details relating to XML Schema
> and particular XML versioning idioms.  Such broader issues might
> 1) how versioning and extensibility choices affect the utility and
> stability of XML-based Web technologies and 2) investigation of a
> broader range of XML use cases, and 3) deeper exploration of the
> characteristics that we might want from any particular solutions.
> The TAG assigned me an action to make more detailed suggestions, and
> help Dave moving forward.  This note is in fulfillment of the first
> that assignment, I.e. to set out some of the directions I'd like to
> explored.  I hope to work informally with Dave on whether and how to
> integrate these ideas.  I'm sure we'll have lots of opportunity to
talk at
> the plenary.  I should say that overall I like a lot of what he and
> have written, and I hope these will be viewed as constructive
> Overview of Comments, Suggestions, Concerns
> -------------------------------------------
> I. Pros and cons of extensibility
> The "first rule" introduced in the draft is a Good Practice Note (GPN)
> that says [5]:  "Allow Extensibility rule: Languages SHOULD be
> for extensibility."  Other GPNs advocate specific idioms for doing
> In my opinion, this somewhat jumps to a conclusion regarding one of
> most difficult and important tradeoffs relating to extensibility:
when do
> the benefits outweigh the costs?
> I think it's fair to say that some of the most successful Web
> have succeeded as much from the ways that they are inflexible as from
> ways that they are extensible.  XML, which is arguably a success, had
> one of its original goals: "The number of optional features in XML is
> be kept to the absolute minimum, ideally zero."[6]. Except for the
> to define your own element and attribute names and choose character
> encodings, XML is remarkably inflexible and not particularly
> Sometimes that's frustrating:  we couldn't use XML Schema in place of
> in the internal subset, and it's proving very hard to roll out the new
> content conventions for XML 1.1.  Users rightly value the very high
> compatibility that results from XML's inflexibility.  Although the
> correctly cites HTML's open content and "must ignore" tag rules as a
> success, there have also been serious interoperability problems as
> vendors exploited that flexibility to introduce their own flavors of
> I suspect that similar tradeoffs will apply as XML vocabularies are
> designed for other purposes:  extensibility tends to stand in
> to interoperability, and both are important.  I think the finding
would be
> much stronger if it explored such tradeoffs, and gave some more
> guidance as to when things should be locked down and when they should
> extensible.  In fact, such analysis could be one of the essential
> contributions of the finding.  Yes, the answer is often to provide for
> certain forms of extensibility, but we shouldn't recommend that
blindly. I
> think this is a subtle question that's particularly appropriate to the
> scope and mission of the TAG.
> II. Relationship to namespaces
> The recent semi-permathread on immutability of namespaces suggests
> the community would welcome a lucid analysis of the relationship of
> namespaces to vocabularies, languages and to versioning of both.
Part 2
> of the drafts does discuss various strategies, but the permathread
> suggests that the community is looking for >principles< relating to
> immutability or lack thereof of a namespace, principles relating the
> of namespaces to the deployment of language versions and schemas, and
> perhaps principles explaining what role if any namespaces should play
> determining how an application should interpret dialects of the
> vocabularies that it processes.
> III. Dealing with partial understanding
> The draft introduces definitions like "forwards-compatible" [7]:
> "A language change is forwards compatible if older processors can
> all instances of the newer language."
> It also suggests that [8]:
> "Forwards compatibility can only be achieved by providing a
> mechanism for Version 2 instances or Version 1 extensions to V1
> knowledge of V2.  A V1 consumer must be able to transform any
> such as V1 + extensions, to a V1 instance in order to process the
> instance."
> The finding would be stronger if it stepped up to the fact that
> is a matter of degree.  In an extensible system, it's common that even
> early version of an application will have partial ability to process
> features introduced later.  Consider a new element introduced into a
> vocabulary.  Can it be completely ignored, I.e. safely eliminated by a
> substitution?  Well, I suspect that if there is a signature on the
> document then the new element is signed along with the others, even if
> otherwise processed.  If you save the document on disk, do you not
> the elements you didn't understand in detail?  Maybe; it depends why
> you're saving.  If you're a SOAP intermediary, do you relay the
> misunderstood elements?  SOAP gives you an attribute [9] that allows
> to request such relay of content that was not otherwise understood,
> SOAP specifically allows content from such elements to be used as
input to
> other processing (e.g. digital signatures, logging, etc.).  If you
> function to print an XML document, do you print content from the new
> element?  Perhaps not, but you might also have default printing rules
> heuristics that you could use.  The version 4 word processor mentioned
> [7] may indeed successfully read version 5 documents, but may produce
> sub-optimal or incorrect output from some of them.  All of these are
> examples of systems in which partial understanding leads to useful
> processing.  Furthermore, if two different applications are deployed
> on version 1 of a language, those applications may differ in their
> to deal with contrstucts that are introduced later.
> I think the drafts jump a bit too quickly to proposals like "a
> substitution mechanism" and "mustIgnore", and thus obscure important
> issues relating to partial understanding.  Indeed, I'm not convinced
> simple substitution mechanisms are the right framework for dealing
> partial interoperation.
> By accurately modeling a more variable notion of compatibility, it
> becomes possible to explore a question that the schema WG has been
> considering in detail:  how can a schema language help an application
> sort out its different levels of understanding of particular content
> what the application should store, what it should print, which content
> should be processed with what conventions)?  Various options have been
> suggested, including:  (a) because W3C XML schemas uniquely attribute
> element in an instance to a particle in a schema content model, you
> tell which elements were validated by wildcards -- that might suggest
> content you can tolerate but don't fully understand; (b) validate
> subsets of the document (different substitutions) against multiple
> or in various forms of fallback mode when content is not found to be
> valid.  The point is that, to explore such questions, you have to be
> careful with assumptions about what it means for an application to
> "process" an instance, and how such assumptions relate to schema
> Thus, I think the finding should more carefully deal with partial
> understanding of language constructs, and the relationship to schemas.
> IV. Need general guidelines for XML and Schema solutions
> I think it's healthy to set up goals and success criteria separately
> proposed solutions.  The draft does some of this, insofar as it makes
> case that flexible extensibility is a goal.  I think there are some
> detailed goals that should be set out or considered before getting
> particular XML and Schema idioms.  Some that occurred to me are in the
> white paper I wrote last year [10,11], including:
> * The same vocabulary may be versioned or fixed repeatedly.
> any general approach should be convenient to use even after 20 or 30
> revisions.  Both instances and schemas of the later versions should be
> easy to create and use.
> * The versioning mechanisms should (in most cases) not presume
> instance constructions such as <extension> elements.
> * In some but not in all cases, some degree of forward and/or backward
> compatibility is be required:  I.e. it should be possible but not
> essential to write early schemas that will somehow accept content that
> not fully defined until later, and schemas for later versions will
> but not always validate earlier forms of the vocabulary.  (The draft
> cover this one, I think.)
> * Conversely, breaking changes should not in all cases be forbidden.
> example, it may be that an early construct is deprecated at some later
> time, and perhaps completely disallowed eventually.  Likewise, later
> versions may introduce constructs that are rejected outright by
> ones.
> * It should be possible to check for or force various sorts of forward
> backward compatibility when desired (this is the notion of partial
> recognition and processing, mentioned in III above).
> * Schemas for versions of a vocabulary may but need not form a
sequence or
> tree, in which later versions somehow directly reference particular
> documents for earlier versions.  This flexibility allows for possible
> redefinition of the same vocabulary by multiple organizations or in
> than one schema (e.g. there's a debug schema and a production schema,
> neither based explicitly on the other).
> * A consequence of the point above is that the schema for version x is
> necessarily expressed as a delta on or by direct reference to the
> for version x-1, if in fact the versions form a sequence at all.  Such
> incremental definition schemes are convenient, but do not necessarily
> scale to the case where the same vocabulary is revised 20 or 30 times.
> such a case one would need up to 30 schema documents to assemble the
> effective schema.   Thus, such incremental schemes should be allowed
> useful, but not presumed in all cases.
> * No unnecessary assumptions should be made regarding the
> between vocabularies and XML Namespaces.   Often, a vocabulary will be
> expressed primarily as a single XML namespace.  Often, to maintain
> and backward compatibility, that same namespace will be used in
> versions as well.  Nothing in the overall XML mechanisms to support
> versioning (e.g. schema language constructs) should prohibit the use
> coordinated evolution of multiple namespaces to define one or more
> languages, the addition of new namespaces in subsequent versions of a
> language, etc. (Here I admit I'm staking out a personal position on
> Namespaces question raised in II above).
> The above is NOT necessarily the right list, but I think the finding
> make a contribution if it set out such principles separately from any
> proposed solutions.  If we do retain a Part 2 that discusses
> extensibility idioms, then they should each be rated against explicit
> goals such as the examples listed above.
> V. The relationship between syntax and semantics
> Though it mentions other options in passing, the finding deals
> with examples in which the syntax of the XML more or less directly
> the evolving semantics of the underlying data or application.  For
> example, a given parent element may allow for elements or attributes
to be
> introduced to express features of the language as it evolves.  This is
> indeed a common idiom, and it's appropriate that the drafts explore
> Nonetheless, such approaches do not cover the full spectrum of common
> mechanisms for versioning XML vocabularies.  Perhaps, as in SOAP
> or RDF, the XML is a serialization for a higher level model,
versioning of
> which is not well expressed at the element and attribute level.  We
> go into more detail about the implications for XML and schemas, I
> Sometimes new versions of a language specify coordinated updates to
> use of or constraints on the contents of elements or attributes
> throughout a document.  Perhaps an attribute changes the meaning of a
> legacy element (e.g. currency="peso").  Perhaps the specification of a
> SOAP header requires that it be used with other headers (which may be
> interspersed with other headers).  In all these cases, it becomes
> difficult to tell the versioning story entirely in terms of XML
> and attributes, and it's often problematic to do a useful job of
> expressing the pertinent constraints in XML Schema languages.
> In such systems, the extensibility of semantics is only indirectly
> to the syntactic structure of the XML.  If the finding is to achieve
> goal of exploring the versioning of XML vocabularies, then it's as
> important to either deal with such approaches, or to make the case
> they are not important.  I think they will be common and are
> (BTW: I suspect that "mustIgnore" at the XML level does not cover such
> higher level versioning particularly well.)
> Summary
> -------
> Taken together, the above represent a proposal to focus the finding
> on the details of particular XML constructions, and more on the
> versioning and evolution strategies that are likely to be essential to
> Web's and XML's continued success.  Indeed, there's some question as
> whether the most useful finding would continue to focus only on XML,
> also might introduce some general principles applicable to many media
> types, and then apply those to XML (or RDF, etc.) in particular.  I do
> recognize that issue XMLVersioning-41 [12] is currently scoped
> specifically to XML.
> In general, following the precedent of the Architecture Document [13],
> should explore high-level tradeoffs and principles, somewhat in
> to making detailed recommendations on syntactic mechanisms.  While
> lots of good work on in the drafts on XML Schema specifics, especially
> Part 2, I think those are only the purview of the TAG insofar as they
> necessary to motivate the broader themes and principles, or are truly
> central to the Web's success.
> Other details of ensuring that W3C XML Schema is usable to support
> versioning scenarios are explicitly in the charter of the XML Schema
> [14];  indeed, I'm delighted that the TAG and Schema WG are now
> more closely together.  I think the general balance should be that the
> Schema WG handles the schema-language-specific parts of the problem,
> help from the TAG, and the TAG discusses the broader architectural
> with help from (among others) the Schema WG.
> There remains a question of whether the TAG will choose to do a formal
> finding in this area at all.  I am cautiously optimistic that we can
> should, but I do feel that our focus should be more on broader themes,
> perhaps including those discussed above.  I certainly think it's worth
> continued effort in the coming weeks to see whether we can do
> that the community would value.
> My recent rereading of the drafts has reminded me once again what a
> careful and diligent job Dave has done to take us to this point, and
> speaking for myself it is much appreciated!  This start will prove to
> very valuable, regardless of how we proceed, or whether any of the
> suggestions made above are adopted.  I look forward to helping Dave
> Norm in any way that I can to improve the drafts.
> Thank you all for your patience with this long note.
> Noah
> [1]
> [2]
> [3]
> [4]
> [5]
> 0071/versioning-part1.html#identify
> [6]
> [7]
> 0071/versioning-part1.html#terminology
> [8]
> 0071/versioning-part1.html#div250901096
> [9]
> [10]
> [11]
> 0010/NRMVersioningProposal.html
> [12]
> [13]
> [14]
> charter.html#Deliverables
> --------------------------------------
> Noah Mendelsohn
> IBM Corporation
> One Rogers Street
> Cambridge, MA 02142
> 1-617-693-4036
> --------------------------------------

Received on Tuesday, 22 February 2005 00:36:27 UTC