W3C home > Mailing lists > Public > w3c-ietf-xmldsig@w3.org > January to March 2000

RE: Enveloped signatures and XPath

From: Jonathan Marsh <jmarsh@microsoft.com>
Date: Mon, 27 Mar 2000 15:17:56 -0800
Message-ID: <5F68209F7E4BD111A5F500805FFE35B91D3FDEDC@RED-MSG-54>
To: "'John Boyer'" <jboyer@PureEdge.com>, "IETF/W3C XML-DSig WG (E-mail)" <w3c-ietf-xmldsig@w3.org>
Cc: "Martin J. Duerst" <duerst@w3.org>, James Clark <jjc@jclark.com>, Joseph Reagle <reagle@w3.org>, Eastlake Donald-LDE008 <Donald.Eastlake@motorola.com>, TAMURA Kent <kent@trl.ibm.co.jp>, "Christopher R. Maden" <crism@exemplary.net>, Ed Simon <ed.simon@entrust.com>
> -----Original Message-----
> From: John Boyer [mailto:jboyer@PureEdge.com]
> <john>
> Right.  The exact point I'm trying to make is that 
> information needed for
> serialization (specifically, the namespace prefix) is not publically
> available.

Yes it is.  It's in the original document as attributes or namespace
information items, and is exposed through the XPath data model as namespace
nodes.  Since you own the document passed to XPath, there is nothing it
knows that can't be derived publicly.

>  However, we can deduce from the XPath 
> specification that the
> required information (e.g. namespace prefix) is available 
> privately to the
> XPath evaluator.  It must be available since otherwise the 
> XPath evaluator
> would be unable to do anything with the initial namespace declarations
> created during evaluation context initialization.

It is available to the XPath evaluator because it is in the document.  It is
passed to the XPath evaluator through the context node.  You still have this
information outside XPath.
> Since the required information is available internally, AND 
> certain existing
> XPath function must be able to access that information, we 
> can deduce that
> the information is available to XPath functions.  Hence, 
> since serialize()
> is defined as an XPath function, it must be able to access namespace
> prefixes and document order positions, whether or not they 
> are in the public
> data model exposed to XPath expression authors.
> </john>

The serialize function has available what is passed into it as arguments.
If it is a nodelist, the function can follow these pointers back to the
original document, and get this information.

> > Also, I think that serialization is the one thing that you
> > would not expect
> > an XPath engine to provide (unlike parsing).
> I do not expect an XPath engine to provide parsing.  
> Microsoft's doesn't, as
> an example.
> <john>Microsoft has an XPath engine.  Firstly, tell me where 
> it is so I can
> look at it.

IE5 shipped with an "XSL Pattern" engine in MSXML - see the selectNodes and
selectSingleNode DOM extensions.  The latest MSXML technology updates this
to an almost complete subset of XPath.  Docs and downloads can be found at

> Is it open source?

> Second, regardless of whether it provides a
> parser in the concrete sense, it must be coupled with a parser in the
> abstract sense.

Indeed XPath evaluation is a service provided through DOM extensions,
specifically as methods on DOM Node objects.  The inverse would also be
possible - an XPath object that was given various context nodes.  Your spec
should not unduly constrain the use of either of these types of

Which XPath processor are you using to evaluate this design?  I'm really
only familiar with a small class of processors - Microsoft, Oracle, and to a
lesser extent others within XSLT implementations.

>  You must initialize the evaluation context, you must
> provide a context node, position and size, the context node, 
> position and
> size together form a node-set, and the XPath evaluator must be able to
> derive from that node-set the namespace prefixes and the 
> document order
> positions.  Regardless of where you got the processor, you have had to
> create enough data structure SOMEHOW to support the described
> operations.</john>

Yes, which brings up a nasty point I hadn't realized before.  Our
implementation does not currently allow you to pass in any function bindings
(except when used within XSLT).  So I wouldn't be able to build a full
implementation of XPath transforms on MSXML without some pretty painful
hackery.  It would be much easier if no extension functions were needed.  I
wouldn't put too much weight on this, but it may be evidence of a class of
simple XPath processors that couldn't be used for XPath transforms.

> Likewise, my XML processor may indeed provide a serialization
> feature.  Microsoft's does, for example.  I'm not able to use 
> the existing
> functionality that may be provided by a processor because a specific
> implementation strategy has been prescribed - the serialize() 
> function.
> Now, I don't believe the Microsoft serializer IS quite 
> adequate because your
> constraints are too severe - but that's a separate potential issue.
> <john>If your serialization function were adequate, then I 
> wonder why you
> can't call it from your serialize() implementation?
> As to its adequacy, if the tool doesn't work, then don't use 
> it.  You can
> only reuse code in  a new application if it fits the 
> application.  However,
> the serialization described in the spec is QUITE easy to 
> implement, so I
> cannot fathom why it would be a 'potential issue'.
> </john>

I will defer this to another thread.

> As I indicated previously, I would not expect an XPath 
> implementation to be
> optimized for returning the huge strings that could result from the
> serialize() function.  And as I indicated previously, the 
> necessity/danger
> of user intervention on the serialized string has not been 
> justified.  Even
> the necessity of a two-step process of serializing and then 
> converting to
> UTF-8 gives me pause for thought - it looks like I might need 
> two separate
> serialize() functionalities to obtain maximum performance.
> <john>
> The need for serialize() has been justified on several 
> levels.  Firstly,
> there is no security threat in allowing user intervention on 
> the serialized
> node-set.

My term "dangerous" was clearly a poor one in this context :-)  I meant not
that a security risk was imposed, but that it would be quite easy to damage
the well-formedness of the XML document.  Manipulating XML through string
manipulation is unquestionably more prone to error than manipulating it
through structural methods.  You can tell the user "well, you're stupid and
you broke it" but unless there is a compelling reason to ALLOW him to break
it, it seems self-evident that it's best not to.

> For you to argue otherwise would require that you 
> explain why it
> is significantly different from writing a different XPath 
> expression. The
> XPath expression is signed, so its modification of the input follows a
> well-known behavior that we can prove is the correct behavior 
> leading to the
> given message digest.
> Secondly, I've justified it from the specification 
> standpoint.  By defining
> the behavior as a function library addition to XPath, the 
> result of the DSig
> WG's work can continue to be viewed as an application of 
> XPath, not as an
> extension of XPath.

XPath transforms is an application of XPath which is comprised of several
- parsing the document from a stream
- identifying the nodes to keep/trim using an XPath
- trimming the tree
- serialization of the result

Note that the last two are described as coincident by the serialize()
function.  This is a perfectly good application of XPath.  Providing
serialize() does not make it a better application of XPath, any more than
removing it would make it less of an appropriate XPath application.

If somehow it were shown that the serialize() function is necessary to make
XPath transforms an application, what about the case when it is performed
automatically?  Does somehow this use constitute an XPath extension?  The
logic of your assertion totally escapes me.

> Thirdly, I've justified it on technical grounds.  There is information
> required for serialization that is not available in the 
> public information
> but which MUST be available to the implementations of certain XPath
> functions.
> Please have another look at how XPath handles the namespace 
> declarations in
> the evaluation context.  If I have misread something, please 
> let me know,
> but I did ask for clarification on this point, and noone 
> (including James
> Clark) has refuted this point yet.
> As for two serializations, that's not happening.  The first act is a
> 'serialization' of the node-set.  The second action is a character
> transcoding to UTF-8.  My original assumption was that 
> implementations would
> not lose track of the input document's encoding.  This was 
> based on the fact
> that I specified a parse() that would do this.  You 
> specifically along with
> others wanted to get rid of the parse().  The natural result 
> is that I can
> no longer claim that the input document stays in the same 
> encoding.  Since I
> also lose the BOM, UTF-8 is the only viable alternative.
> serialize(), however, cannot output UTF-8 because it is an 
> XPath function,
> which must output a string.  The details of what that means 
> are specific to
> the XPath implementation itself.  Transcoding to UTF-8 is not 
> an action that
> fits within XPath.
> </john>

My concern here is that if serialize() is called automatically, I can
envision a single step in which the document is serialize directly to UTF-8
as a performance gain.  When serialize() is called explicitly, it looks like
a two-step process is needed - serialize into a string to be returned from
the XPath engine (presumeably Unicode) and then a conversion to UTF-8.  I'm
not sure if this is a real issue, but it does appear that there may be
difference in optimization potential and thus performance between the two
uses.  Users would need to be aware that explicit use of serialize() may be
slower than automatic use - another small argument against including it.

> >   So another
> > justification for
> > adding serialize() as a function is that we are adding DSig's key
> > functionality using the XPath-defined extension mechanism
> > (function library
> > modification).  This maintains the property that dsig is an
> > application of
> > XPath, not something different that happens to use the XPath
> > expression
> > syntax.
> I still don't understand this argument either.  If the 
> serialize function
> can be called automatically for nodesets as you prescribe, 
> this would seem
> to indicate that there is not a necessity to expose it to the 
> user in all
> cases.  If you are not using the prescribed XPath extension 
> mechanism in all
> cases, how does this contribute to maintaining "the property 
> that dsig is an
> application of XPath"?  The output of the XPath processor is 
> not the end
> result of the transform in all cases.  Doing more or less 
> back-end work is a
> pretty abstract distinction.
> I feel that it is already an appropriate application of XPath 
> to locate
> nodes in an XML document.  Going beyond this to serializing 
> within the XPath
> expression itself doesn't contribute to the impression that 
> dsig is an XPath
> application.  In fact, my gut feeling is that it is abuse 
> because it appears
> to short-circuit the primary XPath return type of nodeset.
> <john>The specification very clearly points out that the 
> XPath can return a
> node-set.  However, to run a digest algorithm, we need a 
> string.</john>

This is true but doesn't address my point.  I think it's great that a user
can return a nodeset and it'll get serialized automatically.  In fact, it's
so great I can't see why anyone would ever need to bother with calling
serialize() explicitly.

> The redundancy argument still applies.  I can call the 
> serialize() function,
> or not, with the same result.  Without some use cases 
> justifying the need
> for the long form, the short form looks perfectly adquate to me.
> Note that there is no standard mechanism defined by XPath to "call"
> extension functions outside of the XPath expression.  The XSLT spec
> carefully uses descriptions like "the resulting object is 
> converted to a
> string _as_if_ by a call to the string function" (emph 
> added.)  A similar
> approach would be appropriate in describing XPath transforms.
> I don't see a concrete problem that serialize() solves.  I do see some
> concrete reasons not to have it - performance, implementation 
> flexibility,
> eliminating redundancy, simplified syntax.
> <john>Sure, as long as you can guarantee that the moment we 
> yank serialize,
> we aren't going to have half a dozen different people writing 
> in to say that
> "my-favorite-XPath-engine doesn't have access to namespace 
> prefixes" or
> "my-favorite-XPath-engine makes it really inefficient to do 
> document order".

First of all, I can't guarantee people won't complain.  I can't even
guarantee their complaints won't be justified.  I only can state that I'm
not aware of any implementations that would have problems with this, and
it's hard to imagine how they could and still conform to namespace nodes in
the XPath data model.  You have not previously presented evidence of such
implementations or even their potential.  My goal is to enable a wider range
of XPath engines to be used, and I would welcome more details about
constraints on specific implementations.

> Also, I do like the idea of being able to tweak the data in 
> certain ways
> once it has been serialized.  One could wrap a root element 
> around a list of
> elements, one could prepend some simple material to make the 
> result of the
> transform more usable to certain software.
> </john>

Structural manipulations like this should be left to XSLT, which is designed
to perform exactly this type of operation without damaging well-formedness
or the meaning of the document.

In summary, I still haven't found even one of your arguments for exposing
serialize() to the user compelling.  Since we aren't converging on
solutions, perhaps we should focus instead on converging on a clean
statement of our different positions?  There appears to be very little in
the way of hard technical arguments either way, so a win/loss outcome seems
Received on Monday, 27 March 2000 18:20:44 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 21:21:33 UTC