XPath action item, was Re: Enveloped Signatures and XPath

Yes, right, forgot about name().  I discarded that function a long time ago
because the spec does not say what I wanted it to say.  name() does not
return the QName, which is what I need for serialization.  The spec says
that implementations can allow name() to return the QName, or they can
return something other than the originating QName if there are multiple
namespace prefixes mapped to the same URI in the namespaces provided to the
evaluation context.

This is not a well-defined 'function', but I don't want to get into XPath
bashing right now since they most likely were forced into the same corner I
see the XPath transform being pushed into, which is that some have created
dumb (I mean DOM) parsers that forget the namespace prefix. Why? Why? Why!

This is a great part of why I specified our own parse() function.

So, the disappointing reality is that there remains no good reason to keep
serialize() as a function because there is no hope of obtaining the original
namespace prefix if the wrong input parser was used.

Moreover, this means that unless one carefully constructs one's signature
transforms so that multiple namespace prefixes do not resolve to the same
URI in the set of namespaces available to the XPath element, one will be
required to do a c14n post-processing to guarantee the same result from all
implementations.

Well, perhaps it's not the end of the world. If one is careful, one can
still avoid c14n.

I guess this warrants another write-up that describes the serialization but
omits the fact that it appears as a function.  This will clear up the
remaining objections expressed so far.  It is not a major change, so I
should be able to squeeze this into my Thursday schedule...

John Boyer
Software Development Manager
PureEdge Solutions, Inc. (formerly UWI.Com)
Creating Binding E-Commerce
jboyer@PureEdge.com





-----Original Message-----
From: w3c-ietf-xmldsig-request@w3.org
[mailto:w3c-ietf-xmldsig-request@w3.org]On Behalf Of Jonathan Marsh
Sent: Monday, March 27, 2000 4:56 PM
To: 'John Boyer'; IETF/W3C XML-DSig WG (E-mail)
Cc: Martin J. Duerst; James Clark; Joseph Reagle; Eastlake
Donald-LDE008; TAMURA Kent; Christopher R. Maden; Ed Simon
Subject: RE: Enveloped signatures and XPath


name() on an element or attribute gives you back a QName for the element.
The prefix can be extracted from that.  The local-name() or name() of a
namespace node gives you a prefix as well.  Since this information is
available through XPath, it is reasonable to assume that an XML parser able
to expose the XPath data model retains adequate prefix information to inform
the serializer.

> -----Original Message-----
> From: John Boyer [mailto:jboyer@PureEdge.com]
> Sent: Monday, March 27, 2000 4:37 PM
> To: Jonathan Marsh; IETF/W3C XML-DSig WG (E-mail)
> Cc: Martin J. Duerst; James Clark; Joseph Reagle; Eastlake
> Donald-LDE008; TAMURA Kent; Christopher R. Maden; Ed Simon
> Subject: RE: Enveloped signatures and XPath
>
>
> Hi John,
>
> If I have the following markup,
>
> <xyz:e xmlns:xyz="URI"></xyz:e>
>
> How do I ask XPath for the xyz part of e?  I seem to be able
> to find out
> that the element has a local name of e and a namespace uri of
> URI, but in
> order to serialize the element, I need to have the xyz.  What
> function or
> node tells me this?
>
> John Boyer
> Software Development Manager
> PureEdge Solutions, Inc. (formerly UWI.Com)
> Creating Binding E-Commerce
> jboyer@PureEdge.com
>
>
> -----Original Message-----
> From: Jonathan Marsh [mailto:jmarsh@microsoft.com]
> Sent: Monday, March 27, 2000 3:18 PM
> To: 'John Boyer'; IETF/W3C XML-DSig WG (E-mail)
> Cc: Martin J. Duerst; James Clark; Joseph Reagle; Eastlake
> Donald-LDE008; TAMURA Kent; Christopher R. Maden; Ed Simon
> Subject: RE: Enveloped signatures and XPath
>
>
> > -----Original Message-----
> > From: John Boyer [mailto:jboyer@PureEdge.com]
> >
> > <john>
> > Right.  The exact point I'm trying to make is that
> > information needed for
> > serialization (specifically, the namespace prefix) is not publically
> > available.
>
> Yes it is.  It's in the original document as attributes or namespace
> information items, and is exposed through the XPath data
> model as namespace
> nodes.  Since you own the document passed to XPath, there is
> nothing it
> knows that can't be derived publicly.
>
> >  However, we can deduce from the XPath
> > specification that the
> > required information (e.g. namespace prefix) is available
> > privately to the
> > XPath evaluator.  It must be available since otherwise the
> > XPath evaluator
> > would be unable to do anything with the initial namespace
> declarations
> > created during evaluation context initialization.
>
> It is available to the XPath evaluator because it is in the
> document.  It is
> passed to the XPath evaluator through the context node.  You
> still have this
> information outside XPath.
>
> > Since the required information is available internally, AND
> > certain existing
> > XPath function must be able to access that information, we
> > can deduce that
> > the information is available to XPath functions.  Hence,
> > since serialize()
> > is defined as an XPath function, it must be able to access namespace
> > prefixes and document order positions, whether or not they
> > are in the public
> > data model exposed to XPath expression authors.
> > </john>
>
> The serialize function has available what is passed into it
> as arguments.
> If it is a nodelist, the function can follow these pointers
> back to the
> original document, and get this information.
>
> > > Also, I think that serialization is the one thing that you
> > > would not expect
> > > an XPath engine to provide (unlike parsing).
> >
> > I do not expect an XPath engine to provide parsing.
> > Microsoft's doesn't, as
> > an example.
> >
> > <john>Microsoft has an XPath engine.  Firstly, tell me where
> > it is so I can
> > look at it.
>
> IE5 shipped with an "XSL Pattern" engine in MSXML - see the
> selectNodes and
> selectSingleNode DOM extensions.  The latest MSXML technology
> updates this
> to an almost complete subset of XPath.  Docs and downloads
> can be found at
> http://msdn.microsoft.com/xml.
>
> > Is it open source?
> No.
>
> > Second, regardless of whether it provides a
> > parser in the concrete sense, it must be coupled with a
> parser in the
> > abstract sense.
>
> Indeed XPath evaluation is a service provided through DOM extensions,
> specifically as methods on DOM Node objects.  The inverse
> would also be
> possible - an XPath object that was given various context
> nodes.  Your spec
> should not unduly constrain the use of either of these types of
> implementation.
>
> Which XPath processor are you using to evaluate this design?
> I'm really
> only familiar with a small class of processors - Microsoft,
> Oracle, and to a
> lesser extent others within XSLT implementations.
>
> >  You must initialize the evaluation context, you must
> > provide a context node, position and size, the context node,
> > position and
> > size together form a node-set, and the XPath evaluator must
> be able to
> > derive from that node-set the namespace prefixes and the
> > document order
> > positions.  Regardless of where you got the processor, you
> have had to
> > create enough data structure SOMEHOW to support the described
> > operations.</john>
>
> Yes, which brings up a nasty point I hadn't realized before.  Our
> implementation does not currently allow you to pass in any
> function bindings
> (except when used within XSLT).  So I wouldn't be able to build a full
> implementation of XPath transforms on MSXML without some
> pretty painful
> hackery.  It would be much easier if no extension functions
> were needed.  I
> wouldn't put too much weight on this, but it may be evidence
> of a class of
> simple XPath processors that couldn't be used for XPath transforms.
>
> > Likewise, my XML processor may indeed provide a serialization
> > feature.  Microsoft's does, for example.  I'm not able to use
> > the existing
> > functionality that may be provided by a processor because a specific
> > implementation strategy has been prescribed - the serialize()
> > function.
> > Now, I don't believe the Microsoft serializer IS quite
> > adequate because your
> > constraints are too severe - but that's a separate potential issue.
> >
> > <john>If your serialization function were adequate, then I
> > wonder why you
> > can't call it from your serialize() implementation?
> > As to its adequacy, if the tool doesn't work, then don't use
> > it.  You can
> > only reuse code in  a new application if it fits the
> > application.  However,
> > the serialization described in the spec is QUITE easy to
> > implement, so I
> > cannot fathom why it would be a 'potential issue'.
> > </john>
>
> I will defer this to another thread.
>
> > As I indicated previously, I would not expect an XPath
> > implementation to be
> > optimized for returning the huge strings that could result from the
> > serialize() function.  And as I indicated previously, the
> > necessity/danger
> > of user intervention on the serialized string has not been
> > justified.  Even
> > the necessity of a two-step process of serializing and then
> > converting to
> > UTF-8 gives me pause for thought - it looks like I might need
> > two separate
> > serialize() functionalities to obtain maximum performance.
> >
> > <john>
> > The need for serialize() has been justified on several
> > levels.  Firstly,
> > there is no security threat in allowing user intervention on
> > the serialized
> > node-set.
>
> My term "dangerous" was clearly a poor one in this context
> :-)  I meant not
> that a security risk was imposed, but that it would be quite
> easy to damage
> the well-formedness of the XML document.  Manipulating XML
> through string
> manipulation is unquestionably more prone to error than
> manipulating it
> through structural methods.  You can tell the user "well,
> you're stupid and
> you broke it" but unless there is a compelling reason to
> ALLOW him to break
> it, it seems self-evident that it's best not to.
>
> > For you to argue otherwise would require that you
> > explain why it
> > is significantly different from writing a different XPath
> > expression. The
> > XPath expression is signed, so its modification of the
> input follows a
> > well-known behavior that we can prove is the correct behavior
> > leading to the
> > given message digest.
> >
> > Secondly, I've justified it from the specification
> > standpoint.  By defining
> > the behavior as a function library addition to XPath, the
> > result of the DSig
> > WG's work can continue to be viewed as an application of
> > XPath, not as an
> > extension of XPath.
>
> XPath transforms is an application of XPath which is
> comprised of several
> parts:
> - parsing the document from a stream
> - identifying the nodes to keep/trim using an XPath
> - trimming the tree
> - serialization of the result
>
> Note that the last two are described as coincident by the serialize()
> function.  This is a perfectly good application of XPath.  Providing
> serialize() does not make it a better application of XPath,
> any more than
> removing it would make it less of an appropriate XPath application.
>
> If somehow it were shown that the serialize() function is
> necessary to make
> XPath transforms an application, what about the case when it
> is performed
> automatically?  Does somehow this use constitute an XPath
> extension?  The
> logic of your assertion totally escapes me.
>
> > Thirdly, I've justified it on technical grounds.  There is
> information
> > required for serialization that is not available in the
> > public information
> > but which MUST be available to the implementations of certain XPath
> > functions.
>
> > Please have another look at how XPath handles the namespace
> > declarations in
> > the evaluation context.  If I have misread something, please
> > let me know,
> > but I did ask for clarification on this point, and noone
> > (including James
> > Clark) has refuted this point yet.
> >
> > As for two serializations, that's not happening.  The first act is a
> > 'serialization' of the node-set.  The second action is a character
> > transcoding to UTF-8.  My original assumption was that
> > implementations would
> > not lose track of the input document's encoding.  This was
> > based on the fact
> > that I specified a parse() that would do this.  You
> > specifically along with
> > others wanted to get rid of the parse().  The natural result
> > is that I can
> > no longer claim that the input document stays in the same
> > encoding.  Since I
> > also lose the BOM, UTF-8 is the only viable alternative.
> >
> > serialize(), however, cannot output UTF-8 because it is an
> > XPath function,
> > which must output a string.  The details of what that means
> > are specific to
> > the XPath implementation itself.  Transcoding to UTF-8 is not
> > an action that
> > fits within XPath.
> > </john>
>
> My concern here is that if serialize() is called automatically, I can
> envision a single step in which the document is serialize
> directly to UTF-8
> as a performance gain.  When serialize() is called
> explicitly, it looks like
> a two-step process is needed - serialize into a string to be
> returned from
> the XPath engine (presumeably Unicode) and then a conversion
> to UTF-8.  I'm
> not sure if this is a real issue, but it does appear that there may be
> difference in optimization potential and thus performance
> between the two
> uses.  Users would need to be aware that explicit use of
> serialize() may be
> slower than automatic use - another small argument against
> including it.
>
> >
> >
> > >   So another
> > > justification for
> > > adding serialize() as a function is that we are adding DSig's key
> > > functionality using the XPath-defined extension mechanism
> > > (function library
> > > modification).  This maintains the property that dsig is an
> > > application of
> > > XPath, not something different that happens to use the XPath
> > > expression
> > > syntax.
> >
> > I still don't understand this argument either.  If the
> > serialize function
> > can be called automatically for nodesets as you prescribe,
> > this would seem
> > to indicate that there is not a necessity to expose it to the
> > user in all
> > cases.  If you are not using the prescribed XPath extension
> > mechanism in all
> > cases, how does this contribute to maintaining "the property
> > that dsig is an
> > application of XPath"?  The output of the XPath processor is
> > not the end
> > result of the transform in all cases.  Doing more or less
> > back-end work is a
> > pretty abstract distinction.
> >
> > I feel that it is already an appropriate application of XPath
> > to locate
> > nodes in an XML document.  Going beyond this to serializing
> > within the XPath
> > expression itself doesn't contribute to the impression that
> > dsig is an XPath
> > application.  In fact, my gut feeling is that it is abuse
> > because it appears
> > to short-circuit the primary XPath return type of nodeset.
> >
> > <john>The specification very clearly points out that the
> > XPath can return a
> > node-set.  However, to run a digest algorithm, we need a
> > string.</john>
>
> This is true but doesn't address my point.  I think it's
> great that a user
> can return a nodeset and it'll get serialized automatically.
> In fact, it's
> so great I can't see why anyone would ever need to bother with calling
> serialize() explicitly.
>
> > The redundancy argument still applies.  I can call the
> > serialize() function,
> > or not, with the same result.  Without some use cases
> > justifying the need
> > for the long form, the short form looks perfectly adquate to me.
> >
> > Note that there is no standard mechanism defined by XPath to "call"
> > extension functions outside of the XPath expression.  The XSLT spec
> > carefully uses descriptions like "the resulting object is
> > converted to a
> > string _as_if_ by a call to the string function" (emph
> > added.)  A similar
> > approach would be appropriate in describing XPath transforms.
> >
> > I don't see a concrete problem that serialize() solves.  I
> do see some
> > concrete reasons not to have it - performance, implementation
> > flexibility,
> > eliminating redundancy, simplified syntax.
> >
> > <john>Sure, as long as you can guarantee that the moment we
> > yank serialize,
> > we aren't going to have half a dozen different people writing
> > in to say that
> > "my-favorite-XPath-engine doesn't have access to namespace
> > prefixes" or
> > "my-favorite-XPath-engine makes it really inefficient to do
> > document order".
>
> First of all, I can't guarantee people won't complain.  I can't even
> guarantee their complaints won't be justified.  I only can
> state that I'm
> not aware of any implementations that would have problems
> with this, and
> it's hard to imagine how they could and still conform to
> namespace nodes in
> the XPath data model.  You have not previously presented
> evidence of such
> implementations or even their potential.  My goal is to
> enable a wider range
> of XPath engines to be used, and I would welcome more details about
> constraints on specific implementations.
>
> > Also, I do like the idea of being able to tweak the data in
> > certain ways
> > once it has been serialized.  One could wrap a root element
> > around a list of
> > elements, one could prepend some simple material to make the
> > result of the
> > transform more usable to certain software.
> > </john>
>
> Structural manipulations like this should be left to XSLT,
> which is designed
> to perform exactly this type of operation without damaging
> well-formedness
> or the meaning of the document.
>
> In summary, I still haven't found even one of your arguments
> for exposing
> serialize() to the user compelling.  Since we aren't converging on
> solutions, perhaps we should focus instead on converging on a clean
> statement of our different positions?  There appears to be
> very little in
> the way of hard technical arguments either way, so a win/loss
> outcome seems
> unlikely.
>

Received on Tuesday, 28 March 2000 12:46:30 UTC