Proposal for new last call issue: Some unprocessed headers should stay

At the face-to-face on the West Coast at SAS, several us noticed and
began privately discussing what we now believe is a problem that needs
at least some attention in the specification.  Specifically, there is
a use case that more or less all of us have come to believe is
important, and for which the specification is at best ambiguous and at
worst incapable of meeting without republication.  Several of us have
come up with a proposed resolution that we believe to be
straightforward, low risk, and within the bounds of what we can
reasonably do without going back to last call. I am indebted to Mark
Jones for first pointing this out, and to numerous others including
Henrik and Marc Hadley for helpful discussion and comment.  Particular
thanks to Henrik for helping me to refine early drafts of this note.
The responsibility for the formulation in this note, and for raising
this now, is mine.

The Problem
-----------

Among the reasons that we have the "next" role is for headers that
carry information to be available to all downstream nodes.  One of the
simplest scenarios would be to have such a header that would travel
with the message, to be available to those nodes that "understand" the
header, and to be ignored and passed on by nodes that do not
understand it.  As a simple example, I might want to put in the header
indicating the time at which the message was originally sent. More
substantively, I might wish to put in a digital signature, to be
checked by those who care, and ignored but retained by others.

SOAP 1.2 as proposed cannot support these simple scenarios. Why?
Because section 2.7.1[1] makes clear that a forwarding intermediary
MUST remove from the message all headers targeted to it, and can
trigger reinsertion only by processing and understanding a header.  In
the case where you don't understand the header, you must remove it.
It's essential to note that we are considering the case of
mustUnderstand='false'; there is no problem with the 'true' case.

Approaches to Resolving the Problem
-----------------------------------

Discussion of this problem has been going on at a low level for
awhile, in part because some of us have been trying to decide whether
the existing specification can somehow be made to do the right thing.
Given that we are in last call, that would obviously be a desirable
approach.  So, here I briefly outline some of the approaches that were
considered, and then in the next section I propose one that seems to
some of us to be the best.

It is tempting to ask whether a user might be able to design specific
headers and/or roles that would meet the need.  In other words: if I
define a role and some headers to use with it, then I could say that
you wouldn't assume that role unless you knew the specification for at
least one of the headers, and that specification would implement a
feature to change the relay rules for that role.  My personal
conclusion is that this is possible in principle, but would not be
practical when one considers the way that most SOAP software will in
fact be built.  Reason: the core rules regarding the relaying of
headers are baked into section 2.7.1, and in practice will often be
implemented by general-purpose SOAP middleware.  The key point is that
such middleware will in general be written to treat all user-defined
roles identically.  Allowing particular roles to change the relay
rules requires that such middleware have pluggable handlers that key
on specific role names, and this is something that few of us
anticipate doing.

There is another alternative which is also coherent, and indeed is
more powerful than the one proposed below, but which seems to be
overkill and would probably take us back to last call.  That
alternative would be to introduce a new attribute for use on header
entries along the lines of relayIfNotProcessed='true'.  Thus:

        <soap:Header>
                <nrm:myHeader role="..any role you like..."
                                mustUnderstand="false"
                                relayIfNotProcessed="true">
                        ...
                </nrm:myHeader>
        </soap:Header>

This seems to work, and has the capability of working with any role.
It just seems like more power and complexity than we need to deal with
the specific use case.  If we were to get serious about this proposal,
there would be several details requiring refinement, but for now I am
just suggesting that this is not the direction to go.

Another possibility that has been raised would be to change the
default behavior to be: "leave in place any header entries that are
not processed".  Optionally, we could additionally define a
relayIfProcessed override if the group felt it to be worth the
trouble.  My impression is that some who have considered changing the
default rules like the idea, but some feel that it doesn't meet a need
to have headers that do indeed disappear when unprocessed.  For the
record, I quite like the idea, but (a) am not ready to take the lead in
pushing it in the face of any significant opposition and (b) would not
want to go back to last call if this were deemed a serious change--
I'm not sure it is.

A Proposal
----------

Anyway, having done the above analysis, several of us conclude that:

* The use case is important that we must find a way to meet it
* The draft as written doesn't give explanation of how
* We need a simple, minimally invasive fix to what we have

We propose one new builtin role to be called:
http://www.w3.org/2002/06/soap-envelope/role/relay .  In most
respects, this role would be like any other.  Any intermediary or
ultimate receiver MAY choose to assume this role.  The one significant
difference, and it is one that we believe has to be baked into the
specification from day one (I.e. would be an incompatible change if
made later) is that section 2.7.1 will be changed as follows:

<original>
Forwarding SOAP intermediaries MUST process the message
according to the SOAP processing model defined in 2.6
Processing SOAP Messages. They MUST also remove from
the SOAP message all SOAP header blocks targeted at
themselves, prior to forwarding, regardless of whether
these header blocks were processed or ignored.

In addition, forwarding SOAP intermediaries MUST also
obey the specification for the SOAP forwarding feature
being used. The specification for such a feature MUST
describe the required semantics, including the rules
describing how the forwarded message is
constructed. Such rules MAY describe placement of
inserted or reinserted SOAP header blocks. Inserted
SOAP header blocks might be indistinguishable from one
or more of the header blocks removed above.
</original>

<proposed>
Forwarding SOAP intermediaries MUST process the message
according to the SOAP processing model defined in
2.6 Processing SOAP Messages. In addition, when
generating a SOAP message for the purpose of forwarding,
they MUST:

* For any processed SOAP header block, as well as for
  ignored SOAP header blocks targeted to the node
  using a role other than
  http://www.w3.org/2002/06/soap-envelope/role/relay:
  remove the header block prior to forwarding

* Retain all SOAP header blocks that were targeted at
  the forwarding node using the role
  "http://www.w3.org/2002/06/soap-envelope/role/relay"
  but ignored during processing.


In addition, forwarding SOAP intermediaries MUST also obey the
specification for the SOAP forwarding feature being used. The
specification for such a feature MUST describe the required semantics,
including the rules describing how the forwarded message is
constructed. Such rules MAY describe placement of inserted or
reinserted SOAP header blocks. Inserted SOAP header blocks might be
indistinguishable from one or more of the header blocks removed above.
</proposed>

I wish to reiterate that I understand it is late in the last call
process, and that I would strongly prefer that we not do anything that
would delay our progress for weeks or months, just to deal with this
one concern.  That said, I think this is a significant use case that
deserves at least some consideration.  Thank you.

Noah

[1] http://www.w3.org/2000/xp/Group/2/06/LC/soap12-part1.html#forwardinter

------------------------------------------------------------------
Noah Mendelsohn                              Voice: 1-617-693-4036
IBM Corporation                                Fax: 1-617-693-8676
One Rogers Street
Cambridge, MA 02142
------------------------------------------------------------------

Received on Monday, 14 October 2002 20:31:28 UTC