Re: Leaky abstractions, protocols and all that.

David Orchard wrote:

> David,
>
>  
>
> I believe we are largely saying the same thing, which is good.  To use 
> your terminology, underlying protocols may or may not have a back 
> channel, and this presence or absence will affect the SOAP meps, WSDL 
> meps, and WS-A Fault configurations.  
>
Yes.  See also 
http://lists.w3.org/Archives/Public/public-ws-addressing/2005Mar/0171.html 
and 
http://lists.w3.org/Archives/Public/public-ws-addressing/2005Mar/0170.html.

> Please note I am using the term "underlying protocol" rather than 
> transport, because this is the heart of the matter.  HTTP provides a 
> back channel - often called a connection - as well as operations, 
> extensible headers, response bodies, caching features, etc.  If it is 
> treated as a "transport", then I believe pretty much all the 
> capabilities, including back channel, are ignored.  Now we can get to 
> a middle ground between "in-only" soap mep and HTTP, which is where I 
> think you and I agreeing.  The middle ground is to accept that there 
> are a subset of underlying protocols that have capabilities in common 
> that we are interested in, such as a connection.  The abstraction we 
> want is to the connection-oriented vs connectionless protocols.
>
I wouldn't term it connection-oriented vs. connectionless.  In an email 
binding that requires at least one of reply-to: and from: always to be 
present, there is a perfectly good back-channel, but no connection.  
There are analogs of this in the MOM world as well.  In any case, yes,  
it's the presence of a back-channel we're trying to capture.

>  
>
> I should be clear that I do not believe that a protocol "fault" 
> without an accompanying SOAP Fault body is-a SOAP Fault.  Therefore an 
> HTTP 4/5xx without a SOAP Fault does not get transferred to the 
> application via the SOAP layer.  I think that would be a very useful 
> thing to do, that is treat a protocol Fault as a synthetic and 
> standardized SOAP Fault.  But that is separate from what I am talking 
> about, which is the treatment of actual SOAP Faults on the wire.
>
I'm not completely sold on that (see below), but I do see the attraction.

>  
>
> On to the meat of things, which is my 3 points about leakiness.  I 
> can't really tell if you are agreeing or not.  It almost seems like 
> you are agreeing and then making some corrections/friendly amendments, 
> but then other times not.
>
>  
>
> 1. I believe that we do already distinguish between transport faults 
> and faults from faults sent by the receiver, namely that the former is 
> not modeled in wsdl or soap, and the latter may be modeled as a SOAP 
> Fault.
>
Right.

> 2. I think you are agreeing with me.  I use "the soap mep leaks into 
> the wsdl mep" and you say "take into account".  To me, the "taking 
> into account" is the leakage.  So everywhere I say "leak" you 
> translate to "take into account" we should be good.
>
<philosophy>

The pieces I've read on "leaky abstractions" seem to take the view that 
/no matter what/ abstraction you choose, there will be some inconvenient 
fact of life that "leaks" through it and can't be handled by the 
abstraction.  From what you've said, I don't think this is the sense in 
which you are using the phrase, and this is probably a source of 
whatever confusion there may be.

My point -- which I believe you agree on -- is just that it makes a 
difference what abstraction you choose.  The right abstraction handles 
inconvenient facts of life /within/ its framework.  Thus "TCP is 100% 
reliable transport" is a different abstraction from "TCP is a mostly 
reliable transport and it will tell you when an unrecoverable failure 
happened."  The first one "leaks" in what I take to be the usual sense, 
while I claim the second one doesn't (at least not as far as reliability 
is concerned).  What I think you're saying is that "you can't pretend 
TCP is 100% reliable" is an example of reality leaking through -- if you 
model unreliability explicitly, that in itself is a leak.

What I'm trying to chip away at here is the notion that "abstraction" 
means "idealized notion of a perfect world".  Abstraction is just 
explicitly modeling what you are interested in and ignoring what you're 
not interested in.  The common ground between the views is that if 
you're interested in (say) transport on real live networks, you /have 
to/ be interested in network failures.  To me, a TCP connection closing 
is an abstract event.  You can know the connection closed without 
knowing whether the network was flooded, your counterpart crashed or 
some lonely squirrel somewhere took out a key transformer in a blaze of 
glory.

</philosophy>

> 3. Here, I really can't tell whether we agree or not.  I roughly 
> proposed that FaultTo support SOAP Faults transmitted on a 
> back-channel AND a discrete address for any protocol.  You agree that 
> there is a visible difference, but I can't tell whether you want the 
> FaultTo to support this "two-step" SOAP Fault transmission model.
>
I'm not sure I'm parsing that.  I believe this is in the context of a 
SOAP/HTTP message with FaultTo: set to a callback, where the server 
produces (say) a 404.

To me, this is a transport fault.  The sender gets back "couldn't 
complete the operation" with a spiffy leather pouch attached.  Inside 
the pouch is more detail, including the 404 code and whatever else is 
known.  This is independent of application faults.  For example, the 
server could receive the request just fine, examine it and successfully 
send a fault to the FaultTo: address, then crash, not having closed the 
socket the original request came on (imagine the two are in different 
threads and the scheduler hiccups before the crash).  The client sees a 
transport fault, but the application fault arrives just fine.

Conversely, the server could determine that an application fault had 
occurred, send back a 204 to the client successfully and close the 
connection, then fail to deliver the fault to the callback.  In a fully 
asynchronous world, the client has to be prepared for both cases.  Given 
that the transport situation is "nuanced", it seems best just to treat 
application faults as just another message and keep them well separate 
from transport problems.  Either of the situations above can just as 
well happen with reply messages, or any variant.

>  
>
> Cheers,
>
> Dave
>
>  
>
> ------------------------------------------------------------------------
>
> *From:* public-ws-async-tf-request@w3.org 
> [mailto:public-ws-async-tf-request@w3.org] *On Behalf Of *David Hull
> *Sent:* Friday, April 08, 2005 2:21 PM
> *To:* public-ws-async-tf@w3.org
> *Subject:* Leaky abstractions, protocols and all that.
>
>  
>
> This is the promised commentary on Dave Orchard's blog entry 
> (http://www.pacificspirit.com/blog/2005/04/05/underlying_protocol_is_a_completely_leaky_abstraction) 
> in the context of the proposed rules I gave for WSDL and SOAP MEPs.
>
> First, since leaky abstractions constitute one of the main themes, 
> here's my take on the matter.
>
> I think the basis of the "leaky abstractions" meme is that you can 
> never pretend that the hardware isn't there.  You can't pretend that 
> numbers, even bignums, have infinite precision.  You can't pretend 
> that the network is reliable (see Deutsch's 8 fallacies of distributed 
> computing).  You can't pretend that HTTP is the same as UDP, and so forth.
>
> The wrong conclusion to draw from this is that, since the familiar 
> "pure" abstractions of mathematics don't cover the full range actual 
> computing machine behavior, all abstractions are inherently "leaky" by 
> virtue of being abstractions.  The right conclusion to draw from this 
> is that the abstractions we use have to account for the facts of life 
> in the computing world.
>
> TCP is an excellent and successful example.  TCP sits on top of mostly 
> one-way protocols that can be noticeably flaky in practice and 
> provides a bidirectional channel that fails seldom enough that 
> failures can be treated as exceptional conditions.  It does /not/ 
> provide a 100% reliable protocol and does not purport to.  Trying to 
> treat a TCP connection abstractly as a 100% reliable connection fails 
> not because all abstractions leak, but because the abstraction is not 
> accurate.  Treating a TCP connection abstractly as a bidirectional 
> channel that may fail under exceptional circumstances is reasonable 
> and accurate.
>
> In my view, a model that treats a TCP connection as a bidirectional 
> channel that may fail but usually doesn't is an abstraction, but is 
> not leaky.  It's an abstraction because of all the details I don't 
> care about.  For example,  I don't care what kind of physical 
> transport is in use, or even whether messages are going out over a 
> physical wire at all.  If there's a failure and the connection closes, 
> the main application doesn't care what caused the connection to close.
>
> The model is not leaky because it accounts for all the TCP behavior 
> we're interested in here.  Granted, if there's a transport error,  the 
> error-handling code /will/ generally care what's going on under the 
> covers, but I don't see this as a leak.  Different modules may see 
> different abstractions.  Setup and error recovery are a classic cases 
> of this -- they generally see a lot more of the physical detail than 
> the rest of the application.
>
> If you consider including transport faults in a model or having setup 
> and recovery code know more about the physical details as leaks, then 
> we just have a different notion of "leak", and that's fine as long as 
> we take care to be clear.
>
> Now to the analysis of WSDL MEPs and underlying protocols.
>
> I believe that the points about status codes in one-way over HTTP and 
> similar concerns are well taken, but I also believe they can be 
> adequately handled by modeling two basic facts:
>
>     * All transports can produce transport faults.
>     * Some transports have built-in back-channels and some don't.
>
> Again, these are both abstractions and as far as I can tell neither of 
> them leaks.  When the main application code tries an operation, it 
> must be prepared for a transport fault.  When such a fault happens, 
> all it knows is that something exceptional happened and a message that 
> was expected to be delivered is known not to have been delivered.  It 
> can then take action, perhaps handing off the fault to a recovery 
> module that can put on its overalls and dive under the hood, toolbox 
> in hand, and try to repair the condition or report that it can't be 
> repaired.
>
> Similarly, the infrastructure implementing WS* will use some form of 
> the various rules we've discussed, together with its knowledge of the 
> binding, to try to realize the desired WSDL MEP at the SOAP transport 
> layer.  To do this, it needs to know if a back-channel is required and 
> if so, whether the transport in question provides one.  It /doesn't/ 
> need to know any physical details like which HTTP codes map to 
> results, transport faults or successful MEP completion without either.
>
> As to the myth of transport independence, I believe there is a 
> spurious all-or-nothing dichotomy lurking in the background, namely 
> the notion that either all transports are treated uniformly, or they 
> all must be treated case-by-case. I don't think anyone would advocate 
> such a position put that baldly, but I think such an assumption or one 
> of its cousins may be sneaking in through the back door.
>
> We can't pretend all transports are equal; some provide back-channels 
> and some don't.  We also can't forget entirely about transport-level 
> details; setup, error recovery and (of course) certain parts of the 
> infrastructure code have to know quite a bit about them.  On the other 
> hand, we can and should abstract large amounts of transport detail 
> away from the 90% code path of applications by adopting a model that 
> takes transport faults and back-channels into account without going 
> into detail about them.
>
> Reading over, I see a possible disconnect concerning what I'm calling 
> a "transport fault".  I believe Dave is modeling, say, a 5xx status 
> return as a fault message being returned on the back-channel.  I'm 
> modeling it as a transport fault and not worrying about where it came 
> from.  As far as I'm concerned, it's coincidence that the 5xx comes 
> back on the same TCP channel that a reply (or application fault) would 
> have.  I /only/ care about the [* endpoint] when the transport is 
> working -- the receiver has received the inbound message, thought 
> about it, and successfully sent back an outbound message, whether 
> reply, fault or something more exotic.
>
> I don't think I've covered everything in the blog entry here; there's 
> quite a bit of meat to it, but I hope that this helps clarify how the 
> proposals I've put forth may relate to it.  Just as a cross-check, I'd 
> like to specifically comment on the list of leaks that Dave gives near 
> the end:
>
>    1. The specific binding leaks into the SOAP mep selected, because
>       HTTP bindings match up well with a to-be-standardized (TBS) SOAP
>       in-optional-out to support faults on the HTTP connection
>    2. The specific binding leaks into the WSDL mep selected because
>       the WSDL mep will logically match up with the SOAP mep
>    3. The specific binding leaks into the WS-Addressing Fault, because
>       an HTTP binding will suggest Faults can be returned on the in
>       HTTP connection AND a separate connection for the FaultTo.
>
> In-reply-to this (using a numbered list for correlation :-)
>
>    1. I believe this is captured by noting that bindings may or may
>       not support back channels, by distinguishing transport faults
>       from faults sent by the receiver, and by asserting that all
>       bindings are liable to produce transport faults.
>    2. When mapping a WSDL MEP to SOAP, you have to take into account
>       which SOAP MEPs the binding supports (i.e., whether there is a
>       back-channel available), as detailed previously, and you may
>       have to complain if the operation needs a back-channel that's
>       not there.  That appears to be all you have to take into account.
>    3. This is where the distinction between transport faults and
>       application faults is important.  Yes, some codes that manifest
>       as transport faults may come back on the same back-channel that
>       may or may not also be intended for application faults.  But
>       this doesn't really matter.  I don't see that HTTP needs to be
>       treated differently from other transports in this respect.  More
>       broadly, I don't see a visible distinction between transports
>       that may send fault codes back on the back channel and those
>       that never will.  By contrast, there /is/ a visible distinction
>       between transports that have back-channels and those that don't
>       (see above).
>
> Whether to classify any of the above as "leakage" is, IMHO, more a 
> matter of terminology than anything else.
>
> In any case, my feeling is we're getting somewhere, and I'm happy 
> about that.
>