Re: The role of transfer protocols

I'm having more and more trouble understanding what you might be
objecting to.

I'm mainly putting forth two fairly simple and (I would hope)
uncontroversial points:

    * In cases where completing an operation involves two sub-operations
      arbitrarily separated in time, it's not enough to know that the
      first operation completed.  If one needs acknowledgment of
      completion of the two-part operation, that acknowledgment has to
      be yet a third operation.
    * Sending streams of, say,  market data around is qualitatively
      different from, say, maintaining an editors' draft of a
      specification, even though bids and such may be viewed as
      documents and changes to an editors' draft may be viewed as events.

Rewinding back to the beginning of the thread, your objection seems to
be to treating a "transfer protocol" as a "transport protocol," but I'm
having a hard time understanding what particular sin that entails.  For
example, a particular scenario I mentioned involved my sending a message
to a forwarding agent, the recipient retrieving that message arbitrarily
later, and my receiving an acknowledgment.  This seems harmless enough,
and I'm afraid I still don't see how it becomes more harmful if one or
more of those three steps involves HTTP.  Will incorrect behavior
result?  If so, what and how?

I would also like a bit of clarification as to what sort of "state"
you're focusing on, as "state-transfer" could apply to any distributed
system at all depending on what you choose to call the "state".  In the
market data example, what state would you say is being transferred?

It's quite possible that we are largely in what an old colleague of mine
would called "violent agreement".  Certainly I would respond to several
of the responses below with "Right, didn't I just say that?".  However,
rather than go that quite probably tedious route, I would prefer to
spend more effort understanding just where we stand and how that plays
out in terms of moving SOAP envelopes and other data around with various
protocols.

Mark Baker wrote:

>On 1/6/06, David Hull <dmh@tibco.com> wrote:
>  
>
>> In the solutions you outline below, what acknowledgment, of what, do I get
>>when?
>>
>> What I want is acknowledgment that my counterpart on the intermittently
>>attached device has received the message I sent (for bonus points, that it
>>has received it and done something meaningful with it).
>>
>> If I understand your terminology correctly, using HTTP as a transfer
>>protocol would mean that I would not get back an HTTP response until my
>>counterpart actually received the message, arbitrarily later.  Similarly,
>>SMTP would not send back an OK message until the counterpart actually
>>received the message.  As I understand it, this is at best bad practice with
>>HTTP and simply not what SMTP does at all (if I want to know that someone
>>actually got my email, I ask for a return receipt).
>>    
>>
>
>Right, that's not what I was saying.
>
>I actually didn't describe a single solution, just outlined some options.
>
>But one solution would be to setup an HTTP gateway for the
>intermittently connected device.  Documents would be POSTed to the
>gateway, and then the device, when it came online and felt up to it,
>would invoke GET to retrieve the documents.  Since GET is safe, the
>gateway shouldn't send acknowledgements to the document sender as a
>result of that action, but instead the device could send the
>acknowledgements after successfully retrieving the documents.
>
>  
>
>> The general point is that that *TP can only tell me that something was
>>transferred to the server on the other end of the connection.
>>    
>>
>
>Not necessarily, as it depends on the protocol.  SMTP, for example, is
>hop-by-hop, so when you send an email and receive a 250 response, all
>you know is that that node/hop got it, not that the recipient got it.
>
>HTTP has both hop-by-hop and end-to-end features, and can be used in
>different configurations too to give it the properties you desire. 
>The traditional proxy configuration (e.g. firewall, cache, ...) would
>behave as you describe, but a gateway configuration as described above
>can be used to further decouple sender and recipient.
>
>But whatever; even if you can't find an existing TP that suits your
>needs, you can always develop a new one.  My objective with this
>thread is not to convince you to use HTTP or any other TP, it's just
>to explain the role of transfer protocols in relation to transport
>protocols and the assumed Web services architecture.
>
>[snip]
>  
>
>> Alternatively, you could define each bid, offer or sale as a document in
>>and of itself.  IMHO, this is stretching the notion of document far beyond
>>any useful point.  Documents simply live in a different design space from
>>notifications.
>>    
>>
>
>I don't think that's any stretch at all.  If I showed anybody an XML
>document which represented a bid, I doubt I'd find one person who'd be
>surprised at hearing it called a document.
>
>  
>
>> Document exchange systems are generally aimed at moving reasonably large
>>hunks of data around with a premium on reliability and without great regard
>>for short and predictable latency.  Event notifications are typically quite
>>small.  Something like "machine 3 just caught fire" or "Goldman bids
>>$20.25/share for 1000 shares of FooCorp" manifestly takes no more than
>>dozens of bytes even without any attempt to remove redundant information.  A
>>carefully designed system can deliver dozens of notifications in the packets
>>it takes TCP to shake hands (I've seen it done :-).
>>    
>>
>
>It seems "document exchange" is overloaded with you, so let's avoid
>it.  Perhaps we can use the more precise and generic name "state
>transfer".  As I see it, both EDI-style bulk-data-exchange based
>systems, and event notification systems, are both state transfer based
>systems.
>
>  
>
>> Guaranteed delivery of notifications is not always appropriate.  It most
>>likely would be for "machine 3 just caught fire", but often with market data
>>"old news is no news" and it's better to deliver the most recent information
>>quickly than to make sure that old information gets there.  Similarly,
>>latency and predictability thereof matter in real-time notification systems.
>>    
>>
>
>Absolutely.
>
>You might be interested in Rohit Khare's dissertation, which describes
>some architectural styles (REST extensions, interestingly) which are
>intended to address issues like you describe, amoungst other related
>ones (multi-agency, concensus, ...);
>
>http://www.ics.uci.edu/~rohit/DecentralizingREST.pdf
>
>Mark.
>
>  
>