RE: Different Levels of Reliable Messaging - Intermediaries

Thanks, David.

Why does "EACK" need a following "IACK" ?  I think "EACK" is affordable to 
be lost.

Rgds, Ricky

At 11:26 AM 12/14/2002 -0800, Burdett, David wrote:
>Ricky, I don't mean this. The message flow you describe is what could be 
>called a "publishing" protocol where you are sending one message to 
>multiple recipients. In this case, the intermediary is not simply 
>forwarding the message to its ultimate destination. There was endless 
>debate on this when developing the ebXML spec. The conclusion was that 
>intermediaries, from a RM perspective, JUST forward the message perhaps 
>changing the protocol and wrapper, they do not change or alter the 
>semantic meaning of the message at all.
>
>To help this is what I would see as being the typical flow
>
>A        B          C         D
>M-------->
><-----IACK
>           M-------->
>           <-----IACK
>                     M-------->
>                     <-----IACK
>                     <-----EACK
>                     IACK----->
>           <-----EACK
>           IACK----->
><-----EACK
>IACK----->
>
>Note that in this example there are intermediate acks on the end-to-end ack.
>
>David
>-----Original Message-----
>From: Ricky Ho [mailto:riho@cisco.com]
>Sent: Saturday, December 14, 2002 12:13 AM
>To: Burdett, David; www-ws-arch@w3.org
>Subject: RE: Different Levels of Reliable Messaging - Intermediaries
>David, can you verify my understanding of what you mean by Intermediary 
>ACK and End-to-end ACK ?
>
>Lets say A sends message M to B (an intermediary), which forwards M to 2 
>destinations C and D.
>"IACK" stands for intermediary acknowledgement, "EACK" stands for 
>end-to-end acknowledgement
>
>The following will describe the sequence of message flow.
>
>1) A --(M)--> B
>2) A <--(IACK)-- B
>3) B --(M)--> C;  B--(M)--> D
>4) B <--(IACK)-- C;  B <--(IACK)-- D
>5) B <--(EACK)-- C;  B <--(EACK)-- D
>6) A <--(EACK)-- B
>
>At step (2), A get an "IACK" from B that indicates B has received the 
>message M.
>At step (6), A get an "EACK" from B indicates ALL final destinations (A 
>may not know who they are) have received the message M.
>
>Is this what you mean ?
>
>Best regards,
>Ricky
>
>At 09:47 PM 12/12/2002 -0800, Burdett, David wrote:
>
>>Ugo.
>>One idea you suggested was that you could do end-to-end reliable 
>>messaging if each of the hops used reliable messaging. Although this is 
>>technically true, where it falls down is that the sender of the original 
>>message might not always *know* that there are multiple hops as perhaps 
>>the path being followed is determined dynamically.Even if they know there 
>>are multiple hops they might not know if each hop is reliable. So, 
>>although your suggestion can work in some cases, it won't always.
>>END-TO-END ACKS ARE THE ONLY GUARANTEE
>>--------------------------------------
>>So to be *sure* that a message has been delivered the Acknowledgement 
>>Message MUST come from the ultimate receiver of the message. However 
>>identifying the true "ultimate receiver" is not always obvious as you can 
>>argue that it should be the (first?) SOAP node at the destination or 
>>perhaps the application running behind it - this isssue is discussed later.
>>However doing end-to-end acks over multiple hops using different 
>>transport protocols can be problematical because of the different speeds 
>>of the transport protocol could be used in each hop.
>>DIFFERENT SPEEDS OF TRANSPORT PROTOCOLS
>>---------------------------------------
>>If you are doing a single hop then, since you can know the *speed* of the 
>>transport protocol being used from beginning to end, you can estimate how 
>>long it "should" be before the acknowledgement arrives. Then, if the 
>>response takes some time longer than this then you can resend the 
>>original message following the Level 1 Simple Reliable Messaging Approach 
>>described in my earlier email.
>>As said earlier, if you have multiple hops then this doesn't always work 
>>since each hop could involve a transport protocol that works at a 
>>different speed. For example the first hop could be HTTP and the next 
>>SMTP. What makes it worse, is that the sender of the original message 
>>might not even know that there is another hop and that it is going over SMTP.
>>CALCULATING THE TIMEOUT
>>-----------------------
>>The simplest way around this problem is to just do an end-to-end ack BUT 
>>base the value of the "timeout" to use on when the message expires as 
>>indicated by the "expires at". So, for example, if the original message 
>>had to be delivered within 1 hour, you might want to do a retry if the 
>>acknowledgement message had not been received within 10 minutes giving 6 
>>attempts to deliver the message before "giving up".
>>If you think about, the "expires at" HAS to be set to a value that allows 
>>sufficient time for the original message to be sent and the 
>>acknowledgement to be received for the message to be sent with any 
>>success at all. So this approach is pretty safe.
>>EARLY DETECTION OF NODE FAILURE
>>-------------------------------
>>However, what this approach does not do is provide early warning that an 
>>intermediate node has failed. For example if you are sending the message 
>>by HTTP, then you will not know, using the timeout value described 
>>earlier, that transportation of the message was impossible until you had 
>>exhausted all the 6 retries.
>>IF (note the capitals) this is a problem then the solution is to use 
>>intermediate acknowledgement messages in addition to the end-to-end ack 
>>where the intermediate ack is targeted (e.g. using the SOAP Actor) at the 
>>"Next" node in the network.
>>INTERMEDIATE ACKS
>>-----------------
>>Since the intermediate ack is targeted at the next node, you usually can 
>>use a timeout value that is based on the speed of the transport protocol. 
>>For example, if you are using HTTP, then you might want to retry after 1 
>>minute. If you are using SMTP, then maybe every 30 minutes. Then work out 
>>a reasonable number of retries to do before "giving up" say 3 or 4.
>>If you do this then you would know, again given the timeout values used 
>>above, that the message could not be sent after a much shorter time, e.g. 
>>3-4 minutes for HTTP.
>>However, the complexity does not end there. It's not much use if an 
>>intermediate node has detected that it cannot send the original message 
>>any further. The sender of the original message needs to know. This means 
>>tha the intermediate node that detected the delivery failure HAS to send 
>>another message back to the sender of the original message to notify them 
>>of the failure.
>>So what intermediate acks really provide is an optimisation that provides 
>>much earlier warning of delivery failure - albeit at a fairly large 
>>increase in complexity.
>>Also remember that if End-to-End acks are used as well as the 
>>intermediate acks then, if Level 3 - Reliable Messaging with Recovery, is 
>>used, it is quite possible that it could cause another retry at sending 
>>the message some time later.
>>OTHER FACTORS TO CONSIDER
>>-------------------------
>>However there are some issues with using intermediate acks which you need 
>>to think about:
>>1. You can get two acks not one - you can now receive more than one ack, 
>>the intermediate ack and the end-to-end ack which adds to the complexity
>>2. The Intermediate Ack *should* also be targeted at the next node in the 
>>network otherwise the sender of the original node would get an ack back 
>>from each and every intermediate node in the path
>>3. What do you do if the delivery failure message does not get through? 
>>Should you send the delivery failure message reliably ... and what do you 
>>do if that doesn't work either.
>>4. If the nodes in the network that are being targeted are SOAP nodes, 
>>then there might be other, non SOAP systems between the SOAP nodes that 
>>use different transport protocols at different speeds to connect. This 
>>makes calculating the timeout value for intermediate acks harder to do 
>>reliably.
>>ULTIMATE RECEIVER
>>-----------------
>>The other issue with end-to-end acks is knowing what actually is the end. 
>>If the last SOAP node, for example, in the path actually consumes the 
>>payload/body of the SOAP message then there isn't much of a problem. But 
>>if it does not, then the message could be lost after the SOAP node has 
>>passed on the message to the application that actually processes it. This 
>>is why the Acknowledgement Message described in the original email in 
>>this thread can optionally provide additional information on the 
>>validation and passing off of the message to the application.
>>BOTTOM LINE
>>-----------
>>I think that doing end-to-end acks where the timeout is based on the 
>>expiry time of the message is the best approach to use. Doing 
>>intermediate acks, only provides early warning of a problem, it isn't a 
>>substitute for end-to-end acks. So I would recommend that intermediate 
>>acks are left out of scope ... if this ever becomes an activity.
>>Thoughts?
>>
>>David
>>
>>
>>
>>
>>
>>-----Original Message-----
>>From: Ugo Corda [<mailto:UCorda@SeeBeyond.com>mailto:UCorda@SeeBeyond.com]
>>Sent: Thursday, December 12, 2002 5:56 PM
>>To: Burdett, David; www-ws-arch@w3.org
>>Subject: RE: Different Levels of Reliable Messaging
>>
>>David,
>>
>>I am glad you mention the case of intermediaries, because I have been 
>>thinking for a while about how that affects Reliable Messaging.
>>In the case of security, what is usually said in order to justify the 
>>need to address it at the SOAP headers level, instead of just using 
>>HTTPS, is that you need that level if you want to do end-to-end security. 
>>Otherwise intermediate nodes would need to have access to security 
>>sensitive information related to the message in order to relay 
>>information from hop to hop, and those nodes might not be authorized to 
>>do that.
>>But in the case of reliable messaging, it seems that you should be able 
>>to use, for example, SOAP over HTTPR on one hop, and SOAP over JMS on the 
>>next hop, and still be able to support reliable messaging end-to-end. 
>>(The message goes reliably from A to C iff it goes reliably from A to B 
>>and from B to C - for example, B waits until it gets the transport-level 
>>ack from C before sending its transport-level ack to A).
>>In fact, I think this was the rationale when IBM designed HTTPR, so that 
>>you could go from Internet to intranet (and vice versa), using SOAP over 
>>HTTPR on the Internet, and then switching to SOAP over MQSeries (or other 
>>MOM) once inside the intranet.
>>Your previous message seems to imply that this approach would not be 
>>sufficient for end-to-end reliable messaging. Could you please elaborate?
>>Thank you,
>>Ugo