RE: Proposed text on reliability in the web services architecture

I think Walden has made a good point. We don't really care about RM, which
itself has been solved in the TCP/IP layer or other messaging layer already.
The whole RM thing is misleading within Web Services context. What we really
care is a reliable way of coordinating a client and its server, although RM
might be helpful.

Making an operation idempotent seems to be a simple and effective solution.
What we need to distinguish
is the difference between logical operations and physical operations.

In the bank account example, depositing money into an account is a logical
operation while sending 
a request to the bank's Web Services to achieve such an operation is
physical. A logical operation can
be idempotent or not, but we do want all physical operations to be
idempotent.  More precisely, we mean:
1. r=f(u,x,t), where f is a physical operation on a Web Services identified
by u, x is a request, and t is time, and r is the results returned by the
WS.
2. r'=f(u,x',t')
3. if x eq. x', and r eq. r', then we say that f is idempotent, where eq.
stands for "is equivalent to".
This idempotent def is different than the one in math.

Is this summary good enough?

Hao



-----Original Message-----
From: Walden Mathews [mailto:waldenm@optonline.net]
Sent: Thursday, January 16, 2003 2:01 PM
To: Assaf Arkin; Peter Furniss; Champion, Mike; www-ws-arch@w3.org
Subject: Re: Proposed text on reliability in the web services
architecture



> Consider depositing funds. You can do that by:
>
> 1. Sending the end-state
> 2. Sending the pre-state+part end-state
> 3. Sending a delta (the amount to deposit)
>
> In order to perform a deposit all you need is to send the delta, you don't
> need to know what the pre-state is.

Deposit is, by definition, a non-idempotent operation.  Now, where
do you want to go with that?

If the deposit itself is a resource of interest, then you can use
idempotence to set the amount of the deposit.  This is one way
to reconcile the account.

If the deposit has no lasting value to you, and all you care about
is setting the account balance to the "right" figure (Quicken has a
feature like this for people who like to "try" to balance their accounts,
but don't want to put too much into the effort), then you could
idempotently set the balance, overriding deposit-based calculations.

In either case, an alternate but unattractive strategy would be to
issue increments against the amount in question until the right
value was converged upon.  This is the problem RM is focused
on, and that's the reason RM is the hard way to do what the
application really needs.

>
> In order to send the end-state you need to know more than the balance. In
> fact, when you make a deposit two states are changed: 1) the balance, 2)
the
> journal (where every change is recorded).

This is a matter of scope.

>
> So the end-state is a composition of the new balance and the journal which
> now contains a new record. It's not practical to send the full end-state.

No, and it shouldn't be necessary. The point is that if you care about the
entries in the journal, then you give them identity and acknowledge the
sovereignty of their state, which gives you a platform from which to use
an idempotent operation in that scope.

>
> However, you can send a partial end-state, just the balance plus a version
> number of the balance. That would allow the server to determine whether
the
> change has been done (based on the version number) and if not change the
> end-state by also adding a record in the journal.

Version numbers are like RM.  They're not what you really care about,
and they carry the potential to screw you despite your good intentions.
What happens to your versioned account when a deposit that's been lost
in the network for a week finally comes home?

> Here is my understanding so far.
>
> When you can use idempotent operations you should do so. But in some cases
> they are impractical and you are better off not doing idempotent
operations.

This overstates the case.  Your application should behave in a way
that makes sense.  You should not be forcing idempotent patterns on a
domain where they do not make sense.  A "deposit" is such a case.  It means
"here's some money to add to my account, whatever the balance was and
whatever it will become."

>
> If you are not doing an idempotent operation then you must perform the
> operation at most once. You actually plan for exactly once, but that's not
> practical either, you need to account for at most once possibility.
> Otherwise, I would not be balancing my checks at the end of the month, but
> even big banks are known to make mistakes.

If the bank erroneously introduced a withdrawal of $100 on your
account, transaction number 29707421, and you and the bank agreed
it was posted in error, then the operation "remove 29707421" fixes
the problem, and is idempotent.

>
> If you are sending a message and know whether the message has been sent
with
> 99.99% success rate (i.e. only in 0.01% of cases you need to somehow
resolve
> the situation) then you are fine.

Maybe yes; maybe no.  If that was the deposit of my paycheck and it got
lost, then I'm not fine.  Not all problems are modeled well by statistics.

>
> If you are sending a message and do not immediately know whether the
message
> has been sent, and the success ration is 90% (e.g. when you use IP
multicast
> or UDP or SMTP) then you need to resolve 10% of the time.

Yes, but WHICH 10%?  Which messages were received and which
were not?  How do we find out?

>
> You can reduce that to 0.01% (or something like that, my numbers are all
> made up) if you have some messaging layer that makes the messages
idempotent
> and keep resending until it gets close to 100% of success rate.

You're dreaming.

>
> So reliable messaging buys you:
>
> - Operation is idempotent or not idempotent, it works the same way
> - You can use a high latency non-reliable messaging exchange
> - The success rate gets as close to 100% as possible
> - The layer is reusable and works the same way for all applications
without
> complicating the application

You are approaching 100% fallacy here; perhaps some retries will
improve the odds.  ;-)

Walden