RE: Proposed text on reliability in the web services architecture

> -----Original Message-----
> From: Walden Mathews [mailto:waldenm@optonline.net]
> Sent: Wednesday, January 15, 2003 7:01 PM
> To: Assaf Arkin; Peter Furniss; Champion, Mike; www-ws-arch@w3.org
> Subject: Re: Proposed text on reliability in the web services
> architecture
>
>
> > Consider depositing funds. You can do that by:
> >
> > 1. Sending the end-state
> > 2. Sending the pre-state+part end-state
> > 3. Sending a delta (the amount to deposit)
> >
> > In order to perform a deposit all you need is to send the
> delta, you don't
> > need to know what the pre-state is.
>
> Deposit is, by definition, a non-idempotent operation.  Now, where
> do you want to go with that?

Pointing out that forcing non-idempotent operations to become idempotent is
not always practical. Did I make that point even if I was totally redundant
in repeating everything that was said?


> If the deposit itself is a resource of interest, then you can use
> idempotence to set the amount of the deposit.  This is one way
> to reconcile the account.

That's just shifting the problem away. In fact that's more precisely how it
works. I've talked about it before with the check example. You cash a check
and that's idempotent because you can create that record any number of
times. But it's just a triger that starts a non-idempotent operation. So A
is doing idempotent operation causing B to do non-idempotent operation. In
other words, if it's non-idempotent it's non-idempotent you can mask it but
you can't make it disappear.


> If the deposit has no lasting value to you, and all you care about
> is setting the account balance to the "right" figure (Quicken has a
> feature like this for people who like to "try" to balance their accounts,
> but don't want to put too much into the effort), then you could
> idempotently set the balance, overriding deposit-based calculations.

I hope Quicken 2003 would let me do that to my bank account ;-)

> In either case, an alternate but unattractive strategy would be to
> issue increments against the amount in question until the right
> value was converged upon.  This is the problem RM is focused
> on, and that's the reason RM is the hard way to do what the
> application really needs.

I don't really see why RM would want to do that or tries to do that. I only
percieve RM as trying to address message loss and while you can do other
things with it. If it's intended to solve one specific problem I feel that
it should just be used to solve that specific problem.


> > So the end-state is a composition of the new balance and the
> journal which
> > now contains a new record. It's not practical to send the full
> end-state.
>
> No, and it shouldn't be necessary. The point is that if you care about the
> entries in the journal, then you give them identity and acknowledge the
> sovereignty of their state, which gives you a platform from which to use
> an idempotent operation in that scope.

The server cares about the entries in the journal. The client doesn't. Two
servers may have two different ways to use the journal. You want to decouple
the client from the server so you could use either server equally as well.
Think of it like checks, banks and the tapes on which they record all
transactions. You want to use a check and you want to not care how they
manage to record the transaction when the check is cashed.


> > However, you can send a partial end-state, just the balance
> plus a version
> > number of the balance. That would allow the server to determine whether
> the
> > change has been done (based on the version number) and if not change the
> > end-state by also adding a record in the journal.
>
> Version numbers are like RM.  They're not what you really care about,
> and they carry the potential to screw you despite your good intentions.
> What happens to your versioned account when a deposit that's been lost
> in the network for a week finally comes home?

Nothing. Go back to the previous e-mails and see how the algorithm is
described. Version numbers have to be increasing but not consecutive, if you
missed it you simply retry the operation with a new version number. This
algorithm is used to build fault tolerant solutions, so I think it would go
a good job of addressing many "what if" scenarios.


> > Here is my understanding so far.
> >
> > When you can use idempotent operations you should do so. But in
> some cases
> > they are impractical and you are better off not doing idempotent
> operations.
>
> This overstates the case.  Your application should behave in a way
> that makes sense.  You should not be forcing idempotent patterns on a
> domain where they do not make sense.  A "deposit" is such a case.
>  It means
> "here's some money to add to my account, whatever the balance was and
> whatever it will become."

Precisely my point.

Ages ago someone made a point in favor of idempotent/query only operations.
I tried to prove that it's not always in the best interest of the
application. I'm glad both of us agree on that.


> > If you are sending a message and know whether the message has been sent
> with
> > 99.99% success rate (i.e. only in 0.01% of cases you need to somehow
> resolve
> > the situation) then you are fine.
>
> Maybe yes; maybe no.  If that was the deposit of my paycheck and it got
> lost, then I'm not fine.  Not all problems are modeled well by statistics.

Let's say that banks start admitting their mistakes and pay the customer $50
every time they miss a deposit. They'll probably be looking at decreasing
their error rate. But they won't be looking at achieving 0% failure because
the cost would be higher than what it costs to pay a penalty and lose a
customer. At the end it's all about cost/benefit which does take statistics
into account.


> > If you are sending a message and do not immediately know whether the
> message
> > has been sent, and the success ration is 90% (e.g. when you use IP
> multicast
> > or UDP or SMTP) then you need to resolve 10% of the time.
>
> Yes, but WHICH 10%?  Which messages were received and which
> were not?  How do we find out?

There's one or two of may five startegies I could think of.


> > You can reduce that to 0.01% (or something like that, my numbers are all
> > made up) if you have some messaging layer that makes the messages
> idempotent
> > and keep resending until it gets close to 100% of success rate.
>
> You're dreaming.

Perhaps.

Here I am running a test sending one million messages over UDP in a tight
loop. I over saturate the connection so I get 90% packet loss at the UDP
level. I have a 5 second timeout so messages not receive within 5 seconds
are considered lost. I have reliable messaging with resend. All one million
messages arrive at their destination.

But it must be a dream. Let me pinch my self. Ouch!

arkin

Received on Thursday, 16 January 2003 00:01:58 UTC