Re: Proposed text on reliability in the web services architecture

Arkin,

> > >From a reliability standpoint, I know of applications for
> > which many intervening queues is not a good fit because is postpones
> > the immediate feedback the app is looking for, namely that the service
> > is offline right now.
>
> I would say that's a QoS issue. You don't want to circumvent the queue if
> the only way to reliabliy process your request is to use a queue. But you
> want to know if it can be processed. You don't want to send a message and
> get a reply 24 hours later saying, "sorry I'm too busy for the next two
> days, please come again". You want to send a synchronous request (what
ebXML
> calls a ping) to ask "can you process?" and then send the request.

No, actually that's not what I want.

>
>
> > No, it's not assuming any such thing.  It's simply that clients
> > and servers
> > both talk about ledgers and deposits and withdrawals, and that detail is
> > in the interface, not hidden in the implementation.
>
> I've done stock trading applications I can certainly tell you that both
> client and server need to keep a ledger and the server ledger is 10x more
> complex. So may the client can send 10% of the information to the server,
> but that's still part of the server-side state change.

No clue what this means.  Let me restate.  Having a client set the state of
an identified deposit is not "shifting the problem away" from anything; it's
something the client understands and means to do.  RM can do something
similar with a message, but an application level error can cause the
application to need to patch the deposit amount, and when it does so,
it "sweeps" RM, in a manner of speaking.  Please don't speak badly about
an application taking care of application needs.

>
>
> > Some applications depend on receiving (at some point) all the messages
> > sent to them, not just the more recent ones.  For example, a stock
ticker
> > store-and-forward system ("ticker plant" where I come from) needs all
> > the ticks, even though it gets summaries (idempotent) periodically.
> > Whether it is successfully receiving 99% or 90% or 50%, it has to
> > implement
> > the same recovery strategy, which involves knowing which messages are
> > received and which are not.
>
> Yep. That's why I said there are multiple strategies, it all depends on
what
> you want to achieve. You may ask for best-effort, which means you either
> receive it or not. You may to always get the message so loss requires
> resend. You may even depend (actually most applications do) on the strict
> ordering, so if X was sent before Y you need X before you can process Y.

What if every client defines its own unique strategy?

>
> Now, if I send/receive multiple messages then TCP solves the packet
problem
> by making sure each message is either full or ignored, but it doesn't
solve
> the multi-sequence message problem. So you need something on top of TCP
> whenever multiple messages are involved.

What is the multi-sequence message problem?

> The definition of reliable is (very loosely and not precise so don't kill
me
> on the wording): you deliver the message exactly once (deliver to the
> application, you can send/recieve it multiple times though), you deliver a
> message only if the message was actually send (no suprious messages), you
> deliver the message in some designated order.

Is the ordering constraint optional?

> The only two questions I have are:
>
> 1. Do you need RM?
> 2. Do you benefit from abstracting it?
>
> I think that in some situations I can get better implementation if I elect
> to use an RM. It's not a free lunch, but it buys me the ability to
> send/process messages asynchronously which is not an "always good thing"
but
> is often something I would like to do.

In the various frameworks I've written for real-time market data,
implementing them directly in TCP or UDP, I'd agree that certain
applications benefit from further support in the stack.  For example,
a popular pattern is for a TCP wrapper to keep connections alive
and re-establish connections, so as to allow the application to view
it as a persistent pipe to somewhere.

Concerning the above pattern, while it's helpful to remove the
tcp connect logic from the application, it can be equally confusing
for a client when there is something about the network that
obstructs connection, because the application can't communicate,
but it may not know it.  So ultimately, no you cannot abstract
away a real network if you are also going to deal with the stuff
that does go wrong.

However, in the one project I've done that I'd call a "web service",
the application protocol was HTTP, and there was no evident
advantage to having an RM in the loop.  It wasn't about messaging
reliability; it was about resource state reliability.  Note that while
TCP was in the stack, I don't call that a "web services RM".

>
> I also think that abstracting is a good thing becuase I can get that layer
> componentized, I can reuse it across multiple applications, or even buy it
> from a 3rd party, or download it as open source. But to be abstract we
need
> some generalized API (or even two or three).

Or even twenty or thirty.  I've developed lots of frameworks, probably
so many because of the fact that one size did not fit all. 8-(

>
> I can get that from any MOM, and I have an abstract API I can use (JMS),
but
> I think it's better if I also have some protocol that MOM X could use to
> talk to MOM Y, and that I could use to talk to the MOM as an intermediary
> Web service, so I can have a consistent way to define that API as a Web
> service.

The IP stack has worked well.  Do we _really_ need more?

I don't fault you for your optimism.  I've heard lots of promises from
developers about "totally generic software" that didn't pan out that way
because irritating little application requirements got in the way.

But...damn the torpedos!

Walden

Received on Saturday, 18 January 2003 23:46:48 UTC