Issue: Synch/Asynch Web services

ISSUE:  The current Web services architecture has technically
substandard definitions of synchronous and asynchronous, and almost no
discussion or use of these concepts in the architecture itself.  The WG
needs to come up with adequate definitions of these terms and address
how they relate to various aspects of the architecture.

This note will discuss in some detail the following four assertions.
The first assertion, which is essentially "why is this important", will
be discussed last so that it may draw on material developed in the other
discussions. The fourth may be viewed as an attempt to suggest
sub-issues, follow-ons or consequences of the main issue.

1 - There is widespread public interest in and understanding of the
concepts of synchronous and asynchronous in the context of Web services,
and there is a need to develop these concepts further in the context of
the Web architecture.

2 - The current discussion of s/as in the WSA is the result of an
aberrant process and is technically substandard.

3 - The current WSA does not satisfy the WSA requirements because of the
lack of treatment of these concepts.

4 - There are a number of questions and issues involving s/as that could
and in some cases should be addressed in the WSA.

--- Process History and Current Treatment ---

To my recollection, the discussion of s/sa involved many lengthy threads
and competing definitions, culminating in a more formal process
facilitated by David Booth involving collecting candidate definitions,
straw polls and discussion of the poll results.  The upshot of this on a
particular telcon (date not recalled) was fairly widespread agreement,
and perhaps even "unrecognized consensus" on the definition,

<definition>
http//lists.w3.org/Archives/Public/www-ws-arch/2003Mar/0146.html 
A request/response interaction is said to be asynchronous when the
request and response are chronologically and procedurally decoupled. In
other words, the client agent can process the response at some
indeterminate point in the future when its existence is discovered, for
example, by polling, notification by receipt of another message, etc. 

A request/response interaction is said to be synchronous when the client
agent must be available to receive and process the response message from
the time it issues the initial request until it is actually received or
some failure condition is determined. The exact meaning of "available to
receive the message" depends on the characteristics of the client agent
(including the transfer protocol it uses); it may, but does not
necessarily, imply tight time synchronization, blocking a thread, etc.
</definition>

There was, however, at least one person who remained particularly
attached to the following definition, which you will note has many
features in common with the first.  One clearly different component is
the explicit mention of "one-way synchronous messaging".  To my
knowledge, this concept has never been fully explained in these WG
discussions.  I believe that a substantial segment of the
public/industry thinks that such a thing is not defined (e.g.
http://lists.w3.org/Archives/Public/xml-dist-app/2000Dec/0089.html) or
exists only in contexts other than distributed systems (e.g.
http://www.cs.nott.ac.uk/~bsl/G52CON/Slides/Remote-Invocation.pdf).
Although it does not seem impossible to me, and perhaps it is indeed
worthwhile, to give this idea a clear conceptual foundation in the
context of Web services, it is frankly difficult for me to find any
evidence of a groundswell of industry interest in this specific idea as
opposed to the more usual sense in which the term synchronous is used.

Here is the alternative definition discussed in the telcon:

<definition>
http://lists.w3.org/Archives/Public/www-ws-arch/2003Feb/0437.html 
Synchronous message exchange (applies to oneway as well as
request/response) requires that both sender and receiver, or initiator
and respondant, processes are running/active at the same time as the
exchange takes place. In the case of request/response, the exchange is
synchronous if both sender and receiver remain in the running/active
state for both the request and response.

Asynchronous message exchange (also applies to oneway or request
response) does not require, but does not preclude, that both sender and
receiver, or initiator and respondant, processes are running/active at
the same time as the exchange takes place. It typically requires some
form of mediation between the sender and receiver such as a message
queue. 
</definition>

The telcon ended without clear resolution of this issue and, to my
recollection, a subteam was given the mandate of reconciling these two
definitions.  This subteam produced, after a rather long time, the
current definition which may be viewed at
http://www.w3.org/TR/2003/WD-ws-gloss-20030514/#synchronous. This
definition is clearly not in any way a reconciliation of the first two
definition alternatives, and so I think arguably was not consistent with
the "charter" of the subteam.  It is, in fact, actually less of a
"definition" than it is a "refusal to define".  It states that "The
terms 'synchronous' and 'asynchronous' are descriptive, and do not
correspond precisely to properties of MEPs," and deprecates the use of
these terms in any normative documents whatsoever: "In view of the
informal way that the terms are used, it is recommended that documents
should not rely upon the use of 'synchronous' or 'asynchronous' in any
normative specification."

This "definition" was then included in the glossary with very little
discussion and, in my view, no substantive demonstration of consensus.
I hasten to add that I do not think this was matter of some sort of
conspiracy or ill-motivated behavior, but mostly an understandable
matter of fatigue with the subject.  Unfortunately, however, since that
time most attempts to discuss issues involving s/sa or to include
verbiage in the document involving those concepts have met with strong
resistance within the WG, in large part presumably because this entry in
the Glossary seems to contend that the terms have no objective meaning
and deprecates their use in this and all other normative documents.

After the process described above, which I consider aberrant (in the
sense that the process agreed on by the WG was not in fact followed and
there was no real demonstration of consensus), I took the time and
energy personally to explore these concepts, concentrating primarily on
the concept of synchronous.  With the assistance of some people with
more Comp Sci background than mine, I engaged in an off-line
correspondence on this subject.  I kept it off-line because of my
perception (based on considerable evidence) that the WG was extremely
tired of hearing about this subject, but I cc-ed the chair on all
correspondence (until it started wandering way off-topic).  Somewhere in
this correspondence (the on-topic portion) I was told that I really
should read the classic paper by Lamport on synchronizing systems of
logical clocks in distributed systems
(http://courseweb.sp.cs.cmu.edu/~cs612/papers/lamport.pdf) because I was
trying to re-derive his analysis.  This was not strictly true, but I was
certainly wading in the same waters and Lamprey's analysis is at least
germane background material.  I earnestly recommend this article to
anyone reading this note, even those who have read the paper a long time
ago.  It is really worth reading or re-reading simply because it is
thoroughly excellent.

At any rate, I have convinced myself personally that it is by no means
impossible to understand these concepts in the context of Web services
and distributed systems.  I also convinced myself that I personally can
understand the concept of synchronous in a slightly more general context
and that the term most definitely has an objective meaning.  I have no
desire to impose my current understanding on the WG nor to propose my
own definitions of the concepts, but I would be glad to explain my
conceptual understanding if it would be useful (although it would take a
bit of time).  My "bottom line" conclusion, however, is that there is
really nothing fundamentally wrong with a number of the candidate
definitions that were examined in the process described above, and that
in particular a (true) reconciliation of the two definitions quoted
above would probably be perfectly satisfactory and usable. 

I emphasize above that I have mostly concentrated my personal efforts on
the concept of synchronous, and although I have some hope that
asynchronous may come along fairly easily as the complement of
synchronous, nonetheless Frank's introduction of the concept of
rendezvous as a necessary component of asynchronous operations gives me
a bit of pause.  Is this a requirement that asynchronous messages carry
information not required in synchronous messaging in order to support a
later rendezvous?  If so such a requirement might well be discussed more
explicitly in the WSA.

--- WSA Requirements ---

To be honest the assertion that the current WSA fails substantially to
meet the requirements because of the treatment of s/sa is not exactly a
slam dunk because the requirements do not explicitly mention the
concepts of synchronous or asynchronous.  If I had had any idea that
this would be a problem I certainly would have drafted something that
did, but who would expect this amount of difficulty over such commonly
understood and used concepts?

On the other hand, the requirements document
(http://www.w3.org/TR/2002/WD-wsa-reqs-20021114) does talk about
supporting the Usage Scenarios.  This is discussed in Section 2 as a
fundamental part of the requirements process, and section 3.2.2 states
that the WSA "architecture should support use cases for all aspects of
Web services" and that "the WSA must be validated against WSA use
cases".  The concepts of s/sa occur many times in the usage scenarios
document (http://www.w3.org/TR/2003/WD-ws-arch-scenarios-20030514/).  In
particular, there are several atomic usage cases that use these words in
the name of the case ("S007 Multiple asynchronous responses", "S070
Asynchronous messaging" and "S071 Asynch/Synchronous specificity") and
there are specific references to the concepts in the texts describing
other usage cases (e.g. "S003 Request/Response", "S300 System
Messages").   In addition, although the terms are not specifically used,
having worked on the extended usage scenarios I know that these are part
of the conceptual underpinnings of these models.  For example, the
editorial comments documenting changes in the Travel Service usage case
mention the concept, and having authored the EDI usage scenario I know
that this concept, although not explicitly stated, was an important part
of the picture.

It is my best judgment that the current WSA does not meet at least the
spirit of the requirements because of the content about s/sa in the
current Usage scenarios and the lack of content in the WSA related to
these issues.  In my view when the WSA does not support significant
content in the usage scenarios it is an indication that there is a real
problem.  I might add that in my view the best solution to that problem
is not to modify the Usage Scenario working draft to eliminate the
content that is not reflected in the WSA.

--- Questions to address ---

The following questions are intended as suggestions for issues that
might be addressed in the WSA related to the concept of synchronous and
asynchronous.  It is not intended to be complete nor is it asserted that
all topics are in scope and nontrivial.  To some extent these questions
may be a start at developing a "stakeholders viewpoint", using the
current nomenclature of the document, but such an analysis might also
reveal more formal elements that need to be included in the concepts and
relationships portion of the document.

- Can some WS's be classified as appropriate for synchronous usage (e.g.
based on response time), some as appropriate for asynchronous usage
(e.g. based on handling of the information required to define a
rendezvous) and others as both?  If so, should this be standardized in
WSDL?

- Should a WS requestor be able to assert what it expects in terms of
synchronous or asynchronous operation?  Or is it inherently a matter of
"try it and see"?  Or recognize from description?

- In the context of late binding, are there potential problems if a
requestor looking for a synchronous Web service actually has found one
intended for asynchronous use or vice versa?  [The answer to this is
obviously "Yes"].  How can such problems be recognized or avoided?

-Could/should a management interface expose information to potential
users of a WS about past performance metrics and/or expectations of
relevance to the issue of s/sa usage?  (e.g. expected/observed response
times)?

-Are there messaging requirements related to information required to
define a later rendezvous for asynchronous messaging?  Is there more
than one way for an asynchronous WS to affect the rendezvous?  Are these
questions addressed, or should they be addressed, by XMLP?

- Do intermediaries for WS's always need to assume that the interactions
are synchronous?  If not, do they need to be able to tell from the
nature of the interaction what it is or do the principles need to know
something about the intermediaries based on whether they expect
synchronous or asynchronous operation?

- Are there security and/or privacy issues that are unique to
synchronous or asynchronous WS's?  If so, are the methods of resolution
fundamentally different?

- Are there timing issues that need to be standardized with respect to
synchronous Web services?  Or Web services that are invoked
synchronously.  I'm not sure if those are different things.  I'd like
the WSA to tell me.

- Are there management functions specific to synchronous or asynchronous
Web services?  Or S/A invocations of WS's, if this is the way to look at
it?

--- Public Interest ---

There is no lack of discussion of the concepts of synchronous and
asynchronous in the context of Web services, and no apparent confusion
about what the terms mean.  Examples include:
http://searchwebservices.techtarget.com/tip/1,289483,sid26_gci905098,00.
html http://java.sun.com/blueprints/webservices/using/webservbp3.html
http://www-106.ibm.com/developerworks/webservices/library/ws-asynch1.htm
l
A number of authors note that asynchronous operations are not explicitly
supported by current standards, although there is seldom any comment
about whether this is good or bad.  The thrust of many of these
treatments is to discuss how asynchronous implementations of Web
services may be achieved in various proprietary settings.  I have not,
however, seen any discussions anywhere of whether these implementations
are interoperable.  This might make one a bit nervous.

There is, of course, a large body of academic work related to the
concepts of synchronous and asynchronous in distributed systems in
general.  Aside from calling attention above to the classic paper by
Lamport I make no attempt to review these materials. 
There is also the issue of the importance of asynchronous Web services
to the business community at large.  In this case I will consider myself
to be a reasonable source and hereby make some comments:

Historically Web services originated from the primarily synchronous
background of the RPC model.  Although many or most vendors implementing
this style of Web services also offer some sort of asynchronous variant,
this is not the thrust of the technique and, as noted above, there is no
explicit support in the standards nor do I know of detailed discussions
of interoperability.  Recent developments in Web services, however, are
directed more at the document exchange model which is central to
automating business processes.  Many or most of core business processes
are, by their nature, asynchronous because the work flows involve steps
that include decisions by humans.  Replacing these steps by automation
is not on the front burner of business priorities and at the very least
would have to proceed by slow, incremental steps.  To start this
process, if one indeed wants to start it at all, one must first
establish the asynchronous operations.

There is a considerable distrust among business end users of the idea of
using Web services for core business processes.  Although the primary
sources of that uneasiness are probably security and reliability, the
ability to handle asynchronous processes is also an often-unstated part
of the equation and many business people have pigeon-holed Web services,
perhaps implicitly, as "suitable only for synchronous operations".  Web
services are simply not widely known as handling asynchronous processes.
Do the Web services standards form an adequate basis for asynchronous
implementations?   An essential refusal to address the concepts would
send a very negative message to the business community.  It would
certainly affect the recommendations I would give to my company with
respect to strategic positioning of alternative technologies.

There is no point whatsoever in worrying about how to choreograph or
manage Web services if you don't trust them enough to use them in the
first place.