Large Messages (was RE: Visibility (was Re: Introducing the Servi ce Oriented Architec tural style, and it's constraints and propertie s.

Mark's statement that "The only downsides of even 100% visibility, AFAIK,
are that messages are typically large, and that data formats are
standardized rather than optimized for the particular app" reminded me of
just how big messages can get when they are used for eBusiness. Here are two
use cases:

AEROSPACE
For legal reasons, when an aeroplane is sold, the Invoice must be
accompanied by a line by line parts breakdown including serial numbers.
Given that there can be millions of parts in a commercial aeroplane, the
estimated maximum message size can be up to 2GB.

TELECOMMUNICATIONS
Perhaps there aren't enough aeroplanes sold to make it worthwhile developing
an approach to just meet their needs, so how about this example from the
telecomms industry ... A major telecommunications company wants to provide
an itemised Invoice detailing each call for a company with hundreds of
locations. Sometimes there can be 100k+ line items. As each line is
approximately 90 characters long (including tags), messages sizes can be
easily over 10Mb or more.

Typically, the way to handle this is to have a "utility protocol" to break
up the message into smaller chunks prior to sending and then re-assemble at
the far end.

Do we want to include this type of requirement in our thinking? Also how
well would this sort of problem be handled by a REST approach, without any
additional work?

David

-----Original Message-----
From: Mark Baker [mailto:distobj@acm.org]
Sent: Saturday, March 01, 2003 9:07 PM
To: Champion, Mike
Cc: www-ws-arch@w3.org
Subject: Re: Visibility (was Re: Introducing the Service Oriented
Architec tural style, and it's constraints and properties.



On Sat, Mar 01, 2003 at 07:56:58PM -0500, Champion, Mike wrote:
> And GET http://www.quotes-be-us.com/ibm is opaque to JMS implementations.

Of course.  Most things are invisible to a vanilla (i.e. without any
prior knowledge of any application semantic) JMS implementation, because
JMS provides an abstraction to OSI layer 6 protocols like BEEP or SOAP
or IIOP.  It does not abstract application protocols.  If it's used to
do so (which it is, of course 8-), then that means HTTP is being used as
a layer 6 protocol too, not the layer 7 protocol that it is.

> That's why SOAP exists to allow SOAP intermediaries to work the same
across
> protocol bindings.  They tunnel over HTTP, FTP, SMTP, JMS impelementation,
> MQ and other proprietary stuff with equal oblivion.    

I'm aware that this is the common view, as you're aware of my opinion
that this use of SOAP has no future.

FWIW, you can track the adoption of this use of SOAP here;

http://www.markbaker.ca/2002/04/WebServicesGrowth/

As you can see, SOAP's growth over the Internet has been approximately
linear.  A far cry from the Web's growth, which can be seen here;

http://www.mit.edu/people/mkgray/net/web-growth-summary.html

There may be other reasons for this, but I have a good technical one
that suffices to explain it; integration complexity is too high due to
each service having a different interface than the next.

> Huh?  The SOAP paradigm gives a single framework -- XML, headers,
> intermediaries -- that provides a place to put semantically meaningful
> information in the content of a message that is equally meaningful
> irrespective of the transport layers it exploits, tunnels, or whatever.
The
> use case is precisely the fact that it is hard to map semantically
important
> information from one protocol-specific form to another, so putting the
> semantics in the content rather than relying on "bridging" the protocols
> makes lots of sense. 

It is hard, but not impossible.  And the good news is, there aren't very
many application protocols.

The downside of avoiding this hard work by just treating all these
application protocols as transport protocols, is that it disregards
the very reasons that these protocols became successful in the first
place, as well as the useful architectural properties that their
associated architectural styles exhibit.  Different application
protocol exist because there are different applications.  Email is not
the Web.  The Web is not Usenet.  Usenet is not IRC.  IRC is not FTP.
And they differ not just in their connectionless/connection,
stateless/stateful, asynch/sync nature, they differ because they have
different methods that are appropriate to those applications; uniform
operations for hypermedia, file/directory-specific operations for FTP,
chat channel specific operations for IRC, etc..

> Let's take this down a few levels of abstraction and just talk about
> personal opinions on best practice.

I think I've made my opinions pretty clear in many discussions
here and elsewhere.  We're miles apart, obviously.

> So, back to "visibility."  I think we have a pretty good outline of the
> costs, benefits, and tradeoffs pertaining to visibility in different
> scenarios.

Is that true?  Does the WG understand the costs?  You've made it clear
that you disagree with the problems I've described with Web services
and firewalls.  These are real problems that real developers have
already felt, and will continue to feel.  Surely the architecture
document should provide some guidance on this?  This, btw, is why I was
chasing Dave as hard as I was; so that if it was recognized that
visibility was reduced with Web services, even in "single protocol
mode" <cough/), that there would be little choice except to say
something about it.  I would ideally like to see something like this
said;

"When it is desirable for a service to be used over the Internet,
serious consideration should be given to using the REST architectural
style, or other architectural styles which have greater visibility
than WSA/SOA (such as those styles used by Internet email, FTP, IRC,
etc..).  This recommendation is being made because visibility is
important to a firewall's ability to do its job, and firewall
adminstrators may choose not to pass messages whose semantics are not
as visible as they would like them to be."

>  It's quite clear that in the typical Web environment of today,
> when security / reliability requirements are not terribly onerous, the
> services under consideration involve moving information representations
> around, and mature firewall technology is in place, then most of what you
> say about HTTP and visibility is true and desireable.  Ratchet up the
> security/reliability requirements and put some complexity on the back-end,
> however, and it's clear that the Web as we know it is not a great platform
> for web sevices without the kind of help that the SOAP-based specs offer.
> All that "visiblity" becomes a liability (as Roger has helped us
understand
> from the IT perspective).

The data-in-URI issue has very little, if anything, to do with
visibility.

The only downsides of even 100% visibility, AFAIK, are that messages are
typically large, and that data formats are standardized rather than
optimized for the particular app.

>  The SOAP framework and the numerous standards and
> proposals that work within it really do add value for the people working
in
> these areas.  There are costs, of course -- XML/SOAP/WS-Security-aware
> firewalls are clearly going to be more expensive in dollars and resource
> requirements than vanilla HTTP-aware firewalls are.  This is neither a
Good
> Thing nor a Bad Thing, just one more tradeoff that Web services architects
> are going to have to take into consideration.  

Time will tell, I suppose.

MB
-- 
Mark Baker.   Ottawa, Ontario, CANADA.        http://www.markbaker.ca
Web architecture consulting, technical reports, evaluation & analysis