Re: SOAP performance

Hi,

Because the SOAP performance experiments in Robert Steele's paper 
reflect the performance results of the gSOAP toolkit, I felt somewhat 
compelled as the developer of this toolkit to comment and give you my 2 
cents worth :-) though I'm not sure if this is the right forum to 
address this topic.

The gSOAP toolkit [4][5] essentially generates a recursive descent 
parser for an XML schema, i.e. the parser is specifically optimized to 
consume instances of that schema [2][5]. The results of this 
"schema-specific" parsing approach on performance are available in [3] 
and a comparison to even faster techniques is presented in [1]. To 
corroborate Steele's results, another interesting real-world 
application (SNMP) is discussed in [6] and [7], which also investigates 
native formats compared to XML with and w/o compression.

Basically, I concluded from our tests that Zlib compression is not only 
CPU intensive but it also exceeds the XML parsing and generation 
overhead in gSOAP by a factor of 4 or more. You will only be able to 
achieve a better overall performance on slow networks with Zlib 
compression, i.e. when XML message exchange becomes IO bound instead of 
CPU bound such as with modem lines. However, V90 modems already apply 
compression techniques and adding Zlib stack only makes the performance 
worse. So there is some consensus that XML web services are CPU bound 
as Alex pointed out. Other compression techniques such as XMill might 
be fast enough to improve performance, because XMill specifically 
targets the XML structure. But I have not seen any results yet.

Cheers,

- Robert

[1] Robert van Engelen, Constructing Finite State Automata for High 
Performance XML Web Services, in the proceedings of the International 
Symposium on Web Services (ISWS), 2004.

[2] Robert van Engelen, Code Generation Techniques for Developing Web 
Services for Embedded Devices, in the proceedings of the 9th ACM 
Symposium on Applied Computing SAC, Nicosia, Cyprus, 2004, pages 
854-861.

[3] Robert A. van Engelen, Pushing the SOAP Envelope with Web Services 
for Scientific Computing, in the proceedings of the International 
Conference on Web Services (ICWS), 2003, pages 346-354.

[4] Robert van Engelen, Gunjan Gupta, and Saurabh Pant, Developing Web 
Services for C and C++, in IEEE Internet Computing Journal, March, 
2003, pages 53-61.

[5] Robert A. van Engelen and Kyle Gallivan, The gSOAP Toolkit for Web 
Services and Peer-To-Peer Computing Networks, in the proceedings of the 
2nd IEEE International Symposium on Cluster Computing and the Grid 
(CCGrid2002), pages 128-135, May 21-24, 2002, Berlin, Germany.

[6] Thomas Drevers, Remco van de Meent, and Aiko Pras. Prototyping web 
services based network          monitoring. In Proceedings of 10th Open 
European Summer School          and IFIP WG 6.3 Workshop (EUNICE 2004). 
(J. Harju, D. Moltchanov          and B. Silverejan, eds.), pp. 
135-142, June 2004.

[7] Thomas Drevers, Thesis: 
http://asna.ewi.utwente.nl/assignments/completed/ARCH-2004-01.pdf

On Tuesday, October 26, 2004, at 03:54 PM, Aleksander Slominski wrote:

> David Orchard wrote:
>
> It's hard to really evaluate these things without more data.
>
> hi,
>
> i agree but i think it is valuable to make efforts to characterize 
> what factors are important for SOAP toolkit performance - we did some 
> work on that in [1] although we concentrated mostly on scientific 
> computing - we have plans for more in future especially related to 
> scalability, memory footprint, and optimizations (and we welcome 
> comments about it!).
>
> also one should consider that there can be big gains if you know the 
> structure of XML messages and optimize infrastructure for those 
> messages (like FIX protocol) and/or have  tools to generate very fast  
> "schema specific" parsing (such as generating schema specific parsing 
> code executed by automata described in [2])
>
> What was
> the cpu load on the slow machine that was doing compression?  What is
> the relationship between cpu speeds on the compression/decompression
> machines and the round trip time?
>
>
> when we were doing our evaluation of use of SOAP toolkits in 
> scientific computing most of interactions for high performance 
> toolkits was CPU-bound when done over fast network and compression 
> would only slow down them - in other words they were CPU bound not IO 
> bound  (you can reproduce it by running our tests [3]).
>
>
> FWIW, I don't think it surprising that certain CPU speeds make
> compression worse.  IMO, the more important question is what is the
> relationship between cpu and network speed?
>
> i think that there are two ends of spectrum: very fast local network 
> and WAN with low bandwidth (modem) and huge bandwidth but noticeable 
> latencies (intercontinental links or space :)).
>
>  And there's also the
> relationship between client versus server cpu and 
> encryption/decryption.
> Imagine that decompression is "cheap" compared to compression, then it
> might be that a telco could upgrade is servers to do compression on
> messages to clients, and thus compression is a net benefit to the
> system.
>
>
> that i think will depend on bandwidth usage - i do not think this a 
> big deal for broadband users but may be big win for mobile/modem users 
> especially if mobile users have to pay for bandwidth usage (Europe?) 
> ...
>
> alek
>
> [1] Madhusudhan Govindaraju, Aleksander Slominski, Kenneth Chiu, Pu 
> Liu, Robert van Engelen, Michael J. Lewis, Toward Characterizing the 
> Performance of SOAP Toolkits. To appear in the proceedings of the 5th 
> IEEE/ACM International Workshop on Grid Computing, (short paper) 
> November 8th, 2004, Pittsburgh, USA.
> [2] Kenneth Chiu and Wei Lu. A Compiler-Based Approach to 
> Schema-Specific XML Parsing. In First International Workshop on High 
> Performance XML Processing(Satellite of WWW2004), May 2004.
> [3] http://www.extreme.indiana.edu/xgws/soap_bench/
>
>
>
>
> -----Original Message-----
> From: www-ws-request@w3.org [mailto:www-ws-request@w3.org] On Behalf
>
>
> Of
>
>
> Francis McCabe
> Sent: Tuesday, October 26, 2004 12:08 PM
> To: Michael Champion
> Cc: David Booth; Robert James Steele; www-ws@w3.org
> Subject: Re: SOAP performance
>
>
> I guess that those pointy brackets are expensive!
> But, really, is this a surprise?
>
> (I was mildly surprised at the result re. compression: it makes things
> worse)
>
> Frank
>
> On Oct 25, 2004, at 2:32 PM, Michael Champion wrote:
>
>
>
>
> On Oct 25, 2004, at 5:13 PM, David Booth wrote:
>
>
>
>
>
> Sounds interesting.  Is there a URL for it?
>
>
>
>
> http://www2003.org/cdrom/papers/alternate/P872/p872-kohlhoff.html
>
>
>
>
>
>
>
>
>
>
>
> -- 
> The best way to predict the future is to invent it - Alan Kay
>

Received on Wednesday, 27 October 2004 17:10:17 UTC