Re: Memory Overhead

Vadim--

As a result of the replies i got from the DOM involved people on the 
mailing list i figured that the problem is in my implementation.
As Joseph Kesselman pointed out, there are a lot of elements that can be 
simply replicated by pointers where duplicates are found.
This is the clear example of a database replication on an XML file, where 
the DB structure can be defined in a XSL file or in a DTD and the DOM 
implementation can use that information to build the tree.

My DOM implementation (which is at its VERY early stages) simply built 
nodes again and again, regardless of the number of occurences of elements 
or attribtes, therefore consuming large amount of memory.

I won't post the file because it is a trivial copy+paste created file, for 
stress-testing only, considering the size of the file. I generated the file 
with XMLSpy from the sample database that you can find in MSAccess. After 
that, i just Copied and pasted its contents growing exponentially the size 
of the file. The number of elements reached about 230.000.
So, it's a trivial example.
--Razvan
At 11:11 AM 9/12/01 +0000, Vadim Plessky wrote:
>On Monday 10 September 2001 15:48, Razvan Costea-Barlutiu wrote:
>|   I read at some point some reference about the DOM level 1 specification,
>|   saying that a DOM heirarchy might take up to 4 times as much memory than
>|   the file it was loaded from, depending on the implementation. I
>| completelly agree with that.
>|
>|   I made an implementation for DOM level 2 (without the events part, which
>|   actually is the real breakthrough of DOM 2) and i begin stress-testing it
>|   on a 1.7 MB file containing some 230.000 entries (I just copied and
>| pasted the contents of the file, almost choking at some point XMLSpy, after
>| i moved to mighty... Notepad).
>|
>|   Anyway, these entries "eat" about 50 MB of memory which i find pretty
>|   scaring, for a 1.7 MB file. I searhed my code for memory leaks or
>|   over-allocations, but found none.
>
>Any chance that you can put this 1.7MB file somewhere online (better -
>zip'ed) so we can test with different browsers?
>Konqueror (www.konqueror.org) has very good DOM-, CSS2- based "KHTML" engine,
>and I can't say I have ever seen such (50mb) memory usage with it. ( I tested
>it on a 1.8MB bug archive file from bugs.kde.org, everything was fine)
>But of course it can vary depending on file structure.
>Anyway, I am ready to test with Konqi your file, and tell you what exactly
>memory usage was.
>
>|
>|   __________________________________________________________________
>|   Razvan Costea-Barlutiu
>|   Department of Radiology,
>|   The University of Chicago
>|   5841 South Maryland Avenue
>|   Chicago, Illinois 60637
>|   Phone: (773)834-5106
>|   E-Mail: cbrazvan@baltan.bsd.uchicago.edu
>
>--
>
>Vadim Plessky
>http://kde2.newmail.ru  (English)
>33 Window Decorations and 6 Widget Styles for KDE
>http://kde2.newmail.ru/kde_themes.html
>Do you have Arial font installed? Just test it!
>http://kde2.newmail.ru/font_test_arial.html

__________________________________________________________________
Razvan Costea-Barlutiu
Department of Radiology,
The University of Chicago
5841 South Maryland Avenue
Chicago, Illinois 60637
Phone: (773)834-5106
E-Mail: cbrazvan@baltan.bsd.uchicago.edu

Received on Wednesday, 12 September 2001 11:23:02 UTC