- From: Toshiki Murata <mura@kansai.oki.co.jp>
- Date: Mon, 03 Mar 1997 17:28:48 +0900
- To: www-jigsaw@w3.org
- Cc: mura@kansai.oki.co.jp (Toshiki Murata)
In message <199702280957.KAA28920@www43.inria.fr>you write:
> Toshiki Murata writes:
> > I used w3c.tools.dbm.jdbm for very large data(size > 7000).
> > Then w3c.tools.dbm.jdbm#lookup(key) cause
> That's a difficult bug to track, I haven't tried storing such big
> objects within that database. You may want to turn traces on (set
> debug = true, in jdbm.java).
I did it.
I put the log in last part of this mail.
> I think a related bug has been fixed (some times ago, check comments)
> in jdbm::write(jdbmBucket, key, value) a long time ago.
I think it isn't related.
(You say about jdbm.java:586, don't you?)
> Also you have to make sure to shtdown the database properly between to
> runs of your application (otherwise you may have the files in a
> inconsistent state), but I guess you did it ?
Of course, I did it.
Before I shutdown the database, all data were available.
I think all index exist in memory image,
but some of them losts when it's saved in a file.
I don't know gdbm algorithm.
I think, after jdbm#splitBucket,
it needs to re-organize all buckets as figure 1(3).
But jdbm is shown as figure 1(2).
In figure 1(3), when splitBucket,
it has to all buckets that exist are copied to new area.
So, in figure1(4) it doesn't need to copy.
figure 1. dbm file image
(1) before jdbm#splitBucket (2) after jdbm#splitBucket
+---------------------------+ +---------------------------+
| index table (diridx[256]) | | index table (diridx[512]) |
|---------------------------| |- -- - - - - |
| AAAA BBBB | | AAAA(over-written) |
| bucket data | split | -- -- -- ---|
| | Bucket | |
| | ---->> | bucket data |
| | | |
| | | |
| | | |
| | | DDDD |
| | | |
+---------------------------+ +---------------------------+
(3) right image? or (4) another image
+---------------------------+ +---------------------------+
| | | index table1(diridx[256]) |
| index table (diridx[256]) | |---------------------------|
| | | AAAA BBBB |
|---------------------------| | bucket data1 |
| AAAA BBBB | | |
| bucket data | +---------------------------+
| | | index table2(diridx[256]) |
| | |---------------------------|
| | | |
| | | bucket data2 |
| DDDD | | DDDD |
+---------------------------+ +---------------------------+
debug log ....
block_size = 1024
dir_bits = 8
dir_size = 256
dir_adr = 1024
cache_size = 32
dir_size = 256
bucket_elems = 41
next_block = 4
avail_count = 0
avail_length = 123
jdbm: lookup: at 38 [h=-1 ,ksize=-1 ,dsize=-1]
jdbm: write: @3072
jdbm: lookup: at 25 [h=-1 ,ksize=-1 ,dsize=-1]
jdbm: write: @3098
jdbm: lookup: at 4 [h=-1 ,ksize=-1 ,dsize=-1]
jdbm: write: @3119
:
:
:
jdbm: lookup: at 6 [h=1416736187 ,ksize=7 ,dsize=10]
jdbm: lookup: at 7 [h=-1 ,ksize=-1 ,dsize=-1]
jdbm: allocateSpace: avail_count=1
jdbm: write: @387329
jdbm: lookup: at 9 [h=1440063550 ,ksize=14 ,dsize=18]
jdbm: lookup: at 10 [h=-1 ,ksize=-1 ,dsize=-1]
jdbm: allocateSpace: avail_count=1
jdbm: write: @387340
jdbm: lookup: at 11 [h=610108405 ,ksize=14 ,dsize=8]
jdbm: lookup: at 12 [h=606777490 ,ksize=9 ,dsize=10]
jdbm: lookup: at 13 [h=606722687 ,ksize=13 ,dsize=6]
jdbm: lookup: at 14 [h=618411419 ,ksize=8 ,dsize=18]
jdbm: lookup: at 15 [h=616040513 ,ksize=7 ,dsize=10]
jdbm: lookup: at 16 [h=608060149 ,ksize=6 ,dsize=6]
jdbm: lookup: at 17 [h=617472888 ,ksize=5 ,dsize=6]
jdbm: lookup: at 18 [h=611605503 ,ksize=6 ,dsize=8]
jdbm: lookup: at 19 [h=612742472 ,ksize=8 ,dsize=14]
jdbm: lookup: at 20 [h=614669885 ,ksize=5 ,dsize=16]
jdbm: lookup: at 21 [h=618872371 ,ksize=8 ,dsize=14]
jdbm: lookup: at 22 [h=605260992 ,ksize=7 ,dsize=6]
jdbm: lookup: at 23 [h=609903908 ,ksize=11 ,dsize=12]
jdbm: lookup: at 24 [h=616285989 ,ksize=10 ,dsize=12]
jdbm: lookup: at 25 [h=614503679 ,ksize=3 ,dsize=12]
jdbm: lookup: at 26 [h=613170120 ,ksize=9 ,dsize=18]
jdbm: lookup: at 27 [h=611838517 ,ksize=8 ,dsize=16]
jdbm: lookup: at 28 [h=606908719 ,ksize=3 ,dsize=18]
jdbm: lookup: at 29 [h=607224297 ,ksize=9 ,dsize=10]
jdbm: lookup: at 30 [h=611586329 ,ksize=10 ,dsize=14]
jdbm: lookup: at 31 [h=611425240 ,ksize=4 ,dsize=20]
jdbm: lookup: at 32 [h=620665084 ,ksize=6 ,dsize=10]
jdbm: lookup: at 33 [h=610350837 ,ksize=6 ,dsize=10]
jdbm: lookup: at 34 [h=614058346 ,ksize=9 ,dsize=16]
jdbm: lookup: at 35 [h=619450954 ,ksize=7 ,dsize=22]
jdbm: lookup: at 36 [h=618764039 ,ksize=7 ,dsize=12]
jdbm: lookup: at 37 [h=614935790 ,ksize=7 ,dsize=6]
jdbm: lookup: at 38 [h=610215707 ,ksize=7 ,dsize=8]
jdbm: lookup: at 39 [h=619627996 ,ksize=6 ,dsize=8]
jdbm: lookup: at 40 [h=619606307 ,ksize=8 ,dsize=12]
jdbm: lookup: at 0 [h=618649613 ,ksize=9 ,dsize=16]
jdbm: lookup: at 1 [h=615923116 ,ksize=11 ,dsize=12]
jdbm: lookup: at 2 [h=605742612 ,ksize=9 ,dsize=6]
jdbm: lookup: at 3 [h=607342474 ,ksize=4 ,dsize=8]
jdbm: lookup: at 4 [h=608180351 ,ksize=3 ,dsize=4]
jdbm: lookup: at 5 [h=612495766 ,ksize=9 ,dsize=10]
jdbm: lookup: at 6 [h=607605529 ,ksize=11 ,dsize=16]
jdbm: lookup: at 7 [h=615167484 ,ksize=9 ,dsize=12]
jdbm: lookup: at 8 [h=608925403 ,ksize=9 ,dsize=6]
jdbm: lookup: at 9 [h=615306266 ,ksize=6 ,dsize=10]
jdbm: lookup: at 10 [h=606082991 ,ksize=7 ,dsize=14]
jdbm: split bucket: 48128
jdbm: allocateSpace: avail_count=2
jdbm: allocateSpace: avail_count=1
jdbm: splited b0=48128
jdbm: splited b1=388096
jdbm: updating dir from 72 to 73
jdbm: updating dir from 73 to 74
jdbm: allocateSpace: avail_count=1
jdbm: write: @387371
jdbm: lookup: at 2 [h=-1 ,ksize=-1 ,dsize=-1]
jdbm: allocateSpace: avail_count=1
jdbm: write: @387385
jdbm: lookup: at 31 [h=-1 ,ksize=-1 ,dsize=-1]
jdbm: allocateSpace: avail_count=1
jdbm: write: @387395
jdbm: lookup: at 20 [h=-1 ,ksize=-1 ,dsize=-1]
jdbm: allocateSpace: avail_count=1
jdbm: write: @387404
jdbm: lookup: at 36 [h=1552340109 ,ksize=9 ,dsize=8]
jdbm: lookup: at 37 [h=1555524701 ,ksize=11 ,dsize=6]
jdbm: lookup: at 38 [h=1556874629 ,ksize=10 ,dsize=8]
jdbm: lookup: at 39 [h=-1 ,ksize=-1 ,dsize=-1]
jdbm: allocateSpace: avail_count=1
jdbm: write: @387423
jdbm: lookup: at 10 [h=-1 ,ksize=-1 ,dsize=-1]
jdbm: allocateSpace: avail_count=1
jdbm: write: @387449
jdbm: lookup: at 2 [h=244041675 ,ksize=14 ,dsize=8]
jdbm: lookup: at 3 [h=248711412 ,ksize=15 ,dsize=20]
jdbm: lookup: at 4 [h=245706849 ,ksize=9 ,dsize=10]
jdbm: lookup: at 5 [h=248310027 ,ksize=12 ,dsize=8]
jdbm: lookup: at 6 [h=250947967 ,ksize=8 ,dsize=12]
jdbm: lookup: at 7 [h=250704961 ,ksize=10 ,dsize=6]
jdbm: lookup: at 8 [h=243874609 ,ksize=12 ,dsize=8]
jdbm: lookup: at 9 [h=249328423 ,ksize=7 ,dsize=12]
jdbm: lookup: at 10 [h=243313065 ,ksize=6 ,dsize=10]
jdbm: lookup: at 11 [h=246125297 ,ksize=7 ,dsize=10]
jdbm: lookup: at 12 [h=244896510 ,ksize=10 ,dsize=14]
jdbm: lookup: at 13 [h=249342756 ,ksize=11 ,dsize=4]
jdbm: lookup: at 14 [h=249573943 ,ksize=5 ,dsize=6]
jdbm: lookup: at 15 [h=249593091 ,ksize=9 ,dsize=8]
jdbm: lookup: at 16 [h=249316219 ,ksize=7 ,dsize=12]
jdbm: lookup: at 17 [h=248763498 ,ksize=7 ,dsize=4]
jdbm: lookup: at 18 [h=246396734 ,ksize=8 ,dsize=8]
jdbm: lookup: at 19 [h=246951095 ,ksize=7 ,dsize=8]
jdbm: lookup: at 20 [h=244988140 ,ksize=11 ,dsize=16]
jdbm: lookup: at 21 [h=248583758 ,ksize=18 ,dsize=10]
jdbm: lookup: at 22 [h=246455033 ,ksize=9 ,dsize=4]
jdbm: lookup: at 23 [h=251216453 ,ksize=8 ,dsize=18]
jdbm: lookup: at 24 [h=249815607 ,ksize=9 ,dsize=8]
jdbm: lookup: at 25 [h=250720887 ,ksize=9 ,dsize=12]
jdbm: lookup: at 26 [h=249416966 ,ksize=4 ,dsize=2]
jdbm: lookup: at 27 [h=248613695 ,ksize=8 ,dsize=6]
jdbm: lookup: at 28 [h=249255469 ,ksize=9 ,dsize=12]
jdbm: lookup: at 29 [h=245873789 ,ksize=11 ,dsize=20]
jdbm: lookup: at 30 [h=249111307 ,ksize=11 ,dsize=12]
jdbm: lookup: at 31 [h=250297033 ,ksize=8 ,dsize=10]
jdbm: lookup: at 32 [h=250055137 ,ksize=5 ,dsize=6]
jdbm: lookup: at 33 [h=243478410 ,ksize=13 ,dsize=6]
jdbm: lookup: at 34 [h=248224284 ,ksize=6 ,dsize=8]
jdbm: lookup: at 35 [h=250830983 ,ksize=11 ,dsize=10]
jdbm: lookup: at 36 [h=245981053 ,ksize=8 ,dsize=12]
jdbm: lookup: at 37 [h=251180384 ,ksize=8 ,dsize=10]
jdbm: lookup: at 38 [h=250942875 ,ksize=9 ,dsize=8]
jdbm: lookup: at 39 [h=250596835 ,ksize=6 ,dsize=8]
jdbm: lookup: at 40 [h=247868574 ,ksize=7 ,dsize=12]
jdbm: lookup: at 0 [h=250492985 ,ksize=13 ,dsize=24]
jdbm: lookup: at 1 [h=250497537 ,ksize=8 ,dsize=10]
jdbm: split bucket: 221184
jdbm: allocateSpace: avail_count=2
jdbm: allocateSpace: avail_count=1
jdbm: splited b0=221184
jdbm: splited b1=389120
jdbm: allocateSpace: avail_count=1
jdbm: updating dir from 58 to 59
jdbm: updating dir from 59 to 60
jdbm: allocateSpace: avail_count=1
jdbm: write: @387466
jdbm: lookup: at 12 [h=766625432 ,ksize=4 ,dsize=4]
jdbm: lookup: at 13 [h=765933846 ,ksize=10 ,dsize=18]
jdbm: lookup: at 14 [h=-1 ,ksize=-1 ,dsize=-1]
jdbm: allocateSpace: avail_count=1
jdbm: write: @387482
jdbm: lookup: at 21 [h=1346806108 ,ksize=12 ,dsize=18]
jdbm: lookup: at 22 [h=1349945686 ,ksize=10 ,dsize=4]
jdbm: lookup: at 23 [h=1347512008 ,ksize=5 ,dsize=10]
jdbm: lookup: at 24 [h=1346643261 ,ksize=10 ,dsize=14]
jdbm: lookup: at 25 [h=1346163103 ,ksize=3 ,dsize=6]
jdbm: lookup: at 26 [h=1344043166 ,ksize=6 ,dsize=8]
jdbm: lookup: at 27 [h=-1 ,ksize=-1 ,dsize=-1]
jdbm: allocateSpace: avail_count=1
jdbm: write: @387500
jdbm: lookup: at 12 [h=-1 ,ksize=-1 ,dsize=-1]
jdbm: allocateSpace: avail_count=1
jdbm: write: @387514
jdbm: lookup: at 19 [h=1525368406 ,ksize=10 ,dsize=10]
jdbm: lookup: at 20 [h=1523550261 ,ksize=13 ,dsize=6]
jdbm: lookup: at 21 [h=1519438822 ,ksize=4 ,dsize=6]
jdbm: lookup: at 22 [h=1524077192 ,ksize=5 ,dsize=4]
jdbm: lookup: at 23 [h=-1 ,ksize=-1 ,dsize=-1]
jdbm: allocateSpace: avail_count=1
jdbm: write: @387530
jdbm: saving header.
jdbm: saving directory.
--
--------------------------------------
Toshiki Murata
Oki Electric Industry Co., Ltd.
Kansai Laboratory
mura@kansai.oki.co.jp
--------------------------------------
Received on Monday, 3 March 1997 03:28:26 UTC