Re: IndexedDB: Binary Keys

On 5/21/12 8:03 PM, ext Jonas Sicking wrote:
> On Mon, May 21, 2012 at 10:09 AM, Joran Greef<joran@ronomon.com>  wrote:
>> IndexedDB supports binary values as per the structured clone algorithm
>> as implemented in Chrome and Firefox.
>>
>> IndexedDB needs to support binary keys (ArrayBuffer, TypedArrays).
>>
>> Many popular KV stores accept binary keys (BDB, Tokyo, LevelDB). The
>> Chrome implementation of IDB is already serializing keys to binary.
>>
>> JS is moving more and more towards binary data across the board
>> (WebSockets, TypedArrays, FileSystemAPI). IDB is not quite there if it
>> does not support binary keys.
>>
>> Binary keys are more efficient than Base 64 encoded keys, e.g. a 128
>> bit key in base 256 is 16 bytes, but 22 bytes in base 64.
>>
>> Am working on a production system storing 3 million keys in IndexedDB.
>> In about 6 months it will be storing 60 million keys in IndexedDB.
>>
>> Without support for binary keys, that's 330mb wasted storage
>> (60,000,000 * (22 - 16)) not to mention the wasted CPU overhead spent
>> Base64 encoding and decoding keys.
> I agree that we should introduce this, but I think it's too late to
> add for version one (which is about to go to last call any day now).

(Yes, IDB LCWD will be published on May 24 (or May 29 at the latest).)
>
> I'd be happy to add it to version 2 though. However the current
> situation regarding binary data in Javascript is still pretty chaotic.
> See for example the resent change to switch a bunch of APIs over from
> ArrayBuffer to ArrayBufferViews.

FYI, I added Binary Keys to the IDB Features wiki 
<http://www.w3.org/2008/webapps/wiki/IndexedDatabaseFeatures>.

(It may be a good idea for one of our active IDB contributors to make 
sure that document is current.)

-AB

Received on Tuesday, 22 May 2012 10:28:39 UTC