Re: IndexedDB: Binary Keys

On Mon, May 21, 2012 at 10:09 AM, Joran Greef <joran@ronomon.com> wrote:

> IndexedDB supports binary values as per the structured clone algorithm
> as implemented in Chrome and Firefox.
>
> IndexedDB needs to support binary keys (ArrayBuffer, TypedArrays).
>
> Many popular KV stores accept binary keys (BDB, Tokyo, LevelDB). The
> Chrome implementation of IDB is already serializing keys to binary.
>
> JS is moving more and more towards binary data across the board
> (WebSockets, TypedArrays, FileSystemAPI). IDB is not quite there if it
> does not support binary keys.
>
> Binary keys are more efficient than Base 64 encoded keys, e.g. a 128
> bit key in base 256 is 16 bytes, but 22 bytes in base 64.
>

Would using DOMString keys work for your use case?  DOMString is defined as
a list of 16-bit integers and I don't see anything in the IndexedDB
specification that treats the strings as if they contained Unicode, so I
would naively expect that packing your 64 bits into a DOMString of length 4
(16 bits / unit) would work fine.

- James


>
> Am working on a production system storing 3 million keys in IndexedDB.
> In about 6 months it will be storing 60 million keys in IndexedDB.
>
> Without support for binary keys, that's 330mb wasted storage
> (60,000,000 * (22 - 16)) not to mention the wasted CPU overhead spent
> Base64 encoding and decoding keys.
>
>