W3C home > Mailing lists > Public > public-webapps@w3.org > April to June 2012

Re: IndexedDB: Binary Keys

From: James Robinson <jamesr@google.com>
Date: Mon, 21 May 2012 13:24:58 -0700
Message-ID: <CAD73md+m5xbONsL-GQhfGKQJ38h623Rh64PQNH9DYWE08Rn4FA@mail.gmail.com>
To: Joran Greef <joran@ronomon.com>
Cc: public-webapps@w3.org
On Mon, May 21, 2012 at 10:09 AM, Joran Greef <joran@ronomon.com> wrote:

> IndexedDB supports binary values as per the structured clone algorithm
> as implemented in Chrome and Firefox.
>
> IndexedDB needs to support binary keys (ArrayBuffer, TypedArrays).
>
> Many popular KV stores accept binary keys (BDB, Tokyo, LevelDB). The
> Chrome implementation of IDB is already serializing keys to binary.
>
> JS is moving more and more towards binary data across the board
> (WebSockets, TypedArrays, FileSystemAPI). IDB is not quite there if it
> does not support binary keys.
>
> Binary keys are more efficient than Base 64 encoded keys, e.g. a 128
> bit key in base 256 is 16 bytes, but 22 bytes in base 64.
>

Would using DOMString keys work for your use case?  DOMString is defined as
a list of 16-bit integers and I don't see anything in the IndexedDB
specification that treats the strings as if they contained Unicode, so I
would naively expect that packing your 64 bits into a DOMString of length 4
(16 bits / unit) would work fine.

- James


>
> Am working on a production system storing 3 million keys in IndexedDB.
> In about 6 months it will be storing 60 million keys in IndexedDB.
>
> Without support for binary keys, that's 330mb wasted storage
> (60,000,000 * (22 - 16)) not to mention the wasted CPU overhead spent
> Base64 encoding and decoding keys.
>
>
Received on Monday, 21 May 2012 20:25:47 GMT

This archive was generated by hypermail 2.3.1 : Tuesday, 26 March 2013 18:49:52 GMT