Re: Starting work on Indexed DB v2 spec - feedback wanted

The level / node.js community already build a prototype on top of IDB (
https://github.com/maxogden/level.js )

It supports:

 open(name)
 get(key) => Uint8Array
 del(key)
 put(key, Uint8Array)
 batch(Array<{ type: 'put' or 'del', key: String, value: Uint8Array>})
 iterator({ start: String, key: String, order: 'asc' or 'desc' }) => {
   next() => Uint8Array or null,
   end()
 }
 close()
 destroy() // cleans up the databases

So it supports async get / del / put + write btaches + iterators.

I dont think it supports snapshots or custom comparators.

It does support multiple instances, i.e. you can open multiple named
databases backed by a file location (which gets mapped to a unique idb db
name).

The values can just be String or Uint8Array and the same for the keys.

The level.js prototype is callback flavored but converting it to promises
would be trivial.

There is value in supporting batch & iterators as those are the primitives
that allow people to build their own transactions and indexes.


On Thu, Apr 17, 2014 at 1:56 PM, Joshua Bell <jsbell@google.com> wrote:

> On Thu, Apr 17, 2014 at 1:22 PM, Tim Caswell <tim@creationix.com> wrote:
>
>> Personally, the main thing I want to see is expose simpler and lower
>> level APIs.  For my uses (backend to git server), the leveldb API is plenty
>> powerful.  Most of the time I'm using IndexedDB, I'm getting frustrated
>> because it's way more complex than I need and gets in the way and slows
>> things down.
>>
>> Would it make sense to standardize on a simpler set of APIs similar to
>> what levelDB offers and expose that in addition to what indexedDB currently
>> exposes?  Or would this make sense as a new API apart from IDB?
>>
>
> That sounds like a separate storage system to me, although you could
> imagine it shares some primitives with Indexed DB (e.g. keys/ordering).
>
> How much of leveldb's API you consider part of the minimum set - write
> batches? iterators? snapshots? custom comparators? multiple instances per
> application? And are IDB-style keys / serialized script values appropriate,
> or is that extra overhead over e.g. just strings?
>
> You may want to try prototyping this on top of Indexed DB as a library,
> and see what others think. It'd basically just be hiding most of the IDB
> API (versions, transactions, stores, indexes) behind functions that return
> Promises.
>
>
>> As a JS developer, I'd much rather see fast, simple, yet powerful
>> primitives over application-level databases with indexes and transactions
>> baked in.  Chrome implements IDB on top of LevelDB, so it has just enough
>> primitives to make more complex systems.
>>
>> But for applications like mine that use immutable storage and hashes for
>> all lookups don't need or want the advanced features added on top.  IDB is
>> a serious performance bottleneck in my apps and when using LevelDB in
>> node.js, my same logic runs a *lot* faster and using a lot less code.
>>
>> -Tim Caswell
>>
>>
>> On Wed, Apr 16, 2014 at 1:49 PM, Joshua Bell <jsbell@google.com> wrote:
>>
>>> At the April 2014 WebApps WG F2F [1] there was general agreement that
>>> moving forward with an Indexed Database "v2" spec was a good idea. Ali
>>> Alabbas (Microsoft) has volunteered to co-edit the spec with me.
>>> Maintaining compatibility is the highest priority; this will not break the
>>> existing API.
>>>
>>> We've been tracking additional features for quite some time now, both on
>>> the wiki [2] and bug tracker [3]. Several are very straightforward
>>> (continuePrimaryKey, batch gets, binary keys, ...) and have already been
>>> implemented in some user agents, and it will be helpful to document these.
>>> Others proposals (URLs, Promises, full text search, ...) are much more
>>> complex and will require additional implementation feedback; we plan to add
>>> features to the v2 spec based on implementer acceptance.
>>>
>>> This is an informal call for feedback to implementers on what is missing
>>> from v1:
>>>
>>> * What features and functionality do you see as important to include?
>>> * How would you prioritize the features?
>>>
>>> If there's anything you think is missing from the wiki [2], or want to
>>> comment on the importance of a particular feature, please call it out -
>>> replying here is great. This will help implementers decide what work to
>>> prioritize, which will drive the spec work. We'd also like to keep the v2
>>> cycle shorter than the v1 cycle was, so timely feedback is appreciated -
>>> there's always room for a "v3".
>>>
>>> [1] http://www.w3.org/2014/04/10-webapps-minutes.html
>>> [2] http://www.w3.org/2008/webapps/wiki/IndexedDatabaseFeatures
>>> [3]
>>> https://www.w3.org/Bugs/Public/buglist.cgi?bug_status=RESOLVED&component=Indexed%20Database%20API&list_id=34841&product=WebAppsWG&query_format=advanced&resolution=LATER
>>>
>>> PS: Big thanks to Zhiqiang Zhang for his Indexed DB implementation
>>> report, also presented at the F2F.
>>>
>>
>>
>