Re: [IndexedDB] Computed indexes

I trimmed this thread quite a bit since there now seems to be less
talking past each other. Feel free to bring back any part if you think
I've trimmed too agressively.

>> >> > What is the difference between an "expressionIndex" and a keyPath?
>> >> >  It
>> >> > seems
>> >> > like they're doing about the same thing.  My proposal to allow
>> >> > keyPath
>> >> > to be
>> >> > artibrary JavaScript and then have the value being inserted be the
>> >> > global
>> >> > scope actually sounds almost identical to what you're doing....except
>> >> > more
>> >> > in the relm of what JS engines already do (since it's much like an
>> >> > eval).
>> >>
>> >> It is about the same thing yes. Though presumably an implementation
>> >> could optimize "evaluating" the keyPath heavily. Much more so than a
>> >> generic expression.
>> >
>> > Howso?  Any implementation should be able to cache the compiled/JITed
>> > form
>> > of the string passed into it's internal implementation of eval.
>>
>> Executing a complied expression, including firing up all the contexts
>> needed to run a piece of javascript, is at least in the firefox js
>> engine, significantly more expensive than simply grabbing the relevant
>> value as you serialize the javascript object into the database.
>>
>> Additionally, for a simple keyPath you don't have to worry about
>> cloning objects or implementing copy-on-write semantics.
>>
>> Finally, you can perform optimizations such as not updating indexes if
>> you notice that the relevant property doesn't change during calls to
>> objectStore.put() and cursor.update().
>>
>> All in all I think keyPaths are preferrable when they are possible.
>> Full on expressions will likely always be slower and more cumbersome,
>> but they seem better than the alternatives when the value isn't stored
>> directly in the stored object.
>
> Ahh!  I think I see why we're both having so hard of a time understanding
> each other.  I've been talking about various proposals for what keyPath
> would be and you're talking about a proposal that'd be in addition to some
> "simple" keyPath proposal (that could easily have the properties you just
> mentioned above).  That explains a lot.
> I definitely like the idea of some simple keyPath concept optionally
> augmented by a more complex one that can actually calculate the key--whether
> represented as a function body or an expression.

Cool! Lets attempt to hash out the details then :)

>> > And as long as there are
>> > fundamental reasons why one can't be optimized, implementational
>> > challenges
>> > really shouldn't impact how we design the API.  At all.
>> > Anyway, I'll continue thinking about this, but I'm far from convinced
>> > that
>> > allowing javascript (whether it's like the body of a function or like
>> > it's
>> > evaled) is necessary for v1.  It adds a lot of complexity and I honestly
>> > doubt we understand the problems well enough at the moment to come up
>> > with
>> > the best solution.  I'd much rather punt for now.  But if you feel
>> > strongly
>> > we need this for v1, I think we need to find some examples in SQL or
>> > other
>> > database engines to base our implementation on.  Otherwise we're
>> > building an
>> > API on top of a tower of assumptions.
>>
>> I'd prefer to leave expressions out of v1 as well. However I would be
>> worried about leaving out the ability to index on computed values
>> entirely. I.e. remove the ability to index on anything that isn't a
>> plain value in the stored object. And it doesn't seem like there is
>> much support for the way that the spec supports them now which is
>> through explicit calls to add/remove items out of an index.
>
> I definitely think we need to remove the concept of managing an index
> manually.

I agree. Filed http://www.w3.org/Bugs/Public/show_bug.cgi?id=10027

> Whether it's worth putting this in v1 or adding some way to
> provide index values when doing an objectStore.put/add/etc, I'm honestly not
> sure.
> To be honest, I don't think any of the use cases presented so far are a slam
> dunk for the need to support computed indexes (vs. just making someone store
> the computed value in the object store value itself and then indexing with a
> simple keyPath), but I definitely do think it'll be a useful feature.  And
> it sounds like you guys are pretty convinced of its importance, so I'm happy
> to go along with it.

I'm pretty convinced that complex indexes is something that people
need to do. However it's entirely possible that simply using a
separate objectStore is good enough for v1. More details below.

>> Though maybe people can use separate objectStores which hold computed
>> values. That might certainly be good enough for version 1. I'll have
>> to think more about that.
>
> Hmmm....I'm not sure I follow.  Could you explain more thoroughly?

To bring back an old example. Say that you're storing objects like

{ name: "Elvis", born: "January 8, 1935", died: "August 16, 1977" }
{ name: "Gustav III", born: "24 January 1746", died: "29 March 1792" }
{ name: "Benny Andersson", born: "16 December 1946" }

and want to index on the age-at-death. What you'd do is that you set
up a separate objectStore using the following calls:

store = createObjectStore("AgeAtDeathStoreIndex", "name", false);
store.createIndex("ageIndex", "age", false);

When the above three objects are stored, you also store the following
two object into the "AgeAtDeathStoreIndex" objecStore:

{ name: "Elvis", age: 42 }
{ name: "Gustav III", age: 46 }

You can then easily search based on age at death by first looking
something up in the ageIndex and then looking up the found name in the
original objecStore. Something like:

// only finds one person
trans = db.transaction(["people", "AgeAtDeathStoreIndex"]);
trans.objectStore("AgeAtDeathStoreIndex").index("ageIndex").get(42).onsuccess
= function(e) {
  trans.objectStore("people").get(e.value).onsuccess = function(e) {
    alert(e.value.name + " was 42 at time of death");
  }
}

This is certainly more work, but at least it avoids *forcing* us to
put anything into v1 of IndexedDB.

/ Jonas

Received on Monday, 28 June 2010 21:27:54 UTC