Re: [IndexedDB] KeyPaths and missing properties.

On Fri, May 21, 2010 at 12:39 AM, Jonas Sicking <jonas@sicking.cc> wrote:

> >> So you'd have to pass in the javascript expression as a string. This
> >> certainly works but is more than a little ugly. It also complicates
> >> the implementation a good bit since it now has to include a javascript
> >> engine. Not a huge issue given that all browsers has one anyway, but
> >> feels a bit iffy.
> >
> > Hm.  I'm looking at the spec, and I can't find where (if anywhere) it
> talks
> > about what's allowed as a key path.  I guess I had assumed that it was
> > essentially javascript (or maybe some subset) and that we'd at least need
> to
> > be parsing it anyway.  If we don't adopt this idea because we're worried
> > about depending on javascript, then we should spec out the keyPath syntax
> > pretty precisely to ensure we don't have such a dependency.
> > <snip>
>
> I think the intent of the current syntax is that it's a '.' separated
> list of property names. But yes, it definitely needs to be precisely
> defined.
>

I'll create a bug for this and the other items we've already decided on in
this thread.

 >> > I still think it's an interesting idea, though I'm not sold on it.
> >> > Especially for the first version of the spec.
> >>
> >> It seems like there would be a lot of edge cases to define here. First
> >> of all, how is the value passed in to this expression? Do we say that
> >> it's available through some "value" variable? So that if you want to
> >> index on the "foo" property, you pass in an expression like
> >> "value.foo"? Or do we want the value to be the global object, so that
> >> if you wanted to index on the "foo" property the expression would
> >> simply be "foo"?
> >
> > Since we're already talking about requiring that data being inserted into
> > objectStores with a keyPath (for its primary key or in one of its
> indexes),
> > setting it as the global object seems reasonable.  And it matches what's
> > currently specced for the simple 1 entityStore entry to 1 index entry
> (per
> > index) case.
>
> You also have to specify how the value is returned. I.e. does the
> expression need to end with 'return X'? Or do we index on the result
> of the value returned from the last executed expression?
>
> Also, I think that if the value is the global object, then I declaring
> variables inside the expression modifies the value as global variables
> are set on the global object.
>
> >> Also, what happens if the javascript expression modifies the value?
> >> Does the implementation have to clone the value before calling each
> >> "index expression"?
> >
> > In order of how much I like the idea:
> > 1) In an ideal world, we'd spec it to be read only, but I'm not sure if
> most
> > JS engines have an easy way to do something like that.
> > 2) Another possibility is to make the order in which indexes are
> processed
> > deterministic.  That way, if someone does modify it, it'll at least
> > be consistent.
> > 3) Cloning is another possibility, but it seems like it'd have a
> performance
> > impact.  Maybe optimized implementations could copy-on-write it, though?
>
> I think 1 and 3 suffers from the same problem of not being something
> that JS engines usually need to do, and so likely isn't implemented.
> In both cases you have to be able to mark an object graph such that
> you take special action if it's modified.
>
> And for 2, do you do the structured clone before or after you run the
> keyPath expressions? Not until after you've created the structured
> clone will you know if the value can even be stored.
>
> In short, I'd like to see a comprehensive proposal :) Then I could
> take that to the JS team and ask if it's implementable.
>

Per other threads, it looks like we should make sure IndexedDB does not
depend on JavaScript, so please ignore my proposal.




On Thu, May 20, 2010 at 4:56 PM, Jonas Sicking <jonas@sicking.cc> wrote:

> On Thu, May 20, 2010 at 7:55 AM, Andrei Popescu <andreip@google.com>
> wrote:
> > Hi,
> >
> > On Thu, May 20, 2010 at 10:47 AM, Jeremy Orlow <jorlow@chromium.org>
> wrote:
> >> On Thu, May 20, 2010 at 1:24 AM, Jonas Sicking <jonas@sicking.cc>
> wrote:
> >>> It seems like there would be a lot of edge cases to define here. First
> >>> of all, how is the value passed in to this expression? Do we say that
> >>> it's available through some "value" variable? So that if you want to
> >>> index on the "foo" property, you pass in an expression like
> >>> "value.foo"? Or do we want the value to be the global object, so that
> >>> if you wanted to index on the "foo" property the expression would
> >>> simply be "foo"?
> >>
> >> Since we're already talking about requiring that data being inserted
> into
> >> objectStores with a keyPath (for its primary key or in one of its
> indexes),
> >> setting it as the global object seems reasonable.  And it matches what's
> >> currently specced for the simple 1 entityStore entry to 1 index entry
> (per
> >> index) case.
> >>>
> >>> Also, what happens if the javascript expression modifies the value?
> >>> Does the implementation have to clone the value before calling each
> >>> "index expression"?
> >>
> >> In order of how much I like the idea:
> >> 1) In an ideal world, we'd spec it to be read only, but I'm not sure if
> most
> >> JS engines have an easy way to do something like that.
> >> 2) Another possibility is to make the order in which indexes are
> processed
> >> deterministic.  That way, if someone does modify it, it'll at least
> >> be consistent.
> >> 3) Cloning is another possibility, but it seems like it'd have a
> performance
> >> impact.  Maybe optimized implementations could copy-on-write it, though?
> >>
> >
> >
> > While it's true that allowing the keyPath to be any javascript
> > expression would be very elegant and flexible (although probably quite
> > difficult to explain in the spec), maybe it's worth considering a
> > simpler solution? For instance, could the keyPath be simply an array
> > of strings, with each string denoting the name of a property of the
> > objects in the store. So in Jonas' example:
> >
> > { id: 5, givenName: "Benny", otherNames: ["Göran", "Bror"],
> > familyName: "Andersson", age: 63, interest: "Music" }
> >
> > The keyPath could be set to
> >
> > ["givenName", "otherNames", "familyName"].
> >
> > The indexable data for this record would therefore be {"Benny",
> > "Göran", "Bror", "Andersson"}.
>
> This wouldn't solve the case when the key is the result of a
> calculation, such as my age-at-time-of-death example. Consider also
> wanting to store objects like
>
> { id: 5, name: "Benny Andersson", address: "1 Ohai Street\n Wahiawa, HI
> 96786" }
>
> but wanting to index on the first name or on the state in the address.


Can you do this (or any of your other use cases) in SQL?  As far as I know,
entries in SQL indexes are always 1:1 with rows.  (In other words, there's
one index entry per index per row.)  I believe all the use cases you've
brought up so far would typically be solved in SQL by creating another table
and normalizing the data.  I recognize that joins are more difficult in
indexedDB, but I still see this as pretty compelling evidence that we don't
_need_ these abilities in v1.

I feel strongly we should remove the add/modify/addOrModify/remove methods
from IDBIndex.  I'm leaning towards saying that the only way to populate an
index is via the keyPath, but I'd be OK with making index values be an
optional parameter to add/modify/addOrModify.

Here's my proposal for if we did the latter:
 IDBObjectStore.add/modify/addOrModify would all take an optional parameter.
 This parameter would be an associative array that maps the index name to a
key.  If the key is an array then we interpret it as being a list of keys
that should map to the value being added to the objectStore.  The omission
of an index name in the associative array is the equivalent of passing in a
zero length array of keys and means that no entries in that index will map
to that value in the objectStore.  IDBIndex would no longer have its
add/modify/addOrModify/remove methods.

Here's a proposal for the former: IDBIndex would no longer have its
add/modify/addOrModify/remove methods.  When calling
IDBObjectStore.createIndex, keyPath must not be null.  If it is, an error
will be thrown.  When inserting an element into an objectStore (via
add/modify/addOrModify), the keyPath will be looked up in the value being
inserted.  If the keyPath doesn't exist in the value, then a key of null is
assumed.

J