Re: [IndexedDB] Two Real World Use-Cases

If you are operating on indexes then you do not have a 'join' language as
you are operating on sets. To have a join you need to be operating on
relations. A relation is commonly visualised as a row in a table in a
relational database, With IDB this would be the union of all the
property-sets of the objects in the index. A complete set of relational
operators would be:

project
restrict
rename
join
union
difference

In most useful syntaxes you don't need rename as the other methods handle
renaming attributes already. Join is traditionally a Cartesian-product, but
a natural-join can be substituted without losing completeness. Intersection
is not included as it is easily derived from union and
(symmetric)difference.


Cheers,
Keean.

On 2 March 2011 06:35, Joran Greef <joran@ronomon.com> wrote:

> On 01 Mar 2011, at 7:27 PM, Jeremy Orlow wrote:
>
> > 1. Be able to put an object and pass an array of index names which must
> reference the object. This may remove the need for a complicated indexing
> spec (perhaps the reason why this issue has been pushed into the future) and
> give developers all the flexibility they need.
> >
> > You're talking about having multiple entries in a single index that point
> towards the same primary key?  If so, then I strongly agree, and I think
> others agree as well.  It's mostly a question of syntax.  A while ago we
> brainstormed a couple possibilities.  I'll try to send out a proposal this
> week.  I think this + compound keys should probably be our last v1 features
> though.  (Though they almost certainly won't make Chrome 11 or Firefox 4,
> unfortunately, hopefully they'll be done in the next version of each, and
> hopefully that release with be fairly soon after for both.)
>
> Yes, for example this user object { name: "Joran Greef", emails: ["
> joran@ronomon.com", "jorangreef@gmail.com"] } with indexes on the "emails"
> property, would be found in the "joran@ronomon.com" index as well as in
> the "jorangreef@gmail.com" index.
>
> What I've been thinking though is that the problem even with formally
> specifying indexes in advance of object put calls, is that this pushes too
> much application model logic into the database layer, making the database
> enforce a schema (at least in terms of indexes). Of course IDB facilitates
> migrations in the form of setVersion, but most schema migrations are also
> coupled with changes to the data itself, and this would still have to be
> done by the application in any event. So at the moment IDB takes too much
> responsibility on behalf of the application (computing indexes, pre-defined
> indexes, pseudo migrations) and not enough responsibility for pure database
> operations (index intersections and index unions).
>
> I would argue that things like migrations and schema's are best handled by
> the application, even if this is more work for the application, as most
> people will write wrappers for IDB in any event and IDB is supposed to be a
> core-level API. The acid-test must be that the database is oblivious to
> schemas or anything pre-defined or application-specific (i.e. stateless).
> Otherwise IDB risks being a database for newbies who wouldn't use it, and a
> database that others would treat as a KV anyway (see MySQL at FriendFeed).
>
> A suggested interface then for putting or deleting objects, would be:
> objectStore.put(object, ["indexname1", "indexname2", "indexname3"]) and then
> IDB would need to ensure that the object would be referenced by the given
> index names. When removing the object, the application would need to provide
> the indexes again (or IDB could keep track of the indexes associated with an
> object).
>
> Using a function to compute indexes would not work as this would entrap
> application-specific schema knowledge within the function (which would need
> to be persisted) and these may subsequently change in the application, which
> would then need a way to modify the function again. The key is that these
> things must be stateless.
>
> The objects must be opaque to IDB (no need for
> serialization/deserialization overhead at the DB layer). Things like
> key-paths etc. could be removed and the object id just passed in to put or
> delete calls.
>
> > 2. Be able to intersect and union indexes. This covers a tremendous
> amount of ground in terms of authorization and filtering.
> >
> > Our plan was to punt some sort of join language to v2.  Could you give a
> more concrete proposal for what we'd add?  It'd make it easier to see if
> it's something realistic for v1 or not.
>
> If you can perform intersect or union operations (and combinations of
> these) on indexes (which are essentially sets or sorted sets), then this
> would be the join language. It has the benefit that the interface would then
> be described in terms of operations on data structures (set operations on
> sets) rather than a custom language which would take longer to spec out.
>
> I've written databases over append-only files, S3, WebSQL and even
> LocalStorage (!) and from what I've found with my own applications, you
> could handle everything from multi-tenant authorization to adequate
> filtering with the following operations:
>
> 1. intersect([ index1, index2 ])
> 2. union([ index1, index2 ])
> 3. intersect([ union([ index1, index2 ]), index3, index4, index5, index6,
> index7 ])
>
> Hopefully, a join language described in terms of pure set operations would
> be much simpler to implement and easier to use and reason with.
>
> In fact I think if IDB offered only a single object store and an indexing
> system described above, it would be completely perfect. That's all that's
> needed. No need for a V2. Just a focus on high-performance thereafter.
>
>
>