W3C home > Mailing lists > Public > public-webapps@w3.org > January to March 2010

[IndexedDB] Inverted indexes (for full text search)

From: Jeremy Orlow <jorlow@google.com>
Date: Thu, 18 Feb 2010 14:16:34 +0000
Message-ID: <5dd9e5c51002180616n6d701b95iabb28ca1b954eea8@mail.gmail.com>
To: public-webapps WG <public-webapps@w3.org>
Cc: Scott Hess <shess@google.com>
In addition to looking at the UA side of the IndexedDB AI, I've been talking
to our web apps teams about it to get their initial impressions.  I'm going
to batch most of the feedback together in another email, but this is a big
enough feature and important enough to all of those teams that I wanted to
keep this in its own thread.

All of the Google apps put a large emphasis on search.  For example, the
search within Gmail, reader, docs, wave, etc.  In fact, I actually can't
(off the top of my head) think of a Google app that we'd want to work
offline that doesn't put a heavy emphasis on search.  Thus search is a
pretty important part of the offline experience for any of our apps.

In Gears (and SQLite) there's full text search.  This did a decent job of
enabling offline search and providing a decent experience to offline users.
 Full text search is a pretty large beast though, and carries with it a lot
of baggage...much of it tied in closely with SQL.

Doing full text search completely in JavaScript (even with IndexedDB) is
almost certainly not manageable for anything other than trivial amounts of
data.  (Definitely not one of our email accounts.)  Implementing full text
search in JS with Inverted
indexes<http://en.wikipedia.org/wiki/Inverted_index> doing
the heavy lifting should be practical though.  As such, we'd like to propose
Inverted Indexes be added to the spec.

Anyhow, I'd like to know what concerns/objections others might have to
adding this to IndexedDB.  From some f2f conversations at TPAC, it sounded
like there was general support at a high level, but I'm sure there will be
many details to hammer out.  I think we (Google) would be happy to come up
with a concrete proposal if that'd be helpful.  Thoughts?

Thanks!
Jeremy

P.S. Scott Hess knows _way_ more about the details of full text search than
I do, so I've cc'ed him on this.
Received on Thursday, 18 February 2010 14:17:26 GMT

This archive was generated by hypermail 2.3.1 : Tuesday, 26 March 2013 18:49:37 GMT