Re: Indexed Database API

On Fri, Mar 4, 2011 at 5:36 PM, Jonas Sicking <jonas@sicking.cc> wrote:

> A few observations:
>
> 1. It seems like a fairly rare use case to have to jump to item #100
> without first also observing item 1-99. When showing a paged view
> which lets the user to jump directly to, say, page 5 it can certainly
> happen, but the page could optimize for the case when the user first
> goes through page 1-4.
> 2. Since it's not a common case, adding support for it just on
> cursors, rather than cursors and objectStores, seems enough. Would be
> as simple as adding a .advance (or similarly named function) which
> simply takes an integer. I don't see that we need to support jumping
> in a arbitrary direction since we don't allow continue() in an
> arbitrary direction.
> 3. We do have a bit of a hole in our index-cursor API. While you can
> start the cursor at an arbitrary key, you can only start it at the
> first entry with that key in the case when there are duplicate keys.
> So if you iterate an index 10 records at a time, even if you never
> need to skip any entries, you can't always resume where you left off,
> even if you know the exact key+primaryKey for the record you want to
> resume at.
>

I agree with all of this reasoning.


> 4. While I agree that count() seems like a useful function, my concern
> is that people might think it's a cheap operation.


This is my concern with your "getAll" function, btw.  :-)


> Getting the count
> for a full objectStore or index should be quick, but getting the count
> for a given key range (such as on a cursor) seems like it could be
> expensive. My b-tree knowledge isn't the best, but isn't there a risk
> that you have to linearly walk the full keyrange? Or does b-trees keep
> an exact count of record in each node? Even if linear walking is
> required, there might not be much we can do, and the best we can do is
> to document that this is a slow operation.
>

I don't think we should limit our thinking to btrees, but it seems as though
implementations could keep track of the number of children under a
particular node, in which case it should be faster than linear.

COUNT(*) is a very popular function in SQL (even with WHERE clauses).  It
seems like there will be some cases where the implementor truly does need a
count but not the data.  And given that at least some implementations should
be able to optimize this, I think we should give them an API call.

J

On Fri, Mar 4, 2011 at 2:32 PM, Jeremy Orlow <jorlow@chromium.org> wrote:
> > On Fri, Mar 4, 2011 at 1:38 PM, ben turner <bent.mozilla@gmail.com>
> wrote:
> >>
> >> Firefox does lazily deserialize cursor values, so the slowdown you're
> >> noticing is most likely due to us preserving the order of request
> >> callbacks by queuing every continue() call in line with the rest of
> >> the transaction. Jonas had proposed a faster, high performance cursor
> >> that did not respect this ordering, maybe that's all that you'd need.
> >>
> >> However, a few thoughts:
> >>
> >> 1. How do you know Page 5 even exists? We haven't exposed a count()
> >> function yet...
> >> 2. I think we should expose a count() function!
> >> 3. Maybe we should expose a getAt(long index, <enum> direction);
> >> function on indexes and objectStores?
> >
> > A count function might make sense.
> > But in this case, you could just jump forward to page 5 and see if you
> get
> > an error or not.
> > I'd lean towards just adding jumping forward to cursors for now though.
>  If
> > getting a single item at some position is popular, then we can always add
> > it.
> > Let's avoid adding prioritization of cursor.continue calls unless we have
> > absolutely no other choice.
> > J
> >>
> >> On Fri, Mar 4, 2011 at 12:11 PM, Olli Pettay <Olli.Pettay@helsinki.fi>
> >> wrote:
> >> > On 03/02/2011 09:02 AM, Ben Dilts wrote:
> >> >>
> >> >> Why is there no mechanism for paging results, a la SQL's "limit"?  If
> I
> >> >> want entries in positions 140-159 from an index, I have to call
> >> >> continue() on a cursor 139 times, which in turn unserializes 139
> >> >> objects
> >> >> from my store that I don't care about, which in FF4 is making a
> lookup
> >> >> in IndexedDB sometimes take many seconds for even a few records.
> >> >
> >> > Sounds like there is something to optimize in the implementation.
> >> > Have you filed a bug
> >> > https://bugzilla.mozilla.org/enter_bug.cgi?product=Core
> >> > component DOM ?
> >> > If not, please do so and attach a *minimal* testcase.
> >> >
> >> >
> >> > Thanks,
> >> >
> >> >
> >> > -Olli
> >> >
> >> >
> >> >  It
> >> >>
> >> >> makes no sense--am I just missing something in the spec?
> >> >>
> >> >>
> >> >> Ben Dilts
> >> >
> >> >
> >>
> >
> >
>