Re: Indexed Database API

Filed: http://www.w3.org/Bugs/Public/show_bug.cgi?id=12310

On Fri, Mar 4, 2011 at 5:45 PM, Jeremy Orlow <jorlow@chromium.org> wrote:

> On Fri, Mar 4, 2011 at 5:36 PM, Jonas Sicking <jonas@sicking.cc> wrote:
>
>> A few observations:
>>
>> 1. It seems like a fairly rare use case to have to jump to item #100
>> without first also observing item 1-99. When showing a paged view
>> which lets the user to jump directly to, say, page 5 it can certainly
>> happen, but the page could optimize for the case when the user first
>> goes through page 1-4.
>> 2. Since it's not a common case, adding support for it just on
>> cursors, rather than cursors and objectStores, seems enough. Would be
>> as simple as adding a .advance (or similarly named function) which
>> simply takes an integer. I don't see that we need to support jumping
>> in a arbitrary direction since we don't allow continue() in an
>> arbitrary direction.
>> 3. We do have a bit of a hole in our index-cursor API. While you can
>> start the cursor at an arbitrary key, you can only start it at the
>> first entry with that key in the case when there are duplicate keys.
>> So if you iterate an index 10 records at a time, even if you never
>> need to skip any entries, you can't always resume where you left off,
>> even if you know the exact key+primaryKey for the record you want to
>> resume at.
>>
>
> I agree with all of this reasoning.
>
>
>> 4. While I agree that count() seems like a useful function, my concern
>> is that people might think it's a cheap operation.
>
>
> This is my concern with your "getAll" function, btw.  :-)
>
>
>> Getting the count
>> for a full objectStore or index should be quick, but getting the count
>> for a given key range (such as on a cursor) seems like it could be
>> expensive. My b-tree knowledge isn't the best, but isn't there a risk
>> that you have to linearly walk the full keyrange? Or does b-trees keep
>> an exact count of record in each node? Even if linear walking is
>> required, there might not be much we can do, and the best we can do is
>> to document that this is a slow operation.
>>
>
> I don't think we should limit our thinking to btrees, but it seems as
> though implementations could keep track of the number of children under a
> particular node, in which case it should be faster than linear.
>
> COUNT(*) is a very popular function in SQL (even with WHERE clauses).  It
> seems like there will be some cases where the implementor truly does need a
> count but not the data.  And given that at least some implementations should
> be able to optimize this, I think we should give them an API call.
>
> J
>
> On Fri, Mar 4, 2011 at 2:32 PM, Jeremy Orlow <jorlow@chromium.org> wrote:
>> > On Fri, Mar 4, 2011 at 1:38 PM, ben turner <bent.mozilla@gmail.com>
>> wrote:
>> >>
>> >> Firefox does lazily deserialize cursor values, so the slowdown you're
>> >> noticing is most likely due to us preserving the order of request
>> >> callbacks by queuing every continue() call in line with the rest of
>> >> the transaction. Jonas had proposed a faster, high performance cursor
>> >> that did not respect this ordering, maybe that's all that you'd need.
>> >>
>> >> However, a few thoughts:
>> >>
>> >> 1. How do you know Page 5 even exists? We haven't exposed a count()
>> >> function yet...
>> >> 2. I think we should expose a count() function!
>> >> 3. Maybe we should expose a getAt(long index, <enum> direction);
>> >> function on indexes and objectStores?
>> >
>> > A count function might make sense.
>> > But in this case, you could just jump forward to page 5 and see if you
>> get
>> > an error or not.
>> > I'd lean towards just adding jumping forward to cursors for now though.
>>  If
>> > getting a single item at some position is popular, then we can always
>> add
>> > it.
>> > Let's avoid adding prioritization of cursor.continue calls unless we
>> have
>> > absolutely no other choice.
>> > J
>> >>
>> >> On Fri, Mar 4, 2011 at 12:11 PM, Olli Pettay <Olli.Pettay@helsinki.fi>
>> >> wrote:
>> >> > On 03/02/2011 09:02 AM, Ben Dilts wrote:
>> >> >>
>> >> >> Why is there no mechanism for paging results, a la SQL's "limit"?
>>  If I
>> >> >> want entries in positions 140-159 from an index, I have to call
>> >> >> continue() on a cursor 139 times, which in turn unserializes 139
>> >> >> objects
>> >> >> from my store that I don't care about, which in FF4 is making a
>> lookup
>> >> >> in IndexedDB sometimes take many seconds for even a few records.
>> >> >
>> >> > Sounds like there is something to optimize in the implementation.
>> >> > Have you filed a bug
>> >> > https://bugzilla.mozilla.org/enter_bug.cgi?product=Core
>> >> > component DOM ?
>> >> > If not, please do so and attach a *minimal* testcase.
>> >> >
>> >> >
>> >> > Thanks,
>> >> >
>> >> >
>> >> > -Olli
>> >> >
>> >> >
>> >> >  It
>> >> >>
>> >> >> makes no sense--am I just missing something in the spec?
>> >> >>
>> >> >>
>> >> >> Ben Dilts
>> >> >
>> >> >
>> >>
>> >
>> >
>>
>
>