Re: IndexedDB >> Proposed API Change: cursor.advance BACKWARD when direction is "prev"

Here is a jsfiddle showing how .advance behaves when the range is
restricted by .only

Create new e.g. 7 items with names like "marc" and tags like "w1 w3 w5 w2"
(random selection of tags with some tags appearing across multiple records
(per the attached image)

Enter "w2" or "w5" in the box next to 'get by tag' and click 'get by tag'

You'll see the first 2 matching items, with primary keys 7 and 6

Click 'get by tag' again and you'll see the next 2 matching items, with
primary keys 4 and 2

Click 'get by tag' again and you'll see the next and last matching item,
with primary key 1

Notice the way I advance the cursor each time in order to re-continue the
search from where I left off in the previous invocation is by using the
number of items already found

http://jsfiddle.net/marcfawzi/y5ELj/

It's the correct behavior but it would be easier imo if we have .find() for
what .continue() and .continue(key) does and use .continue() to mean
.advance(1) and .continue(n) to mean .advance(n)

But I could be totally wrong. Just a harmless feedback at this point.

:)


On Fri, May 23, 2014 at 1:07 PM, marc fawzi <marc.fawzi@gmail.com> wrote:

> <<
> Thanks for following up! At least two IDB implementers were worried that
> you'd found some browser bugs we couldn't reproduce.
> >>
> Yup. I had to figure this stuff out as the API is very low level (which is
> why it can also be used in very powerful ways and also potentially very
> confusing for the uninitiated)
>
> <<Assuming the store has [1,2,3,4,5,6,7,8,9] and the cursor's range is not
> restricted, if the cursor's key=7 and direction='prev' then I would expect
> after advance(2) that key=5. If you're seeing key=2 can you post a sample
> somewhere (e.g. jsfiddle.com?)>>
>
> In the case I have say 7 items [1,2,3,4,5,6,7] and the cursor's range is
> restricted by IDBKeyRange.only(val, "prev") ... so if the matching (or in
> range) items are at 7, 6, 4, 2, 1 then I can obtain them individually or in
> contiguous ranges by advancing the cursor on each consecutive invocation of
> my search routine, like so: on first invocation advance(1) from 7 to 6, on
> second invocation advance(2) from 7 to 4, on third invocation advance(3)
> from 7 to 2 and on fourth invocation advance(4) from 7 to 1. I could also
> use advance to advance by 1 within each invocation until no matching items
> are found but only up to 2 times an invocation (for a store with 700 or
> 70000 items we can advance by 1 about 200 times per invocation, but that's
> arbitrary)
>
>  I can definitely post a jsfiddle if you believe the above is not in
> accordance with the spec.
>
> As to continue(n) or continue(any string), i would make that
> .find(something)
>
>
>
> On Fri, May 23, 2014 at 10:41 AM, Joshua Bell <jsbell@google.com> wrote:
>
>> On Fri, May 23, 2014 at 9:40 AM, marc fawzi <marc.fawzi@gmail.com> wrote:
>>
>>> I thought .continue/advance was similar to the 'continue' statement in a
>>> for loop in that everything below the statement will be ignored and the
>>> loop would start again from the next index. So my console logging was
>>> giving confusing results. I figured it out and it works fine now.
>>>
>>
>> Thanks for following up! At least two IDB implementers were worried that
>> you'd found some browser bugs we couldn't reproduce.
>>
>>
>>>  For sanity's sake, I've resorted to adding a 'return'  in my code in
>>> the .success callback after every .advance and .continue so the execution
>>> flow is easier to follow. It's very confusing, from execution flow
>>> perspective, for execution to continue past .continue/.advance while at
>>> once looping asynchronously. I understand it's two different instances of
>>> the .success callback but it was entirely not clear to me from reading the
>>> docs on MDN (for example) that .advance / .continue are async.
>>>
>>
>> Long term, we expect JS to evolve better ways of expressing async calls
>> and using async results. Promises are a first step, and hopefully the
>> language also grows some syntax for them. IDB should jump on that train
>> somehow.
>>
>>
>>> Also, the description of .advance in browser vendor's documentation,
>>> e.g. on MDN, says "Advance the cursor position forward by two places" for
>>> cursor.advance(2) but what they should really say is "advance the cursor
>>> position forward by two results." For example, let's say cursor first
>>> landed on an item with primary key = 7, and you issue the statement
>>> cursor.advance(2), I would expect it to go to the item with primary key 5
>>> (for cursor direction = "prev") but instead it goes to the item with
>>> primary key 2 because that's the 2nd match for the range argument from the
>>> cursor's current position
>>>
>>
>> What "range argument" are you referring to?
>>
>> Assuming the store has [1,2,3,4,5,6,7,8,9] and the cursor's range is not
>> restricted, if the cursor's key=7 and direction='prev' then I would expect
>> after advance(2) that key=5. If you're seeing key=2 can you post a sample
>> somewhere (e.g. jsfiddle.com?)
>>
>>
>>> , which means that .advance(n) would be far more clear semantically
>>> speaking if it was simply done as .continue(n)  ... I guess if there is an
>>> understanding that the cursor is always at a matching item and that it
>>> could only continue/advance to the next/prev matching item, not literal
>>> 'positions' in the table (i.e. sequentially through the list of all items)
>>> then there would be no confusion but the very concept of a cursor is
>>> foreign to most front end developers, and that's where the confusion comes
>>> from for many.
>>>
>>> My inclination as a front end developer, so far removed from database
>>> terminology, would be
>>>
>>> 1) to deprecate .advance in favor of .continue(n) and
>>>
>>
>> continue(n) already has meaning - it jumps ahead to the key with value n
>>
>>
>>>
>>> 2) if it makes sense (you have to say why it may not) have
>>> .continue()/.continue(n) cause the return of the execution flow similar to
>>> 'continue' in a for loop.
>>>
>>
>> The API can't change the language - you return from functions via return
>> or throw. Further, there are reasons you may want to do further processing
>> after calling continue() - e.g. there may be multiple cursors (e.g. in a
>> join operation) or for better performance you can call continue() as early
>> as possible so that the database can do its work while you're processing
>> the previous result.
>>
>>
>>
>>>
>>> What do you think?
>>>
>>>
>>>
>>> On Wed, May 21, 2014 at 10:42 AM, Joshua Bell <jsbell@google.com> wrote:
>>>
>>>>
>>>>
>>>>
>>>> On Wed, May 21, 2014 at 7:32 AM, Arthur Barstow <art.barstow@gmail.com>wrote:
>>>>
>>>>> [ Bcc www-tag ; Marc - please use public-webapps for IDB discussions ]
>>>>>
>>>>> On 5/20/14 7:46 PM, marc fawzi wrote:
>>>>>
>>>>>> Hi everyone,
>>>>>>
>>>>>> I've been using IndexedDB for a week or so and I've noticed that
>>>>>> cursor.advance(n) will always move n items forward regardless of cursor
>>>>>> direction. In other words, when the cursor direction is set to "prev" as
>>>>>> in: range = IDBKeyRange.only(someValue, "prev") and primary key is
>>>>>> auto-incremented, the cursor, upon cursor.advance(n), will actually advance
>>>>>> n items in the opposite direction to the cursor.continue() operation.
>>>>>
>>>>>
>>>> That runs contrary to the spec. Both continue() and advance() reference
>>>> the "steps for iterating a cursor" which picks up the direction from the
>>>> cursor object; neither entry point alters the steps to affect the direction.
>>>>
>>>> When you say "you've noticed", are you observing a particular browser's
>>>> implementation or are you interpreting the spec? I did a quick test and
>>>> Chrome, Firefox, and IE all appear to behave as I expected when intermixing
>>>> continue() and advance() calls with direction 'prev' - the cursor always
>>>> moves in the same direction regardless of which call is used.
>>>>
>>>> Can you share sample code that demonstrates the problem, and indicate
>>>> which browser(s) you've tested?
>>>>
>>>>
>>>>
>>>>
>>>>>  This is not only an issue of "broken symmetry" but it presents an
>>>>>> obstacle to doing things like: keeping a record of the primaryKey of the
>>>>>> last found item (after calling cursor.continue for say 200 times) and, long
>>>>>> after the transaction has ended, call our search function again and, upon
>>>>>> finding the same item it found first last time, advance the cursor to the
>>>>>> previously recorded primary key and call cursor.continue 200 times, from
>>>>>> that offset, and repeat whenever you need to fetch the next 200 matching
>>>>>> items. Such algorithm works in the forward direction (from oldest to newest
>>>>>> item) because cursor.advance(n) can be used to position the cursor forward
>>>>>> at the previously recorded primary key (of last found item) but it does not
>>>>>> work in the backward direction (from newest to oldest item) because there
>>>>>> is no way to make the cursor advance backward. It only advances forward,
>>>>>> regardless of its own set direction.
>>>>>>
>>>>>> This example is very rough and arbitrary. But it appears to me that
>>>>>> the cursor.advance needs to obey the cursor's own direction setting. It's
>>>>>> almost like having a car that only moves forward (and can't u-turn) and in
>>>>>> order to move backward you have to reverse the road. That's bonkers.
>>>>>>
>>>>>> What's up with that?
>>>>>>
>>>>>> How naive or terribly misguided am I being?
>>>>>>
>>>>>> Thanks in advance.
>>>>>>
>>>>>> Marc
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>