Re: IndexedDB >> Proposed API Change: cursor.advance BACKWARD when direction is "prev"

On Fri, May 23, 2014 at 6:24 PM, marc fawzi <marc.fawzi@gmail.com> wrote:

> Here is a jsfiddle showing how .advance behaves when the range is
> restricted by .only
>
> Create new e.g. 7 items with names like "marc" and tags like "w1 w3 w5 w2"
> (random selection of tags with some tags appearing across multiple records
> (per the attached image)
>
> Enter "w2" or "w5" in the box next to 'get by tag' and click 'get by tag'
>
> You'll see the first 2 matching items, with primary keys 7 and 6
>
> Click 'get by tag' again and you'll see the next 2 matching items, with
> primary keys 4 and 2
>
> Click 'get by tag' again and you'll see the next and last matching item,
> with primary key 1
>
> Notice the way I advance the cursor each time in order to re-continue the
> search from where I left off in the previous invocation is by using the
> number of items already found
>
> http://jsfiddle.net/marcfawzi/y5ELj/
>
>
Thanks for sharing this example.


> It's the correct behavior but it would be easier imo if we have .find()
> for what .continue() and .continue(key) does and use .continue() to mean
> .advance(1) and .continue(n) to mean .advance(n)
>
> But I could be totally wrong. Just a harmless feedback at this point.
>
>
It's a reasonable suggestion. Unfortunately, we already have multiple
shipping implementations and code in the wild depending on the API as
specified, and this would be a breaking change. It's useful feedback if we
add new cursor/iteration APIs in the future, though - the current choice of
"continue" and "advance" vs. e.g. "find" or "seek" is pretty arbitrary and
can be a source of confusion.

Thanks again for following up with examples to ensure we understood your
feedback!


> :)
>
>
> On Fri, May 23, 2014 at 1:07 PM, marc fawzi <marc.fawzi@gmail.com> wrote:
>
>> <<
>> Thanks for following up! At least two IDB implementers were worried that
>> you'd found some browser bugs we couldn't reproduce.
>> >>
>> Yup. I had to figure this stuff out as the API is very low level (which
>> is why it can also be used in very powerful ways and also potentially very
>> confusing for the uninitiated)
>>
>> <<Assuming the store has [1,2,3,4,5,6,7,8,9] and the cursor's range is
>> not restricted, if the cursor's key=7 and direction='prev' then I would
>> expect after advance(2) that key=5. If you're seeing key=2 can you post a
>> sample somewhere (e.g. jsfiddle.com?)>>
>>
>> In the case I have say 7 items [1,2,3,4,5,6,7] and the cursor's range is
>> restricted by IDBKeyRange.only(val, "prev") ... so if the matching (or in
>> range) items are at 7, 6, 4, 2, 1 then I can obtain them individually or in
>> contiguous ranges by advancing the cursor on each consecutive invocation of
>> my search routine, like so: on first invocation advance(1) from 7 to 6, on
>> second invocation advance(2) from 7 to 4, on third invocation advance(3)
>> from 7 to 2 and on fourth invocation advance(4) from 7 to 1. I could also
>> use advance to advance by 1 within each invocation until no matching items
>> are found but only up to 2 times an invocation (for a store with 700 or
>> 70000 items we can advance by 1 about 200 times per invocation, but that's
>> arbitrary)
>>
>>  I can definitely post a jsfiddle if you believe the above is not in
>> accordance with the spec.
>>
>> As to continue(n) or continue(any string), i would make that
>> .find(something)
>>
>>
>>
>> On Fri, May 23, 2014 at 10:41 AM, Joshua Bell <jsbell@google.com> wrote:
>>
>>> On Fri, May 23, 2014 at 9:40 AM, marc fawzi <marc.fawzi@gmail.com>wrote:
>>>
>>>> I thought .continue/advance was similar to the 'continue' statement in
>>>> a for loop in that everything below the statement will be ignored and the
>>>> loop would start again from the next index. So my console logging was
>>>> giving confusing results. I figured it out and it works fine now.
>>>>
>>>
>>> Thanks for following up! At least two IDB implementers were worried that
>>> you'd found some browser bugs we couldn't reproduce.
>>>
>>>
>>>>  For sanity's sake, I've resorted to adding a 'return'  in my code in
>>>> the .success callback after every .advance and .continue so the execution
>>>> flow is easier to follow. It's very confusing, from execution flow
>>>> perspective, for execution to continue past .continue/.advance while at
>>>> once looping asynchronously. I understand it's two different instances of
>>>> the .success callback but it was entirely not clear to me from reading the
>>>> docs on MDN (for example) that .advance / .continue are async.
>>>>
>>>
>>> Long term, we expect JS to evolve better ways of expressing async calls
>>> and using async results. Promises are a first step, and hopefully the
>>> language also grows some syntax for them. IDB should jump on that train
>>> somehow.
>>>
>>>
>>>> Also, the description of .advance in browser vendor's documentation,
>>>> e.g. on MDN, says "Advance the cursor position forward by two places" for
>>>> cursor.advance(2) but what they should really say is "advance the cursor
>>>> position forward by two results." For example, let's say cursor first
>>>> landed on an item with primary key = 7, and you issue the statement
>>>> cursor.advance(2), I would expect it to go to the item with primary key 5
>>>> (for cursor direction = "prev") but instead it goes to the item with
>>>> primary key 2 because that's the 2nd match for the range argument from the
>>>> cursor's current position
>>>>
>>>
>>> What "range argument" are you referring to?
>>>
>>> Assuming the store has [1,2,3,4,5,6,7,8,9] and the cursor's range is not
>>> restricted, if the cursor's key=7 and direction='prev' then I would expect
>>> after advance(2) that key=5. If you're seeing key=2 can you post a sample
>>> somewhere (e.g. jsfiddle.com?)
>>>
>>>
>>>> , which means that .advance(n) would be far more clear semantically
>>>> speaking if it was simply done as .continue(n)  ... I guess if there is an
>>>> understanding that the cursor is always at a matching item and that it
>>>> could only continue/advance to the next/prev matching item, not literal
>>>> 'positions' in the table (i.e. sequentially through the list of all items)
>>>> then there would be no confusion but the very concept of a cursor is
>>>> foreign to most front end developers, and that's where the confusion comes
>>>> from for many.
>>>>
>>>> My inclination as a front end developer, so far removed from database
>>>> terminology, would be
>>>>
>>>> 1) to deprecate .advance in favor of .continue(n) and
>>>>
>>>
>>> continue(n) already has meaning - it jumps ahead to the key with value n
>>>
>>>
>>>>
>>>> 2) if it makes sense (you have to say why it may not) have
>>>> .continue()/.continue(n) cause the return of the execution flow similar to
>>>> 'continue' in a for loop.
>>>>
>>>
>>> The API can't change the language - you return from functions via return
>>> or throw. Further, there are reasons you may want to do further processing
>>> after calling continue() - e.g. there may be multiple cursors (e.g. in a
>>> join operation) or for better performance you can call continue() as early
>>> as possible so that the database can do its work while you're processing
>>> the previous result.
>>>
>>>
>>>
>>>>
>>>> What do you think?
>>>>
>>>>
>>>>
>>>> On Wed, May 21, 2014 at 10:42 AM, Joshua Bell <jsbell@google.com>wrote:
>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, May 21, 2014 at 7:32 AM, Arthur Barstow <art.barstow@gmail.com
>>>>> > wrote:
>>>>>
>>>>>> [ Bcc www-tag ; Marc - please use public-webapps for IDB discussions ]
>>>>>>
>>>>>> On 5/20/14 7:46 PM, marc fawzi wrote:
>>>>>>
>>>>>>> Hi everyone,
>>>>>>>
>>>>>>> I've been using IndexedDB for a week or so and I've noticed that
>>>>>>> cursor.advance(n) will always move n items forward regardless of cursor
>>>>>>> direction. In other words, when the cursor direction is set to "prev" as
>>>>>>> in: range = IDBKeyRange.only(someValue, "prev") and primary key is
>>>>>>> auto-incremented, the cursor, upon cursor.advance(n), will actually advance
>>>>>>> n items in the opposite direction to the cursor.continue() operation.
>>>>>>
>>>>>>
>>>>> That runs contrary to the spec. Both continue() and advance()
>>>>> reference the "steps for iterating a cursor" which picks up the direction
>>>>> from the cursor object; neither entry point alters the steps to affect the
>>>>> direction.
>>>>>
>>>>> When you say "you've noticed", are you observing a particular
>>>>> browser's implementation or are you interpreting the spec? I did a quick
>>>>> test and Chrome, Firefox, and IE all appear to behave as I expected when
>>>>> intermixing continue() and advance() calls with direction 'prev' - the
>>>>> cursor always moves in the same direction regardless of which call is used.
>>>>>
>>>>> Can you share sample code that demonstrates the problem, and indicate
>>>>> which browser(s) you've tested?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>  This is not only an issue of "broken symmetry" but it presents an
>>>>>>> obstacle to doing things like: keeping a record of the primaryKey of the
>>>>>>> last found item (after calling cursor.continue for say 200 times) and, long
>>>>>>> after the transaction has ended, call our search function again and, upon
>>>>>>> finding the same item it found first last time, advance the cursor to the
>>>>>>> previously recorded primary key and call cursor.continue 200 times, from
>>>>>>> that offset, and repeat whenever you need to fetch the next 200 matching
>>>>>>> items. Such algorithm works in the forward direction (from oldest to newest
>>>>>>> item) because cursor.advance(n) can be used to position the cursor forward
>>>>>>> at the previously recorded primary key (of last found item) but it does not
>>>>>>> work in the backward direction (from newest to oldest item) because there
>>>>>>> is no way to make the cursor advance backward. It only advances forward,
>>>>>>> regardless of its own set direction.
>>>>>>>
>>>>>>> This example is very rough and arbitrary. But it appears to me that
>>>>>>> the cursor.advance needs to obey the cursor's own direction setting. It's
>>>>>>> almost like having a car that only moves forward (and can't u-turn) and in
>>>>>>> order to move backward you have to reverse the road. That's bonkers.
>>>>>>>
>>>>>>> What's up with that?
>>>>>>>
>>>>>>> How naive or terribly misguided am I being?
>>>>>>>
>>>>>>> Thanks in advance.
>>>>>>>
>>>>>>> Marc
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>