Re: [IndexedDB] Interaction between transactions and objects that allow multiple operations

Hey folks,

I'm working with Shawn on the Firefox implementation. Here's our idea
as of now, would you all please comment about things you like or
dislike? Hopefully this follows the gist of the comments shared
already.

  interface IndexedDatabaseRequest {
    IDBRequest open(in DOMString name,
                    in DOMString description,
                    in optional boolean modifyDatabase);
  };

  interface IDBDatabaseRequest : IDBDatabase {
    IDBRequest openTransaction(in optional DOMStringList storeNames,
                               in optional unsigned long timeout);
  };

  interface IDBTransactionRequest : IDBTransaction {
    IDBRequest abort();

    IDBRequest commit();

    IDBRequest createObjectStore(in DOMString name,
                                 in DOMString keyPath,
                                 in optional boolean autoIncrement);

    IDBRequest openObjectStore(in DOMString name,
                               in optional unsigned short mode);

    IDBRequest createIndex(in DOMString name,
                           in DOMString storeName,
                           in DOMString keyPath,
                           in optional boolean unique);

    IDBRequest openIndex(in DOMString name);

    IDBRequest removeObjectStore(in DOMString storeName);

    IDBRequest removeIndex(in DOMString indexName);

    IDBRequest setVersion(in DOMString version);
  };

We've made some additional changes to IDBRequest and actually
specified the success/error events, but they're not really relevant
here and I'll post about them later.

Note that if we go this route then the mode parameter of the
openObjectStore method becomes nearly meaningless since transactions
are currently supposed to exclusively lock the stores and indexes they
access.

As Shawn has said previously we're looking to make the async API as
close to the synchronous API as possible (never allowing anything to
block, of course) to avoid tons of nested functions that will make web
developers' lives harder. To that end we're wondering how much of the
IDBTransactionRequest interface (as written above) can be made
synchronous-like. For instance, we could simply wait to fire the
success callback until we've read some metadata back from the database
file. Then we could make openObjectStore simply return an
IDBObjectStoreRequest instead of another IDBRequest as we would
already know if the object store exists. It would be even simpler if
the caller used the storeNames parameter in the openTransaction call
since we wouldn't call the success callback unless those stores
existed. We could do similar cheats for the other methods.

What do you guys think?

-Ben

On Thu, May 6, 2010 at 3:17 AM, Jeremy Orlow <jorlow@chromium.org> wrote:
> On Thu, May 6, 2010 at 9:14 AM, Nikunj Mehta <nikunj@o-micron.com> wrote:
>>
>> On May 4, 2010, at 7:17 PM, Pablo Castro wrote:
>>
>> > The interaction between transactions and objects that allow multiple
>> > operations is giving us trouble. I need to elaborate a little to explain the
>> > problem.
>> >
>> > You can perform operations in IndexedDB with or without an explicitly
>> > started transaction. When no transaction is present, you get an implicit one
>> > that is there for the duration of the operation and is committed and the end
>> > (or rolled-back if an error occurs).
>>
>> To provide context to those who might be missing it, an explicit
>> transaction is active in an IndexedDB Database as long as it has not been
>> explicitly committed or aborted. An implicit transaction's life time is
>> under the control of the implementation and spans no more than the operation
>> requested.
>>
>> >
>> > There are a number of operations in IndexedDB that are a single step.
>> > For example, store.put() occurs either entirely in the current transaction
>> > (if the user started one explicitly) or in an implicit transaction if there
>> > isn't one active at the time the operation starts. The interaction between
>> > the operation and transactions is straightforward in this case.
>> >
>> > On the other hand, other operations in IndexedDB return an object that
>> > then allows multiple operations on it. For example, when you open a cursor
>> > over a store, you can then move to the next row, update a row, delete a row,
>> > etc. The question is, what is the interaction between these operations and
>> > transactions? Are all interactions with a given cursor supposed to happen
>> > within the transaction that was active (implicit or explicit) when the
>> > cursor was opened? Or should each interaction happen in its own transaction
>> > (unless there is a long-lived active transaction, of course)?
>>
>> The transactional context of a series of operations is the transaction
>> that was created in the database. Each and every operation from that point
>> on till one of the following happens is performed in that transaction:
>>
>> 1. The transaction is committed
>> 2. The transaction is aborted
>> 3. The database object goes out of scope.
>>
>> >
>> > We have a few options:
>> > a) make multi-step objects bound to the transaction that was present
>> > when the object is first created (or an implicit one if none was present).
>> > This requires new APIs to mark cursors and such as "done" so implicit
>> > transactions can commit/abort, and has issues around use of the database
>> > object while a cursor with an implicit transaction is open.
>> >
>> > b) make each interaction happen in its own transaction (explicit or
>> > implicit). This is quite unusual and means you'll get inconsistent reads
>> > from row to row while scanning unless you wrap cursor/index scans on
>> > transactions. It also probably poses interesting implementation challenges
>> > depending on what you're using as your storage engine.
>> >
>> > c) require an explicit transaction always, along the lines Nikunj's
>> > original proposal had it. We would move most methods from database to
>> > transaction (except a few properties such as version and such, which it may
>> > still be ok to handle implicitly from the transactions perspective). This
>> > eliminates this whole problem altogether at the cost of an extra step
>> > required always.
>> >
>> > We would prefer to go with option c) and always require explicit
>> > transactions. Thoughts?
>>
>> The current specification allows using an explicit transaction and once
>> initiated, the explicitly created transaction is applicable for its life
>> time as described above. IOW a) is the same as c)
>>
>> If you intend to perform multiple steps, then an explicit transaction
>> appears to be in order unless the application can tolerate inconsistent
>> results. Therefore b) is not a good idea for multi-step operations. In
>> addition, it is not a good idea to create/commit explicit transactions for
>> each operation.
>
> Nikunj, I don't really understand your responses.  I'm pretty sure Pablo's
> whole question revolved around implicit transactions and whether we should
> get rid of them (option c), make cursors only available within transactions,
> etc.  It looks like most of your response was clarifying how explicit
> transactions should work?
>>
>> There has been some discussion for nested transactions and the original
>> proposal had support for those, but recall that some implementors were not
>> convinced of the cost/benefit tradeoff on that one.
>
> One of the strongest pieces of feedback I've gotten from Google developers
> who have worked with gears/SQLDatabase is in support of open nested
> transactions.  This is because it allows multiple layers of abstraction to
> all use the database without needing a lot of code to coordinate between
> them.  Implementing them should be fairly straight forward: just keep a
> count of the nesting level and only commit the whole transaction if that
> count becomes 0.  Aborts of course require a little extra work, but I don't
> believe it'll be a major burden on implementors.
>
> On Thu, May 6, 2010 at 9:18 AM, Nikunj Mehta <nikunj@o-micron.com> wrote:
>>
>> On May 5, 2010, at 1:56 PM, Shawn Wilsher wrote:
>>
>> > On 5/5/2010 1:09 PM, Jeremy Orlow wrote:
>> >> I'd also worry that if creating the transaction were completely
>> >> transparent
>> >> to the user that they might not think to close it either.  (I'm mainly
>> >> thinking about copy-and-paste coders here.)
>> > I should have been more clear.  That statement goes along with the
>> > suggestion to make everything work off of a transaction - object stores,
>> > indexes, cursors, etc.  They'd have to know about the transaction because
>> > they'd have to use it.
>>
>> I feel that auto transaction creation is syntactic sugar and should be
>> left to libraries. On the other hand, I'd be worried if we were developing
>> for complex multi-tab applications and not explicitly managing transactions.
>
> I agree with both points: implicit transactions really are just syntactic
> sugar and can be handled by libraries (which we expect will also do things
> like joins for users).
> And concurrency is a problem.  Web developers assume run to completion
> semantics.  To me, that means that at very least we should make the implicit
> transaction last the length of the javascript execution (i.e. the same
> semantics as the storage mutex...which exists because many browsers now have
> multiple event loops running concurrently).  The problem with that is that
> the final result may not be serializable and thus need to be aborted.
> So the two options I see are to have implicit transactions essentially lock
> the entire database (so we're guaranteed the transaction can be committed)
> and have it last the length of the javascript execution (so essentially
> we're creating another storage mutex).  Or we can get rid of implicit
> transactions and make users think about them.
> Given that we all implement LocalStorage and it's good for the super simple
> use cases, I guess I don't see why we would put so much effort into hiding
> the concept of a transaction from IndexedDB users.
> J

Received on Thursday, 6 May 2010 20:40:19 UTC