Re: [IndexedDB] Transaction Auto-Commit

I have been spending time on IDB lately and wanted to give feedback as to the transaction auto-commit interface:

I am trying to write a wrapper around IDB to match the interface of my server-side data store, which allows you to:

1. Request a read or write transaction asynchronously.
2. GET, MGET, EXISTS or SET against that transaction asynchronously.
3. COMMIT when done to release and commit the transaction or ABORT to release but not commit the transaction.
4. Have many concurrent read transactions.
5. Have one write transaction at a time (without blocking readers - MVCC).

As you can imagine, IDB does not support this, since it forces you to issue requests against an IDB transaction synchronously (from the viewpoint of the rest of the application). In other words, once you have obtained an IDB transaction, it is automatically released when your code returns control so there is no way to do something such as get a value from IDB, do something taking a millisecond or two such as reading from WebSQL and then writing the a value back to IDB, all within the same IDB transaction. You'd have to use multiple IDB transactions which would be fine if the user only had your application open in one tab, but not in multiple tabs.

To get around this, I thought one could use optimistic concurrency control to write a nonce to IDB whenever a write transaction is requested from my IDB wrapper, use separate IDB transactions, and when writing, generate a conflict error if the nonce has changed.

The problem is it's significantly slower to do each GET, MGET, EXISTS, or SET on a separate IDB transaction. I think it works out to an extra millisecond or two overhead. If you're doing 10 or 20 operations, however small, that's an extra 10-20ms wasted overhead.

So then I thought I would request an IDB transaction when a transaction is requested from my wrapper, and then check the active flag when it's needed, and if active is set to false then re-request the transaction. The trouble is that the active flag does not appear to be exposed to JS as far as I can see.

Then I tried using a try/catch whenever an object store is requested from an IDB transaction so as to reset the IDB transaction if it's expired. Chrome returns "NOT_ALLOWED_ERR" instead of "...INACTIVE…" as it should. But I also found that the UA sometimes updates the active flag when my code has not returned control so there's a race condition somewhere in there I think, which may make this trick impossible. It works fine if I schedule a delay between operations of 10ms or more. When it gets down to 1ms though, it starts failing every now and then.

I tried the same thing using transaction.oncomplete to set my own active flag, but this did not work either.

Throughout, IDB in Chrome performs at least an order of magnitude slower than the same code running against an in-house mvcc database on the same machine. Firefox is significantly slower than Chrome. Would anyone know what the LevelDB benchmark would look like if through IDB on Chrome?

>> "Note that reads are also blocked if the long-running transaction is a READ_WRITE transaction."

Is it acceptable for a writer to block readers? What if one tab is downloading a gigabyte of user data (using a workload-configurable Merkle tree scheme), and another tab for the same application needs to show data?

On 25 Jul 2011, at 8:38 PM, Jonas Sicking wrote:

On Mon, Jul 25, 2011 at 6:28 AM, Joran Greef <joran@ronomon.com> wrote:
> Regarding transactions in the IndexedDB specification (3.1.7 Transaction):
> 
>>> "Once a transaction no longer can become active, and if the transaction hasn't been aborted, the implementation must automatically attempt to commit it. This usually happens after all requests placed against the transaction has been executed and their returned results handled, but no new requests has been placed against the transaction."
> 
> What does "no longer can become active" mean?

Well.. generally it's exactly the text you are quoting. "after all
requests placed against the transaction has been executed and their
returned results handled, but no new requests has been placed against
the transaction".

If you want the full exact definition, look for all the places that
references the "active" flag for transactions.

>>> "Authors can still cause transactions to run for a long time, however this is generally not a usage pattern which is recommended and can lead to bad user experience in some implementations."
> 
> How exactly can an author still cause a transaction to span several asynchronous events?

All transactions span all the asynchronously firing events that are
fired against the requests placed against the transaction. So as long
as you're scheduling requests against the transaction it'll keep
running. So if you schedule a million requests against a transaction,
it'll take a while for it to finish. That's all the above quote says.

> For example, start a transaction, read a value, use that value to do something asynchronous outside of IDB (perhaps for a millisecond or two or up to a second), and then write the result of that back to the transaction?

That you can't do. You should do this as two separate transactions.
This is intentional since we want transactions to be short lived.

> If it is indeed possible for an author to prolong a transaction, does that mean the UA is implementing a delay to give transactions with asynchronous dependencies the chance to add requests?

It's not possible. Not other than scheduling requests against a transaction.

> Surely an explicit commit in this case would be preferable for performance reasons (with a UA timeout protecting against developer forgetfulness)? Then again, if a developer forgot an explicit commit, it would only block writes for his particular application.

Note that reads are also blocked if the long-running transaction is a
READ_WRITE transaction.

/ Jonas

Received on Tuesday, 2 August 2011 12:39:26 UTC