Re: [IndexedDB] Transaction Auto-Commit

On Tue, Aug 2, 2011 at 5:38 AM, Joran Greef <joran@ronomon.com> wrote:
> I have been spending time on IDB lately and wanted to give feedback as to the transaction auto-commit interface:
>
> I am trying to write a wrapper around IDB to match the interface of my server-side data store, which allows you to:
>
> 1. Request a read or write transaction asynchronously.
> 2. GET, MGET, EXISTS or SET against that transaction asynchronously.
> 3. COMMIT when done to release and commit the transaction or ABORT to release but not commit the transaction.
> 4. Have many concurrent read transactions.
> 5. Have one write transaction at a time (without blocking readers - MVCC).
>
> As you can imagine, IDB does not support this, since it forces you to issue requests against an IDB transaction synchronously (from the viewpoint of the rest of the application). In other words, once you have obtained an IDB transaction, it is automatically released when your code returns control so there is no way to do something such as get a value from IDB, do something taking a millisecond or two such as reading from WebSQL and then writing the a value back to IDB, all within the same IDB transaction. You'd have to use multiple IDB transactions which would be fine if the user only had your application open in one tab, but not in multiple tabs.
>
> To get around this, I thought one could use optimistic concurrency control to write a nonce to IDB whenever a write transaction is requested from my IDB wrapper, use separate IDB transactions, and when writing, generate a conflict error if the nonce has changed.
>
> The problem is it's significantly slower to do each GET, MGET, EXISTS, or SET on a separate IDB transaction. I think it works out to an extra millisecond or two overhead. If you're doing 10 or 20 operations, however small, that's an extra 10-20ms wasted overhead.

Yeah, you can implement your own concurrency control by writing a
token value into a objectStore while you are retriving data. But it
doesn't surprise me that performance is affected.

> So then I thought I would request an IDB transaction when a transaction is requested from my wrapper, and then check the active flag when it's needed, and if active is set to false then re-request the transaction. The trouble is that the active flag does not appear to be exposed to JS as far as I can see.

I'm not sure I understand how it would help to be able to read the
active flag. Do you not know if you're inside a database callback or
not? Technically you could expose the active flag yourself by setting
a property on the transaction any time you receive a onerror/onsuccess
callback, and clear it at the end of the callback.

But if it helps, I see no reason we couldn't expose the active flag.
But I'd like to understand how it helps first.

> Then I tried using a try/catch whenever an object store is requested from an IDB transaction so as to reset the IDB transaction if it's expired. Chrome returns "NOT_ALLOWED_ERR" instead of "...INACTIVE…" as it should. But I also found that the UA sometimes updates the active flag when my code has not returned control so there's a race condition somewhere in there I think, which may make this trick impossible. It works fine if I schedule a delay between operations of 10ms or more. When it gets down to 1ms though, it starts failing every now and then.
>
> I tried the same thing using transaction.oncomplete to set my own active flag, but this did not work either.

Note that a transaction isn't active from when it's created to when
it's complete. It's only active during onsuccess/onerror callbacks,
and between the point when it's created and when the program returns
to the event loop.

> Throughout, IDB in Chrome performs at least an order of magnitude slower than the same code running against an in-house mvcc database on the same machine. Firefox is significantly slower than Chrome. Would anyone know what the LevelDB benchmark would look like if through IDB on Chrome?

Firefox is currently backed by SQLite so it doesn't surprise me that
performance is bad. The database part is far from optimized. Our plan
is to transition to LevelDB as soon as we find someone who can do the
work.

>>> "Note that reads are also blocked if the long-running transaction is a READ_WRITE transaction."
>
> Is it acceptable for a writer to block readers? What if one tab is downloading a gigabyte of user data (using a workload-configurable Merkle tree scheme), and another tab for the same application needs to show data?

This is exactly why transactions are auto-committing. We don't want
someone to start a transaction, download a gigabyte of data, write it
to the database, and only after commit the transaction. The
auto-committing behavior forces you to download the data first, only
then can you start a transaction to insert that data into the
database.

IndexedDB allows MVCC in that it allows writers to start while there
are still reading transactions running. Firefox currently isn't
implementing this though since our underlying storage engine doesn't
permit it.

IndexedDB does however not allow readers to start once a writing
transaction has started. I thought that that was common behavior even
for MVCC databases. Is that not the case? Is it more common that
readers can start whenever and always just see the data that was
committed by the time the reading transaction started?

Also note that IndexedDB allows any transaction to overlap any other
transaction as long as they don't overlap in which objectStores they
use.

/ Jonas

Received on Wednesday, 3 August 2011 17:34:25 UTC