RE: [IndexedDB] Current editor's draft

From: jorlow@google.com [mailto:jorlow@google.com] On Behalf Of Jeremy Orlow
Sent: Wednesday, July 14, 2010 12:10 AM

On Wed, Jul 14, 2010 at 3:52 AM, Pablo Castro <Pablo.Castro@microsoft.com> wrote:

From: public-webapps-request@w3.org [mailto:public-webapps-request@w3.org] On Behalf Of Andrei Popescu
Sent: Monday, July 12, 2010 5:23 AM

>> >> Dynamic transactions:
>> >> I see that most folks would like to see these going away. While I like the predictability and simplifications that we're able to make by using static scopes for transactions, I worry that we'll close the door for two scenarios: background tasks and query processors. Background tasks such as synchronization and post-processing of content would seem to be almost impossible with the static scope approach, mostly due to the granularity of the scope specification (whole stores). Are we okay with saying that you can't for example sync something in the background (e.g. in a worker) while your app is still working? Am I missing something that would enable this class of scenarios? Query processors are also tricky because you usually take the query specification in some form after the transaction started (especially if you want to execute multiple queries with later queries depending on the outcome of the previous ones). The background tasks issue in particular looks pretty painful to me if we don't have a way to achieve it without freezing the application while it happens.

>> Well, the application should never freeze in terms of the UI locking up, but in what you described I could see it taking a while for data to show up on the screen.  This is something that can be fixed by doing smaller updates on the background thread, sending a message to the background thread that it should abort for now, doing all database access on the background thread, etc.

This is an issue regardless, isn't it? Let's say you have a worker churning on the database somehow. The worker has no UI or user to wait for, so it'll run in a tight loop at full speed. If it splits the work in small transactions, in cases where it doesn't have to wait for something external there will still be a small gap between transactions. That could easily starve the UI thread that needs to find an opportunity to get in and do a quick thing against the database. As you say the difference between freezing and locking up at this point is not that critical, as the end user in the end is just waiting.

>> One point that I never saw made in the thread that I think is really important is that dynamic transactions can make concurrency worse in some cases.  For example, with dynamic transactions you can get into live-lock situations.  Also, using Pablo's example, you could easily get into a situation where the long running transaction on the worker keeps hitting serialization issues and thus it's never able to make progress.

While it could certainly happen, I don't remember seeing something like a live-lock in a long, long time. Deadlocks are common, but a simple timeout will kill one of the transactions and let the other make progress. A bit violent, but always effective. 

>> I do see that there are use cases where having dynamic transactions would be much nicer, but the amount of non-determinism they add (including to performance) has me pretty worried.  I pretty firmly believe we should look into adding them in v2 and remove them for now.  If we do leave them in, it should definitely be in its own method to make it quite clear that the semantics are more complex.
 
Let's explore a bit more and see where we land. I'm not pushing for dynamic transactions themselves, but more for the scenarios they enable (background processing and such). If we find other ways of doing that, then all the better. Having different entry points is reasonable.

>> >> Nested transactions:
>> >> Not sure why we're considering this an advanced scenario. To be clear about what the feature means to me: make it legal to start a transaction when one is already in progress, and the nested one is effectively a no-op, just refcounts the transaction, so you need equal amounts of commit()'s, implicit or explicit, and an abort() cancels all nested transactions. The purpose of this is to allow composition, where a piece of code that needs a transaction can start one locally, independently of whether the caller had already one going.

>> I believe it's actually a bit more tricky than what you said.  For example, if we only support static transactions, will we require that any nested transaction only request a subset of the locks the outer one took?  What if we try to start a dynamic transaction inside of a static one?  Etc.  But I agree it's not _that_ tricky and I'm also not convinced it's an "advanced" feature.

>> I'd suggest we take it out for now and look at re-adding it when the basics of the async API are more solidified.  I hope we can get it into v1, but we have too much in the air right now as is.

Sounds good. Also look at my reply to Jonas just a minute ago. Maybe we can get the scenario in good shape by other means.

>> Other points:
>>
>> *_NO_DUPLICATES:
>> I'm still not convinced we need this in v1.  It will help performance in some cases, but it adds more API surface area than immediately meets the eye.  If we do decide to have it in v1, we need to resolve the issues Jonas brought up.  Ideally we would do this on the thread Jonas started ("[IndexedDB] .value of no-duplicate cursors").

Yep, given that we have to make some arbitrary choices (first/last/etc), this may be something we just don't do and give developers a bit more work to do the skipping of dups themselves. I wonder if there was a specific scenario that was really important that we wanted this for originally. 

>> Pre-loaded cursors + getAll:
>> I'm glad we've decided to take these out for the time being.

+1

-pablo

Received on Thursday, 15 July 2010 00:22:29 UTC