Re: [WebSQL] Any future plans, or has IndexedDB replaced WebSQL?

> ------------------------------------------------------------------------
>
> 	Joran Greef <mailto:joran@ronomon.com>
> April 4, 2011 April 4, 201110:18 AM
>
>
> On 04 Apr 2011, at 6:10 PM, Mikeal Rogers wrote:
>
>> it's not very hard to write the abstraction you're talking about on top of IndexedDB, and until you do it i'm going to have a hard time taking you seriously because it's clearly doable.
>
>
> You assume I have not written the abstraction I am talking about on top of IndexedDB?
>
>> the constructs in IndexedDB are pretty low level but sufficient if you know how to implement databases. performance is definitely an issue, but making these constructs faster would be much easier than trying to tweak an off the shelf SQL implementation to your use case.
>
>
> How exactly would you make a schema-enforcing interface faster than a stateless interface?
>
> How would you implement application-managed indices on top of IndexedDB without being slower than SQLite?
this assumes your indexes can be describe well in sqlite and that you 
want them generated at write time. one of the performance optimizations 
couchdb makes is to generate secondary indexes at read time which is 
much more efficient as you can batch the processing and do a single bulk 
transaction which limits IO. if you need, exactly, the indexes sqlite 
provides then i'm sure they will be well optimized in sqlite, but for 
everyone else.....
> How would you implement set operations on indices in IndexedDB without being slower or less memory efficient than SQLite?
you can do pretty much anything if you write a good abstraction and 
leverage transactions. if it's too slow complain to the vendor and it'll 
get improved, that's how we've been improving performance in the browser 
for many, many years and it seems to be working.
> How would you create an index on an existing object store in IndexedDB containing more than 50,000 objects on an iPad, without incurring any object deserialization/serialization overhead, without being an order of magnitude slower than SQLite, and without bringing the iPad to its knees? If you can do it with even one IndexedDB implementation out there then kudos and hats off to you. :)
the biggest bottleneck here in the current implementation would be the 
transaction overhead on a database this size, which is because of 
performance problems in sqlite which is underlying the implementation. 
sqlite can't fix this, it's currently the problem. the object 
serialization is not a huge performance issue, performance issues in 
databases are almost always do to IO or transaction locks.
> I understand your point of view. I once thought the same. You would think that IndexedDB would be more than satisfactory for these things. The question is whether IndexedDB provides adequate and performant database primitives, to the same degree as SQLite (and of course SQL is merely an interface to database storage primitives, I do not recalling saying otherwise).
sqlite doesn't expose it's primitives well, it exposes it's 
abstractions. berkeleydb exposes it's primitives well, and you could 
make the case that it would be a better thing to standardize on, 
although i wouldn't think that would be a great idea either.
> You can build IndexedDB on top of SQLite (as some browsers are indeed doing), but you cannot build SQLite on IndexedDB.
you should most definitely be able build sqlite on top of IDB, there 
would be a performance penalty of course, which we can address, but you 
should be able to do it. if you can't then we need to extend the 
specification.
> ------------------------------------------------------------------------
>
> 	Keean Schupke <mailto:keean@fry-it.com>
> April 4, 2011 April 4, 20118:55 AM
>
>
> Yes, it already has well defined set operations. Solid is a matter of 
> testing by enough people (and if you wanted to try it and feed back 
> that would be a start). Fast should not be a problem, as the SQL 
> database does all the heavy lifting.
>
> In more detail, Codd's six primitive operators are project, restrict, 
> cross-product, union and difference. Relations are an extension of 
> Sets, so intersection and difference on compatible relations behave 
> like they would on sets.
>
> RelationalDB already implements the following 5 methods making it 
> relationally-complete. Meaning it can do anything you could possibly 
> want to do with relations using combinations of these 5 methods.
>     Relation.prototype.project = function(attributes) { // this 
> implements rename as well.
>     Relation.prototype.restrict = function(exp)
>     Relation.prototype.join = function(relation, exp) {
>     Relation.prototype.union = function() {};
>     Relation.prototype.difference = function() {};
>
> Of course some things can be made easier, so the following methods, 
> although they can be defined in terms of the above 5, will be provided 
> (in future implementations) to keep user code concise and 
> implementations thin and fast.
>
>     // derived methods
>     Relation.prototype.intersection = function() {};
>     Relation.prototype.thetajoin = function() {};
>     Relation.prototype.semijoin = function() {};
>     Relation.prototype.antijoin = function() {};
>     Relation.prototype.divide = function() {};
>     Relation.prototype.leftjoin = function() {};
>     Relation.prototype.rightjoin = function() {};
>     Relation.prototype.fulljoin = function() {};
>
>
> We also hope to provide the lattice operators "meet" and "join":
>
> http://en.wikipedia.org/wiki/Lattice_(order) 
> <http://en.wikipedia.org/wiki/Lattice_%28order%29>
>
> Just these two operators can replace all 5 of Codd's primitives 
> (including all set operations). With just these two you can do 
> anything that you can with _all_ the above. Meet is actually the same 
> as Codd's natural-join (unfortunately terminology in different 
> mathematical fields is not consistent here) and Join is a generalised 
> union operator.
>
> See:
>
> http://www.arxiv.com/pdf/cs/0501053v2
>
> To see how Meet and Join can be used to construct each of the above 
> operators.
>
>
> Cheers,
> Keean.
>
>
>
> ------------------------------------------------------------------------
>
> 	Joran Greef <mailto:joran@ronomon.com>
> April 4, 2011 April 4, 20118:36 AM
>
>
>
> Yes, if an implementation of RelationalDB arrives which is solid and 
> fast with support for set operations that would be great. The 
> important thing is that we have two competing APIs (and preferably a 
> strong API with a great track record).
> ------------------------------------------------------------------------
>
> 	Keean Schupke <mailto:keean@fry-it.com>
> April 4, 2011 April 4, 20118:26 AM
>
>
> This is ignoring the possibility that something like RelationalDB 
> could be used, where a well defined common subset of SQL can be used 
> (and I use well-defined in the formal sense). This would allow a 
> relatively thin wrapper on top of most SQL implementations and would 
> allow SQLite (or BDB) to be used as the backend.
>
> As a seasoned C++ programmer, I could even write a Firefox plugin 
> using XPCOM as a reference implementation using the same API as the 
> JavaScript RelationalDB implementation on my GitHub. Although I am not 
> keen on putting in the time to do this if nobody is interested.
>
> To me is seems this thread is going in circles. RelationalDB does not 
> have the standardisation problem that WebSQL has, but is still a 
> relatively thin API layer that can be implemented over the top of a 
> fast and well tested SQL implementation. It is based on sound theory 
> and research defining the abstraction layer, and has a relationally 
> complete API, so there should be no need to change the core API in the 
> development of a standard.
>
>
> Cheers,
> Keean.
>
>
>
> ------------------------------------------------------------------------
>
> 	Jonas Sicking <mailto:jonas@sicking.cc>
> April 4, 2011 April 4, 20117:39 AM
>
>
>
> Competition might be a great thing. But it doesn't address the issue
> in the least. It would still be the case that some developers would
> choose to use WebSQL, and browser makers would still have to support
> it, including support the SQL dialect it uses.
>
> Hence it would still be the case that we would be relying on the
> SQLite developers to maintain a stable SQL interpretation to keep a
> healthy and functional web.
>
> / Jonas
>