Re: Replacing WebSQL with a Relational Data Model.

Hi,

> So I'm not saying that IndexedDB in its current state is enough to
> build what you want and get good performance. I'm saying that I think
> a prototype implementation on top of IndexedDB would be very
> educational.

Thinking about this, it would be possible to implement on top of IndexedDB,
but as I have said, to produce a full implementation with a query optimiser
would be a lot of work.

We could however produce two backends for the API, a _very_simple_ one on
top of IndexedDB, and one that sits on top of WebSQL. This would provide a
useful performance comparison. As you say it could be educational to have
the implementation on top of IndexedDB.


Cheers,
Keean.


On 26 October 2010 10:03, Keean Schupke <keean@fry-it.com> wrote:

> Hi,
>
> I will comment inline:
>
> On 26 October 2010 09:42, Jonas Sicking <jonas@sicking.cc> wrote:
>
>> On Tue, Oct 26, 2010 at 12:02 AM, Keean Schupke <keean@fry-it.com> wrote:
>> > Hi,
>> > It will be a lot faster with SQLite as the backend. Mobile apps depend
>> on
>> > access to the SQLite engine, and although it _could_ be implemented on
>> top
>> > of IndexedDB, there is no way its going to be fast enough... And the
>> thought
>> > of writing a decent query optimiser is a bit daunting.
>> > The benefit of making it a standard, is that browser implementers can do
>> > what they want, implement in JavaScript on top of IndexedDB, or pass
>> through
>> > to a proper database.
>> > BTree's are old hat in the database space. Modern storage engines are
>> using
>> > Fractal-Trees for a 50-80 times speedup in write performance. The
>> browser
>> > should not be trying to compete with experienced database coders.
>> > The relational storage model I am proposing will return a database table
>> as
>> > a list of objects, where each object represents a row, and the object
>> has a
>> > set of named properties representing each column.
>> > Part of the power of the Relational Data Model is that it abstracts data
>> > into columns and tables, and this is precisely what we want.
>>
>> I think you are misunderstanding what I'm proposing.
>>
>
> This is a definite possibility... My intelligence is borrowed from other
> people, my stupidity is my own...
>
>
>>
>> The problem is two-fold. I don't think we want to create a entirely
>> new database backend. We already have two different APIs (three if you
>> count localStorage), and none of them have full buy-in from all major
>> browser vendors. Adding a third one, and on top of that one that is
>> more complex than all other ones that have been suggested, I don't
>> think would be successful.
>>
>
> Backend? Frontend? Are you suggesting that IndexedDB could be implemented
> on top of an SQL database?
>
>
>>
>> However I do think that what you are proposing could be done as
>> additional API for the IndexedDB spec. However since we are currently
>> trying to finish up the currently discussed feature set, I don't think
>> we want to take on what you are proposing *at this time*. I do however
>> think that it's something we should look into for the next version of
>> IndexedDB.
>>
>
> Which version is not important. I guess what I am looking for is a
> willingness from the implementers to provide a relational API, backed by a
> fast database (like SQLite).
>
>
>>
>> In the *meantime*, what you could do is build *prototypes* in
>> javascript on top of IndexedDB. This would have several benefits:
>>
>> * It would be easier for everyone to understand exactly what is being
>> proposed as we could look at your prototype implementation.
>>
>
> Totally agree.
>
>
>> * It would give web authors something to experiment with so that they
>> can say if they like it or not. Or if they would like something
>> changed.
>>
>
> Ditto.
>
>
>> * It would allow us to profile to see which parts are slow and so that
>> we can prioritize those areas first for specification in IndexedDB v2
>> and native implementation in browsers.
>>
>
> Here is where there is some miscommunication (likely to be mine). What I am
> proposing is a relational API, that is not an implementation of the RDBMS
> itself. Although you could directly implement a relational algebra in
> JavaScript, this would be a lot of work.
>
> The way to view this is to think of a reverse-parser. You might construct a
> query something like:
>
> q1 = rdb.table(myTable);
> q2 = q1.restrict(rdb.eq(function(row) {row[myColumn] == myValue}));
> result_table = rdb.query(q2);
>
> This would construct an abstract syntax tree. So q1 is a tree with a single
> node representing the table "myTable". q2 is a tree with a root that is a
> "restrict" node representing the equality (itself an expression tree) and a
> child (the original q1 node).
>
> When we execute this by passing it to "rdb.query" the syntax tree is
> "rendered" to a string in this case we would produce:
>
> "select * from myTable where myColumn = 'myValue';"
>
> This is then passed through to the RDBMS to execute, and the table returned
> in marshalled into the correct format.
>
> So to implement this on top of WebSQL would be a the plan for a prototype.
> To implement a complete relational database engine on top of IndexedDB would
> be hard. To get good performance would be even harder.
>
>
>>
>> So I'm not saying that IndexedDB in its current state is enough to
>> build what you want and get good performance. I'm saying that I think
>> a prototype implementation on top of IndexedDB would be very
>> educational.
>>
>
> Implementing the RDBMS would be hard. Implementing the API on-top of an
> external RDBMS (SQLite) would be the way to go. Could this still be part of
> IndexedDB?
>
>
>>
>> Of course, if you disagree and think writing a separate database
>> interface is the better way to go, then don't let me stop you. You are
>> totally free to write a specification and I'd imagine W3C would be
>> happy to publish it. I'd think it would fit under our current charter
>> and so very little red tape is needed.
>>
>
> I would prefer to fit in with what everyone is already doing. There would
> be no point in starting a new standard if none of the browser implementers
> are interested in this. I would also prefer to have the cooperation of an
> experiences W3 standards editor if this is necessary.
>
>
> Cheers,
> Keean.
>
>
>