Re: What do we mean by "parking" Web Database? [Was: Re: TPAC report day 2]

On Nov 9, 2009, at 3:12 AM, Robert O'Callahan wrote:

> On Mon, Nov 9, 2009 at 9:58 PM, Maciej Stachowiak <mjs@apple.com>  
> wrote:
> On Nov 8, 2009, at 11:12 PM, Jonas Sicking wrote:
>
> Indeed. I still personally wouldn't call it multiple independent
> implementations though.
>
> Would you call multiple implementations that use the standard C  
> library independent? Obviously there's a judgment call to be made  
> here.
>
> Yes. Multiple implementations passing query strings (more or less)  
> verbatim to SQLite for parsing and interpretation would not pass  
> that judgement call ... IMHO, but wouldn't you agree?

If that were the extent of the implementation, I might agree. However,  
that doesn't accurately characterize at least WebKit's WebDatabase  
implementation. WebKit has around 15k lines of code which implement  
asynchronicity, do checking and rewrites on the queries, export DOM  
APIs, manage transactions, expose result sets, etc. It's true that  
SQLite has a fair bit more code. But then again, the image decoding  
library we use has more code than our <img> element implementation.

I think it's worth specifying and testing the layers above the query  
language even if, for whatever reason, the query language does not get  
specified.

>
>
> * If we do specify a specific SQL dialect, that leaves us having to
> implement it. It's very unlikely that the dialect would be compatible
> with SQLite (especially given that SQLite uses a fairly unusual SQL
> dialect with regards to datatypes) which likely leaves us implementing
> our own SQL engine.
>
> The SQL dialect could, of course, be specified in a way that it  
> could work on top of SQLite with some reasonable filtering and  
> preprocessing steps. I don't see a reason to expect otherwise, and  
> in particular that it would be so different that it would require  
> writing a new SQL engine.
>
> I think the problem is rather coming up with a SQL definition that  
> can be implemented by anything other than SQLite (or from scratch,  
> of course). One weird thing about SQLite is that column types aren't  
> enforced. So either the spec requires something like SQLite's "type  
> affinity" (in which case it doesn't fit well with most other SQL  
> implementations, and precludes common performance optimizations), or  
> it requires strict type checking (which perhaps you could implement  
> in SQLite by adding CHECK constraints?). But the latter course is  
> probably incompatible with deployed content, so contrary to Jonas I  
> expect the spec would be implementable *only* on top of SQLite (or  
> from scratch, of course), or perhaps some unnatural embedding into  
> other engines where all values are text or variants. Experience with  
> alternative implementations would be important.

Indeed, it is a challenge to find the right design point. Right now it  
looks like no one is going to take up this challenge.

>
> Does SimpleDB give performance guarantees? Does any Web platform  
> spec give performance guarantees? Even DOM Core methods have  
> different Big-O complexity in Gecko and WebKit. So this doesn't seem  
> very persuasive.
>
> You're right that precedent is weak here, but I think database APIs  
> aspire to work with larger data sets than the DOM does, so big-O  
> guarantees are more important.

Is SimpleDB going to give big-O performance guarantees? (The current  
draft does not appear to.)

>
> So basically, at this point the spec for the SQL dialect is blocked  
> on Mozilla, since no one is interested in doing that spec work if it  
> won't actually lead to wider implementation.
>
> That doesn't seem logical to me. The spec work has to be done if Web  
> Database is to advance through the spec process. You want it to  
> advance even if Mozilla doesn't agree to implement it. Therefore,  
> the spec work has to be done. (And if there was a SQL dialect spec  
> that's simple and elegant and amenable to multiple implementations  
> and compatible with your deployed content, that would certainly be a  
> large point in favour of Web Database.)

At the Web Apps WG face-to-face meeting at TPAC, all parties agreed  
(in the room at least) to let the spec continue without fully  
specifying the SQL dialect. The reason is that all parties who  
currently have or are in the process of developing implementations did  
not appear to need it, and the parties that would be blocked (Mozilla,  
Microsoft) said their decision would not be swayed by having a spec,  
and would not implement regardless. Thus, it did not seem there would  
be a practical benefit to specifying the SQL dialect. Thus, those  
present said they were satisfied to specify that SQLite v3 is the  
dialect.

Note: I would try to find Apple resources to help write a SQL dialect  
spec if anyone says it will materially help them to have such a spec  
(and the level of interest doesn't reach the threshold where Hixie  
wants to write it himself). I don't think I could get resources if  
it's just a busywork exercise.

>
> I don't think SimpleDB is useless for mobile platforms. You  
> certainly *could* use it. But it does have three significant  
> downsides compared to the SQL database: (1) it's very different from  
> what developers have already (happily) been using on mobile; (2) the  
> target design point is that it's primarily expected to be used  
> through JavaScript libraries layered on top, and not directly (so  
> you have to ship more code over the wire); and (3) for more complex  
> queries, more of the work has to be done in JavaScript instead of in  
> the database engine (so performance will likely be poor on low-power  
> CPUs).
>
> Do you have easy access to knowledge about the sort of complex  
> queries these mobile apps do? That would be very useful.

I don't have easy access to that knowledge, but I can attempt to  
inquire.

>
> We already ship SQLite and implementing Web Database using SQLite  
> would definitely be the path of least resistance for us. We're just  
> concerned it might not be the right thing for the Web.

My thoughts on this are as follows:

1) It seems that a database layer that's less "opinionated" than SQL  
is a better target for library authors; by having less conceptual  
baggage and policy, it makes it easier for different conceptual models  
(such as object oriented stores) to be layered on top. Thus, library  
developers are likely to want a lower-level solution.

2) It seems that a database layer with a good amount of high-level  
concepts (including some kind of query language) is likely to be  
easier to code against directly for many use cases. Thus, application  
programmers, particularly in environments where extra abstraction  
layers are particularly costly

3) Some mobile web developers have existing investment in SQL in  
particular, and do not appear to have had problems with it as a model.  
It would be a shame to abandon them, as in many ways they have been  
better pioneers of offline Web apps than mainline desktop-focused Web  
developers.


It seems plausible to me that SQL is not the best solution for all  
storage use cases. But it seems like a pretty aggressive position to  
say that, as a result, it should be out of the Web platform (and not  
just augmented by other facilities). It seems like that would  
underserve other use cases


Regards,
Maciej