Re: Web Storage & SQL

On Apr 10, 2009, at 3:13 PM, Ian Hickson wrote:

> On Fri, 10 Apr 2009, Nikunj Mehta wrote:
>>
>> Can someone state the various requirements for Web Storage? I did not
>> find them enunciated anywhere.
>
> There's only one requirement that I know of:
>
> * Allow Web sites to store structured data on the client.
>
> There are many use cases, e.g. Google is interested in this to  
> enable its
> applications to be taken offline. We recently released offline GMail  
> using
> this SQL backend; one could easily imagine other applications like
> Calendar, Reader, Docs&Spreadsheets, etc, supporting offline mode. A  
> while
> back we released a demo of Reader using Gears' SQL database.

Last time I tried this trick I was asked to come back with more  
precise use cases [1]. Then I put together more detailed use cases  
[2], and even those were not considered to be written precisely  
enough. So it looks like the bar for what constitutes a use case or  
requirement seems to be quite high.

> But we would rather use a standard API than rely on Gears.

I think if we are serious about building a good foundation for local  
persistence, then we should have more precise requirements for Web  
Storage. Otherwise, we risk prematurely standardizing some dialect of  
SQL supported by SQLite as "Web Storage".

Here's a compilation of requirements from what I have read in this  
thread and other work I have done in this area:
data items may be small and record like objects or large and document  
like.
Must support data items that are hundreds of megabytes or tens of bytes.
Must support data items that are binary or text
Should support sending and receiving large data items directly over  
the network without bringing them into memory.
handle large data sets efficiently; in particular, it should not  
require that the entire data set fit into memory at one time,
execute queries over the entire dataset,
queries may involve full-text search, parameterized or query by  
example search, or a combination
queries provide operators for all JavaScript data types
queries permit user defined functions
integrate well with JavaScript.
to enable offline operation, permit a local view over or update of  
data that may be remotely mastered
integrate well with Web - MIME types, URL, and HTTP methods
not require a rigid schema definition and support decentralized schema  
evolution
Application data may only be accessed by authorized users

[1] http://lists.w3.org/Archives/Public/public-webapps/2008AprJun/0079.html
[2] http://lists.w3.org/Archives/Public/public-webapps/2008OctDec/0104.html