Re: [IndexedDB] Granting storage quotas

On Apr 20, 2010, at 3:19 PM, Jeremy Orlow wrote:

> This way of thinking is incompatible with offline web apps.  If I'm offline and I "send" and email, it needs to stay queued up to send until I'm reconnected to the internet.

I think the problem is that data loss could occur regardless of "guarantees". 

Here are the classes of storage I hear you asking for:

1. temporary (no likelihood of data being being stored after session ends)
2. evictable (no limit per site, except global limits, eviction candidates chosen arbitrarily, including while application is running)
3. non-evictable (no eviction, but data loss possible, limited by user)

Is this making sense?

> 
> Anyone wanting to debate whether or not the UA should be free to clean up "persistent storage" without asking the user should read http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-August/thread.html#22289 and the various other threads it spawned first and only re-start things if they have new information to add.
> 
> J
> 
> On Tue, Apr 20, 2010 at 3:10 PM, Nikunj Mehta <nikunj@o-micron.com> wrote:
> As I see it, there's no such thing as "permanent" storage for Web browser managed data. Even if a site expresses preferences that it would like to keep its data resident for a long time, there cannot be a "guarantee" for the data to be there permanently. If applications are bound to have to deal with data disappearing while they are not running, we should not need to spec browser behavior around the notion of purgeable or permanent. 
> 
> It makes sense for an application, OTOH, to say that it does not need data to be stored on disk. IOW, create a database that is non-durable and, hence, kept only in memory. Such databases are required to be empty upon creation. They may be spilled over to disk, if implementations like to, but they will not be retained from session to session.
> 
> Nikunj
> 
> 
> On Apr 20, 2010, at 2:37 PM, Michael Nordman wrote:
> 
>> I'd like to back up and challenge the notion of a per-site quota.
>> 
>> In this discussion and others there is an underlying assumption that each site has some well defined limit that the user-agent has granted it. I doubt that's the best model. (Fyi: the chrome team's overly simplistic model whereby each site gets 5M was not chosen because its a good model, this was done just to proceed with building out the storage APIs independent of a real storage management strategy).
>> 
>> I'd like to set aside the per-site quota assumption and explore some alternative models for web storage management.
>> 
>> Some thoughts about the world we're designing for. There are an open ended number of sites, each of which *could* use web storage in some form. From that fact alone, it's impossible to come up with a quota that could be granted to each and every site. It seems likely that the number of sites that will actually require "permanent" storage is small compared to the number of sites that *could* make use of more "volatile" storage, to borrow jorlow's term, where the volatile data on disk can get scrapped by the user-agent as needed. Maybe a better term for that class of storage is "purgeable"?
>> 
>> Maybe we should be designing for what seems to be the more common case, lots of sites that make use of volatile/purgeable storage. But also come up a means whereby the smaller number of sites that require stronger guarantees can express the need for more permanent storage.
>> 
>> "What if" by default all local storage is "purgeable". A lot of quota issues melt away in this case since the user agent is free to reclaim anything at anytime. I think it'd be reasonable if the user-agent never asked the user anything on a per-site basis. A user-agent could warn when system disk space crossed thresholds and give the user an option to set limits  on system disk space usage for webstorage as a whole.
>> 
>> "What if" when creating local storage repositories (WebDBs or IndexedDBs or WebFileSystems or AppCaches) there was an optional means for the webapp to express "please consider this a permanent storage repository". The first time a site request "permanent" storage could be a reasonable time to interact with the user in some form, or to consult the user prefs about allowing permanent storage w/o being asked.
>> 
>> I think ericu is baking in a distinction in between 'permanent' and 'temporary' in the FileSystem API he's working on. Some harmony across all flavors of local storage could be good.
>> 
>> I actually think local storage management is an area where the webplatform has a chance to do a much better job then the desktop platforms have historically done. We don't need no stinking quotas ;) But we also don't need untold amounts of unused permanent storage littering disk drives needlessly around the globe (until the user gets a new system). A silly analogy. A computer is like a ship at sea. After years of usage, a whole bunch of barnacles build up on the hull and slow the vessel down. The webplatform to date is barnacle free in this area because there are no permanent local storage facilities... lets try to make these new features not so barnacle prone too.
>> 
>> Cheers
>> -Michael
>> 
>> On Tue, Apr 20, 2010 at 11:17 AM, Shawn Wilsher <sdwilsh@mozilla.com> wrote:
>> On 4/20/2010 4:11 AM, Mark Seaborn wrote:
>> 1) It doesn't allow a web app to ask for a storage allocation up front,
>> before it starts to consume the storage.
>> Why does that matter?
>> 
>> 
>> 2) In Opera, the quota can only be increased in multiples of about 15, so it
>> takes three prompts to get up into the range of gigabytes.
>> But there is an unlimited option, yeah?
>> 
>> 
>> 3) The web app can't choose when the question is put to the user.
>> 4) The web app doesn't know how much storage has been allocated, so it
>> doesn't know when a question will be asked.
>> 5) In Opera, if the user chooses "Reject", they don't get prompted again.
>> This means that asking the user at an appropriate time is important for the
>> continued functioning of the web app.  Prompting the user at the wrong time
>> will interrupt them with a page-modal dialog which they might want to get
>> rid of with "Reject", which would potentially break the web app by leaving
>> it unable to get more storage.
>> These all feel like user-agent specific worries on how the user agent wants to bring this to the attention of the user.
>> 
>> Cheers,
>> 
>> Shawn
>> 
>> 
> 
>