Re: [IndexedDB] Granting storage quotas

On Tue, Apr 20, 2010 at 5:18 PM, Nikunj Mehta <nikunj@o-micron.com> wrote:

>
> On Apr 20, 2010, at 3:19 PM, Jeremy Orlow wrote:
>
> This way of thinking is incompatible with offline web apps.  If I'm offline
> and I "send" and email, it needs to stay queued up to send until I'm
> reconnected to the internet.
>
>
> I think the problem is that data loss could occur regardless of
> "guarantees".
>
> Here are the classes of storage I hear you asking for:
>
> 1. temporary (no likelihood of data being being stored after session ends)
> 2. evictable (no limit per site, except global limits, eviction candidates
> chosen arbitrarily, including while application is running)
> 3. non-evictable (no eviction, but data loss possible, limited by user)
>
> Is this making sense?
>

That categorization does make sense, including the temporary.

I think we're (i know i am) most interested in the distinction between
evictable and non-evictable right now because the quota management and ui
implications. We've introduced these storage APIs that result in files on
disk, and now we want to put in some real-world storage management features.
The dumbed down 5MB per origin was a stop gap measure.

This is reminding me of Mike Wilson's  "state handling" post about different
scopes and lifetimes...
http://old.nabble.com/html5-state-handling:-overview-and-extensions-td24034773.html


>
> Anyone wanting to debate whether or not the UA should be free to clean up
> "persistent storage" without asking the user should read
> http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-August/thread.html#22289 and
> the various other threads it spawned first and only re-start things if they
> have new information to add.
>
> J
>
> On Tue, Apr 20, 2010 at 3:10 PM, Nikunj Mehta <nikunj@o-micron.com> wrote:
>
>> As I see it, there's no such thing as "permanent" storage for Web browser
>> managed data. Even if a site expresses preferences that it would like to
>> keep its data resident for a long time, there cannot be a "guarantee" for
>> the data to be there permanently. If applications are bound to have to deal
>> with data disappearing while they are not running, we should not need to
>> spec browser behavior around the notion of purgeable or permanent.
>>
>> It makes sense for an application, OTOH, to say that it does not need data
>> to be stored on disk. IOW, create a database that is non-durable and, hence,
>> kept only in memory. Such databases are required to be empty upon creation.
>> They may be spilled over to disk, if implementations like to, but they will
>> not be retained from session to session.
>>
>> Nikunj
>>
>>
>> On Apr 20, 2010, at 2:37 PM, Michael Nordman wrote:
>>
>> I'd like to back up and challenge the notion of a per-site quota.
>>
>> In this discussion and others there is an underlying assumption that each
>> site has some well defined limit that the user-agent has granted it. I doubt
>> that's the best model. (Fyi: the chrome team's overly simplistic model
>> whereby each site gets 5M was not chosen because its a good model, this was
>> done just to proceed with building out the storage APIs independent of a
>> real storage management strategy).
>>
>> I'd like to set aside the per-site quota assumption and explore some
>> alternative models for web storage management.
>>
>> Some thoughts about the world we're designing for. There are an open ended
>> number of sites, each of which *could* use web storage in some form. From
>> that fact alone, it's impossible to come up with a quota that could be
>> granted to each and every site. It seems likely that the number of sites
>> that will actually require "permanent" storage is small compared to the
>> number of sites that *could* make use of more "volatile" storage, to borrow
>> jorlow's term, where the volatile data on disk can get scrapped by the
>> user-agent as needed. Maybe a better term for that class of storage is
>> "purgeable"?
>>
>> Maybe we should be designing for what seems to be the more common case,
>> lots of sites that make use of volatile/purgeable storage. But also come up
>> a means whereby the smaller number of sites that require stronger guarantees
>> can express the need for more permanent storage.
>>
>> "What if" by default all local storage is "purgeable". A lot of quota
>> issues melt away in this case since the user agent is free to reclaim
>> anything at anytime. I think it'd be reasonable if the user-agent never
>> asked the user anything on a per-site basis. A user-agent could warn when
>> system disk space crossed thresholds and give the user an option to set
>> limits  on system disk space usage for webstorage as a whole.
>>
>> "What if" when creating local storage repositories (WebDBs or IndexedDBs
>> or WebFileSystems or AppCaches) there was an optional means for the webapp
>> to express "please consider this a permanent storage repository". The first
>> time a site request "permanent" storage could be a reasonable time to
>> interact with the user in some form, or to consult the user prefs about
>> allowing permanent storage w/o being asked.
>>
>> I think ericu is baking in a distinction in between 'permanent' and
>> 'temporary' in the FileSystem API he's working on. Some harmony across all
>> flavors of local storage could be good.
>>
>> I actually think local storage management is an area where the webplatform
>> has a chance to do a much better job then the desktop platforms have
>> historically done. We don't need no stinking quotas ;) But we also don't
>> need untold amounts of unused permanent storage littering disk drives
>> needlessly around the globe (until the user gets a new system). A silly
>> analogy. A computer is like a ship at sea. After years of usage, a whole
>> bunch of barnacles build up on the hull and slow the vessel down. The
>> webplatform to date is barnacle free in this area because there are no
>> permanent local storage facilities... lets try to make these new features
>> not so barnacle prone too.
>>
>> Cheers
>> -Michael
>>
>> On Tue, Apr 20, 2010 at 11:17 AM, Shawn Wilsher <sdwilsh@mozilla.com>wrote:
>>
>>> On 4/20/2010 4:11 AM, Mark Seaborn wrote:
>>>
>>>> 1) It doesn't allow a web app to ask for a storage allocation up front,
>>>> before it starts to consume the storage.
>>>>
>>> Why does that matter?
>>>
>>>
>>>  2) In Opera, the quota can only be increased in multiples of about 15,
>>>> so it
>>>> takes three prompts to get up into the range of gigabytes.
>>>>
>>> But there is an unlimited option, yeah?
>>>
>>>
>>>  3) The web app can't choose when the question is put to the user.
>>>> 4) The web app doesn't know how much storage has been allocated, so it
>>>> doesn't know when a question will be asked.
>>>> 5) In Opera, if the user chooses "Reject", they don't get prompted
>>>> again.
>>>> This means that asking the user at an appropriate time is important for
>>>> the
>>>> continued functioning of the web app.  Prompting the user at the wrong
>>>> time
>>>> will interrupt them with a page-modal dialog which they might want to
>>>> get
>>>> rid of with "Reject", which would potentially break the web app by
>>>> leaving
>>>> it unable to get more storage.
>>>>
>>> These all feel like user-agent specific worries on how the user agent
>>> wants to bring this to the attention of the user.
>>>
>>> Cheers,
>>>
>>> Shawn
>>>
>>>
>>
>>
>
>