Re: Updating Quota API: Promise, Events and some more

Hi Kinuko,

Very excited to see movement on this API. We are very interested in
implementing this in Gecko once we reach an API that has the right feature
set and polish! The recent changes definitely brings us closer to that
which is great. Definitely looking forward to working with you on this.

I have a few pieces of high-level feedback, before getting into details.

Persistence type granularity:
First off, I don't think that we in Gecko are interested in adding support
for per-objectStore or per-Entry persistence type.

For a couple of reasons we want to use separate files for temporary and
persistent storage. This enables us to both measure the amount of
temporary/persistent storage, as well as clear temporary storage, without
doing slow operations involving running transactions or reading a lot of
files.

Additionally, in our backends we wouldn't have the ability to measure how
much storage a particular objectStore is using. Or to measure how much the
metadata for a particular Entry uses.

So I'd prefer to only support setting storage type on a per-database and
per-filesystem basis.

getStorageType/setStorageType:
I'm not convinced that it makes sense to have a cross-storage-API API for
moving data between the different storage types.

I definitely think that we need to support moving IDB databases or
Filesystems between temporary and permanent storage. In particular moving
data from temporary storage to persistent storage seems very useful.

But I think this is something that we should do on a per-storage-API basis.
I.e. the API for moving an IDB database should probably be part of IDB. And
moving data between filesystems already exists in both the Google
Filesystem proposal as well as the Mozilla filesystem proposal.

requestQuota for temporary storage:
I'd like to better understand the use case for requestQuota for temporary
storage. Are implementations allowed to bring up a prompt when an increased
temporary storage quota is requested? I thought one of the big use cases
for temporary storage was that it would never trigger prompts, though
obviously an exception could be made for the explicit requestQuota function.

If the idea is for it not to bring up a prompt, why would we not simply
always allocate the largest value that could be requested?

Also, I think that we for temporary storage in Gecko would not want to
guarantee that the allocated quota (as reported by queryQuota) for
temporary storage will remain allocated for longer than the running page.
I.e. once a page is closed, we might want to release the quota allocated
and give it to another website.

/ Jonas

On Tue, Aug 13, 2013 at 10:57 PM, Kinuko Yasuda <kinuko@chromium.org> wrote:
> Hi all,
>
> It's been a while since Quota API's FPWD (http://www.w3.org/TR/quota-api/)
> was published and we've gotten several requests/feedbacks so far.
> To address some of the requests and to gain more consensus, I'm thinking
> about making following changes to the Quota API:
>
> * Use Promises rather than callbacks
> * Add Events to notify webapps of important changes in the local storage
> space
> * Establish a way to get and set the storage types (temporary or
persistent)
> of each storage object
>
> This breaks compatibility in the existing implementation, but currently
> it's implemented only in Chrome behind the flag, so I hope/assume it'll be
> ok
> to make incompatible changes. I'm also strongly hoping these changes
> (and debate on them) help building more consensus.
>
> There're also some requests those are not (yet) addressed in this new
draft:
>
> * More granularity in storage types or priorities, rather than sticking to
> the
> rigid two types, so that webapps can indicate which data should be evicted
> first / when.
> * Helper method to estimate 'actual' size of each storage object
> * Helper method to trigger GC/compaction on the local storage
>
> While they look nice-to-have in some situations but may also add more
> complexity in implementation, so I tentatively concluded that they can
> be put off until the next iteration.
>
> New draft needs some more polish but I'd like to get early feedback
> on the new draft.
>
> Detailed draft:
>
> enum StorageType { "temporary", "persistent" };
>
> partial interface Navigator {
> readonly attribute StorageQuota storageQuota;
> };
>
> [NoInterfaceObject] interface StorageInfo {
> unsigned long long usageInBytes;
> unsigned long long quotaInBytes;
> };
>
> [NoInterfaceObject] interface StorageQuota {
> readonly attribute StorageType[] supportedTypes;
>
> Promise<StorageInfo> queryStorageInfo(StorageType type);
> Promise<StorageInfo> requestQuota(StorageType type, unsigned long long
> newQuotaInBytes);
>
> Promise<StorageType> getStorageType((IDBObjectStore or Database or
> Entry) object);
> Promise<void> setStorageType((IDBObjectStore or Database or Entry)
> object, StorageType type);
>
> StorageWatcher createStorageWatcher(StorageType type)
> };
>
> This new draft uses string enums to specify storage types rather than
> separate attributes on navigator (e.g. navigator.temporaryStorage),
> mainly because some methods (like {get,set}StorageType do not fit well
> in split interface) and to preserve greater flexibility to add more
storage
> types in a future. I'm open to discussions though.
>
> supportedTypes are list of all StorageType's supported by the UA.
>
> * queryStorageInfo and requestQuota are Promise version of
> queryUsageAndQuota and requestQuota, which is for querying the current
> storage info (usage and quota) and requesting a new quota, respectively.
> Both return the current (or updated) StorageInfo.
>
> * getStorageType and setStorageType are new methods which are intended to
> work horizontally across multiple storage APIs. getStorageType(object)
> returns the current storage type for the given storage object, and
> setStorageType(object, type) changes the object's storage type.
> They may fail if the storage backend of the object does not support
> Quota API or does not support getting or setting (changing) storage types.
>
> We're aware that this API may not work very well with FileSystem API(s),
> and
> also will need coordination with IndexedDB. Feedback is strongly
> encouraged
> here.
>
> * createStorageWatcher returns a new StorageWatcher object for the given
> storage type, with which webapps can listen to important changes in the
> local storage and can react appropriately (i.e. discard their
> lower-priority
> cache data before UA-initiated merciless eviction happens).
>
> [NoInterfaceObject] interface createStorageWatcher {
> attribute EventHandler onstoragelow;
> attribute EventHandler onstorageok;
> attribute EventHandler onstoragechange;
> };
>
> storagelow and storageok events are modeled after DEVICE_STORAGE_LOW
> and DEVICE_STORAGE_OK intents on android, to give more chances to
> webapps to adjust their local data quickly.
>
> * storagelow event is fired when the remaining storage space becomes lower
> than 10% of all available space for the type, or before the quota backend
> triggers eviction (for temporary case), whichever happens first.
> This event must be also fired once when either one of these conditions has
> been already met when a StorageWatcher is created.
>
> * storageok event is fired when the remaining available storage goes out
of
> 'storagelow' state.
>
> * storagechange event is fired about every 1 second (TBD) or every time
> the quota backend detects the amount of remaining storage is changed,
> whichever is least frequent. This event must be fired before storagelow
> or storageok when the remaining space crosses the threshold.
>
> Again, your review/feedback is really encouraged. Thanks!
>
> Kinuko
>