Re: Updating Quota API: Promise, Events and some more

Hi Jonas,

On Thu, Aug 15, 2013 at 1:20 PM, Jonas Sicking <jonas@sicking.cc> wrote:

> Hi Kinuko,
>
> Very excited to see movement on this API. We are very interested in
> implementing this in Gecko once we reach an API that has the right feature
> set and polish! The recent changes definitely brings us closer to that
> which is great. Definitely looking forward to working with you on this.
>
It's great to know that Mozilla is interested in this API, and thanks very
much for the valuable feedbacks.

> I have a few pieces of high-level fedback, before getting into details.
>
> Persistence type granularity:
> First off, I don't think that we in Gecko are interested in adding support
> for per-objectStore or per-Entry persistence type.
>
> For a couple of reasons we want to use separate files for temporary and
> persistent storage. This enables us to both measure the amount of
> temporary/persistent storage, as well as clear temporary storage, without
> doing slow operations involving running transactions or reading a lot of
> files.
>
> Additionally, in our backends we wouldn't have the ability to measure how
> much storage a particular objectStore is using. Or to measure how much the
> metadata for a particular Entry uses.
>
> So I'd prefer to only support setting storage type on a per-database and
> per-filesystem basis.
>
> getStorageType/setStorageType:
> I'm not convinced that it makes sense to have a cross-storage-API API for
> moving data between the different storage types.
>
> I definitely think that we need to support moving IDB databases or
> Filesystems between temporary and permanent storage. In particular moving
> data from temporary storage to persistent storage seems very useful.
>
> But I think this is something that we should do on a per-storage-API
> basis. I.e. the API for moving an IDB database should probably be part of
> IDB. And moving data between filesystems already exists in both the Google
> Filesystem proposal as well as the Mozilla filesystem proposal.
>
I mostly agree with your points that 1. it's desirable to support moving
data across temporary and persistent storage, and 2. it's more natural to
support it on a per-storage-API basis.  Also regarding the persistence type
granularity we (Chrome) are actually in a very similar situation, i.e. it's
not very easy to support per-objectStore or per-Entry persistence type.

As I wrote in the response to Joshua's feedback the real 'desire' to have
this kind of methods is to give more control to webapps on which data to be
kept or evicted when the storage is getting tight, and I'm not willing to
stick to the current proposal.  If each storage API can support the desire
of mobile/offline webapps that'd be good enough (if not great).

After hearing feedbacks from you and Joshua, and knowing that IDB is going
to support storage types in a near future, I feel we can drop these methods
from this draft.  (I'm also going to reach out the original requester to
see if it works for them too)

>  requestQuota for temporary storage:
> I'd like to better understand the use case for requestQuota for temporary
> storage. Are implementations allowed to bring up a prompt when an increased
> temporary storage quota is requested? I thought one of the big use cases
> for temporary storage was that it would never trigger prompts, though
> obviously an exception could be made for the explicit requestQuota function.
>
> If the idea is for it not to bring up a prompt, why would we not simply
> always allocate the largest value that could be requested?
>
> Also, I think that we for temporary storage in Gecko would not want to
> guarantee that the allocated quota (as reported by queryQuota) for
> temporary storage will remain allocated for longer than the running page.
> I.e. once a page is closed, we might want to release the quota allocated
> and give it to another website.
>
This is a very good feedback/question.  In Chrome we actually don't really
support requestQuota for temporary storage, we just silently ignore the
request but returns the current available quota to the requesting webapp.
 I also agree that it won't be desirable to preserve the quota allocated
for temporary storage.

Seems like having this method take a storage type just confuses
readers/implementors, we should probably just drop the storage type
parameter and rename it to something like 'requestPersistentQuota', or only
add the method to 'navigator.persistentStorage' attribute?


> / Jonas
>
> On Tue, Aug 13, 2013 at 10:57 PM, Kinuko Yasuda <kinuko@chromium.org>
> wrote:
> > Hi all,
> >
> > It's been a while since Quota API's FPWD (
> http://www.w3.org/TR/quota-api/)
> > was published and we've gotten several requests/feedbacks so far.
> > To address some of the requests and to gain more consensus, I'm thinking
> > about making following changes to the Quota API:
> >
> > * Use Promises rather than callbacks
> > * Add Events to notify webapps of important changes in the local storage
> > space
> > * Establish a way to get and set the storage types (temporary or
> persistent)
> > of each storage object
> >
> > This breaks compatibility in the existing implementation, but currently
> > it's implemented only in Chrome behind the flag, so I hope/assume it'll
> be
> > ok
> > to make incompatible changes. I'm also strongly hoping these changes
> > (and debate on them) help building more consensus.
> >
> > There're also some requests those are not (yet) addressed in this new
> draft:
> >
> > * More granularity in storage types or priorities, rather than sticking
> to
> > the
> > rigid two types, so that webapps can indicate which data should be
> evicted
> > first / when.
> > * Helper method to estimate 'actual' size of each storage object
> > * Helper method to trigger GC/compaction on the local storage
> >
> > While they look nice-to-have in some situations but may also add more
> > complexity in implementation, so I tentatively concluded that they can
> > be put off until the next iteration.
> >
> > New draft needs some more polish but I'd like to get early feedback
> > on the new draft.
> >
> > Detailed draft:
> >
> > enum StorageType { "temporary", "persistent" };
> >
> > partial interface Navigator {
> > readonly attribute StorageQuota storageQuota;
> > };
> >
> > [NoInterfaceObject] interface StorageInfo {
> > unsigned long long usageInBytes;
> > unsigned long long quotaInBytes;
> > };
> >
> > [NoInterfaceObject] interface StorageQuota {
> > readonly attribute StorageType[] supportedTypes;
> >
> > Promise<StorageInfo> queryStorageInfo(StorageType type);
> > Promise<StorageInfo> requestQuota(StorageType type, unsigned long long
> > newQuotaInBytes);
> >
> > Promise<StorageType> getStorageType((IDBObjectStore or Database or
> > Entry) object);
> > Promise<void> setStorageType((IDBObjectStore or Database or Entry)
> > object, StorageType type);
> >
> > StorageWatcher createStorageWatcher(StorageType type)
> > };
> >
> > This new draft uses string enums to specify storage types rather than
> > separate attributes on navigator (e.g. navigator.temporaryStorage),
> > mainly because some methods (like {get,set}StorageType do not fit well
> > in split interface) and to preserve greater flexibility to add more
> storage
> > types in a future. I'm open to discussions though.
> >
> > supportedTypes are list of all StorageType's supported by the UA.
> >
> > * queryStorageInfo and requestQuota are Promise version of
> > queryUsageAndQuota and requestQuota, which is for querying the current
> > storage info (usage and quota) and requesting a new quota, respectively.
> > Both return the current (or updated) StorageInfo.
> >
> > * getStorageType and setStorageType are new methods which are intended to
> > work horizontally across multiple storage APIs. getStorageType(object)
> > returns the current storage type for the given storage object, and
> > setStorageType(object, type) changes the object's storage type.
> > They may fail if the storage backend of the object does not support
> > Quota API or does not support getting or setting (changing) storage
> types.
> >
> > We're aware that this API may not work very well with FileSystem API(s),
> > and
> > also will need coordination with IndexedDB. Feedback is strongly
> > encouraged
> > here.
> >
> > * createStorageWatcher returns a new StorageWatcher object for the given
> > storage type, with which webapps can listen to important changes in the
> > local storage and can react appropriately (i.e. discard their
> > lower-priority
> > cache data before UA-initiated merciless eviction happens).
> >
> > [NoInterfaceObject] interface createStorageWatcher {
> > attribute EventHandler onstoragelow;
> > attribute EventHandler onstorageok;
> > attribute EventHandler onstoragechange;
> > };
> >
> > storagelow and storageok events are modeled after DEVICE_STORAGE_LOW
> > and DEVICE_STORAGE_OK intents on android, to give more chances to
> > webapps to adjust their local data quickly.
> >
> > * storagelow event is fired when the remaining storage space becomes
> lower
> > than 10% of all available space for the type, or before the quota backend
> > triggers eviction (for temporary case), whichever happens first.
> > This event must be also fired once when either one of these conditions
> has
> > been already met when a StorageWatcher is created.
> >
> > * storageok event is fired when the remaining available storage goes out
> of
> > 'storagelow' state.
> >
> > * storagechange event is fired about every 1 second (TBD) or every time
> > the quota backend detects the amount of remaining storage is changed,
> > whichever is least frequent. This event must be fired before storagelow
> > or storageok when the remaining space crosses the threshold.
> >
> > Again, your review/feedback is really encouraged. Thanks!
> >
> > Kinuko
> >
>
>