Re: Updating Quota API: Promise, Events and some more

Hi Joshua,

On Thu, Aug 15, 2013 at 6:33 AM, Joshua Bell <jsbell@chromium.org> wrote:
>
>      Promise<StorageType> getStorageType((IDBObjectStore or Database or
>> Entry) object);
>>     Promise<void> setStorageType((IDBObjectStore or Database or Entry)
>> object, StorageType type);
>>
>
> For IndexedDB, an object store is (probably) too low a level to specify a
> storage type; ISTM that a database makes more sense as the level of
> granularity for specifying storage, since that avoids the complexity of a
> store disappearing out from within a database. Was the use of object store
> here intentional?
>
> From an API perspective, passing an IDBObjectStore instance also doesn't
> make much sense as that sort of object is really a transaction-specific
> handle. Before delving deeply, my gut reaction is that to fit into this API
> you would need to pass an IDBDatabase connection object, and it would
> generate an error unless called during a versionchange transaction (which
> guarantees there are no other connections).
>

Thanks for the very informative feedback. The use of object store wasn't
really intentional (it was partly due to lack of my understanding), and if
IDBDatabase connection object makes more sense I'm open to making the
change (more discussion follows).

That still feels like an odd mix of two APIs. An approach that we (Moz +
> Google) have talked about would be to extend the IDBFactory.open() call
> with an options dictionary, e.g.
>
> request = indexedDB.open({ name: ..., version: ..., storage: "temporary"
> });
>

Honestly I was debating whether I should add these methods to this
proposal, as I had the same feeling as yours when I heard the original
request, but if IDB is going to have the ability to set storage type at
creation that sounds great, and would make the current situation much
better.

After all what is lacking in the current quota (+idb or other storage API)
world is:
(1) to know which storage type a storage object belongs to (or in other
words how the data stored in idb is going to be managed by UA), and
(2) to adjust the amount of data in each storage type to avoid undesirable
data deletion/eviction by UA.

On a tangent...
>
> An open question is if the storage type (1) can be assigned only when an
> IDB database is created, or (2) can be changed, allowing an IDB database to
> be moved while retaining data, or (3) defines a namespace between origin
> and database, i.e. "example.com" / "permanent" / "db-1" and "example.com"
> / "temporary" / "db-1" co-exist as separate databases.
>
> What are your thoughts on those 3 options with respect to other storage
> systems?
>

If I were to develop a mobile offline app I could live with (1), though I
may not fall into a typical webapp developer category. If the granularity
to set storage type is IDB database (2) does not sound very useful.  (3)
sounds interesting and useful, though I'm not sure how much complexity it
would add in the implementation.


>     StorageWatcher createStorageWatcher(StorageType type)
>>   };
>>
>> This new draft uses string enums to specify storage types rather than
>> separate attributes on navigator (e.g. navigator.temporaryStorage),
>> mainly because some methods (like {get,set}StorageType do not fit well
>> in split interface) and to preserve greater flexibility to add more
>> storage
>> types in a future. I'm open to discussions though.
>>
>>  supportedTypes are list of all StorageType's supported by the UA.
>>
>> * queryStorageInfo and requestQuota are Promise version of
>>   queryUsageAndQuota and requestQuota, which is for querying the current
>>   storage info (usage and quota) and requesting a new quota, respectively.
>>   Both return the current (or updated) StorageInfo.
>>
>> * getStorageType and setStorageType are new methods which are intended to
>>   work horizontally across multiple storage APIs. getStorageType(object)
>>   returns the current storage type for the given storage object, and
>>   setStorageType(object, type) changes the object's storage type.
>>   They may fail if the storage backend of the object does not support
>>   Quota API or does not support getting or setting (changing) storage
>> types.
>>
>>   We're aware that this API may not work very well with FileSystem
>> API(s), and
>>   also will need coordination with IndexedDB. Feedback is strongly
>> encouraged
>>   here.
>>
>>
> Since it doesn't seem like a good fit for FileSystem or (IMHO) IndexedDB,
> can you give an example where this API does make sense? That might help
> inform what the shape of such a horizontal API would be like that would
> work with IDB, etc.
>

Putting aside the API details now (I admit these methods are not in a
really good shape), the real desire behind this request (and another
request to have more granularity in storage types, which is not addressed
in this draft version) is, to allow webapps to specify a part of their data
not to be evicted when the storage pressure is getting high, and the
decision on which data should be kept or evicted may vary depending on the
storage situation.  If an webapp can move some of their critical data into
'persistent' while leaving other in 'temporary' whenever it gets
'storagelow' events, the webapp can hopefully continue to work in a much
consistent way.

In other words, if each storage API provides the same or similar set of
features to support this desire I don't think the API details matter much.


 * createStorageWatcher returns a new StorageWatcher object for the given
>>   storage type, with which webapps can listen to important changes in the
>>   local storage and can react appropriately (i.e. discard their
>> lower-priority
>>   cache data before UA-initiated merciless eviction happens).
>>
>> [NoInterfaceObject] interface createStorageWatcher {
>>   attribute EventHandler onstoragelow;
>>   attribute EventHandler onstorageok;
>>   attribute EventHandler onstoragechange;
>> };
>>
>> storagelow and storageok events are modeled after DEVICE_STORAGE_LOW
>> and DEVICE_STORAGE_OK intents on android, to give more chances to
>> webapps to adjust their local data quickly.
>>
>> * storagelow event is fired when the remaining storage space becomes lower
>>   than 10% of all available space for the type, or before the quota
>> backend
>>   triggers eviction (for temporary case), whichever happens first.
>>   This event must be also fired once when either one of these conditions
>> has
>>   been already met when a StorageWatcher is created.
>>
>> * storageok event is fired when the remaining available storage goes out
>> of
>>   'storagelow' state.
>>
>>
> I should go look at the DEVICE_STORAGE_OK docs, but what actions would a
> typical web app take in response to storageok? Is this to allow an app to
> enter/exit a "low storage" state (i.e. restrict how much data it writes)?
> Otherwise it doesn't seem particularly actionable.
>

Hmm... I probably need to study how STORAGE_OK is being handled by mobile
apps too.  The reason I added storagelow event is because I often hear that
polling the remaining storage space is not very efficient/desirable, but
storageok is added merely because I just thought following preceding
examples (e.g. android) would make sense.


> * storagechange event is fired about every 1 second (TBD) or every time
>>   the quota backend detects the amount of remaining storage is changed,
>>   whichever is least frequent. This event must be fired before storagelow
>>   or storageok when the remaining space crosses the threshold.
>>
>>
> This sounds like storagechange would fire every second even if storage
> usage is constant - is that correct or am I misreading?
>

storagechange should not fire if the amount of remaining storage is
constant.  If UA exposes the 'real' storage status to webapps and if
multiple apps are contending with on the same storage space it may end up
firing storagechange event every second.

This event might be a bit overkill (as well as storageok), and I think we
can start with the simplest 'storagelow' event, as it has the clearest
reason to be added.


> If the user performs a manual action (e.g. "clear browsing data"), I
> assume this event would fire as well. Does a user-initiated action merit
> any other sort of notification?
>

If a user-initiated action results more available space for webapps they
may want to change their policy about how they store their data in
temporary or persistent storage space.  But again, other events than
storagelow might be arguable and they could be put off for now.



>
>> Again, your review/feedback is really encouraged.  Thanks!
>>
>> Kinuko
>>
>>
>