RE: flush() | was Re: FileSystem API Comments

Hi Arun,

We believe that the flush property should be specified when getting the file handle as in option 1. One benefit of this is that it will enable the buffer of both reads and writes for the same handle. On the other hand, if we specify it on every write operation (as in option 2) we could run into inconsistencies when invoking the write method with and without the flag.

Based on our interpretation of option 1, it seems as though the flush() function would not be available. This is how we perceived its usage would look like:

navigator.getFileSystem().then(function(root) {
        return root.openWrite("path/to/file.txt", {autoFlushing: true});
}).then(function(fileHandle) {
        fileHandle.write(blob);  // data is written to the system cache and is flushed to disk without delay
});

Thank you,
Ali


From: Arun Ranganathan [mailto:arun@mozilla.com] 
Sent: Friday, October 31, 2014 11:19 AM
To: Ali Alabbas
Cc: Web Applications Working Group WG
Subject: flush() | was Re: FileSystem API Comments

Greetings Ali!

I've been thinking about the discussion of flush(), and would like to see if I can make my previous statement a bit more nuanced. It turns out that flush() (in the vein of fsync/sync) is pretty useful, and after discussion with a few folks within Mozilla, I realize that it isn't as simple as tacking it on to the "write-family" of Promises - as you point out, it is a potentially expensive operation.

Something like a flush feature might help the following use cases:

1. Creating a database technology on top of the filesystem technology. This might include IndexedDB, but also WebSQL (as a hypothetical example). Most transactional operations like this need the ability to do something like flush.

2. Then, there's the use case of compiling C++ codebases to JS. Well-known examples of this are games, leveraging asm.js. In this genre of use case, sometimes a large database is brought over (e.g. sqlite). It could be memory backed, but it is a definite bonus if it could be filesystem backed. Something like flush helps make that a possibility.

Now the question is how to do this in a WebAPI, allowing for the power along with the mitigations that a web app might need, notably for performance? A few ideas below:

On Oct 21, 2014, at 4:36 PM, Ali Alabbas <alia@microsoft.com> wrote:


 * flush()
     - This is costly functionality to expose and is likely to be overused by callers. It would be beneficial to automatically flush changes to disk by allowing the default file write behavior by the OS. For example, on Windows, we would leave it up to the filesystem cache to determine the best time to flush to disk. This is non-deterministic from the app's point of view, but the only time it is a potential problem is when there's a hard power-off. Most apps should not be concerned with this; only apps that have very high data reliability requirements would need the granular control of flushing to disk. In those cases a developer should use IndexedDB. So we should consider obscuring this functionality since it's not a common requirement and has a performance impact if it's widely used.


I agree with the idea of obscuring the functionality a bit, especially given that it might not be necessary for a large class of operations. A few ways to do that:

1. Add this to a dictionary option when coining the FileHandleWritable from the Directory (e.g. add it to something like the OpeWriteOptions: http://w3c.github.io/filesystem-api/Overview.html#widl-Directory-openWrite-Promise-FileHandleWritable--DOMString-File-path-OpenWriteOptions-options).

This way, the developer has the ability to "coin" a "more expensive" promise, if that particular set of write operations needs this feature.

2. Add this to the set of options on the FileHandleWritable.

This could be by dictionary, again. Or, it could be a boolean on the FileHandleWritable's write(). This latter might not be specific enough. Like other implementations, ours is not going to buffer anything, but rely on the underlying operating system's buffer for writes and reads.

3. Stick with the idea of a method, like flush(). In this case, we might have to caveat the use of this, since the possibility of inexperienced developer misuse is high :-) It might help to see if we can determine some boundaries on this.

Any feedback on some of these options would be valuable. I am thinking of 1. and 2.

- A*

Received on Thursday, 6 November 2014 22:39:55 UTC