Re: Polished FileSystem API proposal

On Tue, Jul 23, 2013 at 4:48 AM, Jonas Sicking <jonas@sicking.cc> wrote:

> On Mon, Jul 22, 2013 at 11:18 AM, Jan Varga <jan.varga@gmail.com> wrote:
> > On Sat, Jul 13, 2013 at 2:31 AM, Jonas Sicking <jonas@sicking.cc> wrote:
> >>
> >> Hi All,
> >>
> >> Yesterday a few of us at mozilla went through the FileSystem API
> >> proposal we previously sent [1] and tightened it up.
> >>
> >
> > It was also pointed out that we should address multi-file locking too.
> > One of the options is to make openRead() and openWrite() take a sequence.
> >
> > interface Directory {
> >   Promise<FileHandle> openRead((DOMString or File) file);
> >   Promise<FileHandleWritable> openWrite((DOMString or File) file,
> > OpenWriteOptions options);
> >   Promise<sequence<FileHandle>> openRead(sequence<(DOMString or File)>
> > files);
> >   Promise<sequence<FileHandleWritable>> openWrite(sequence<(DOMString or
> > File)> files, OpenWriteOptions options);
> > }
> >
> > So, this works with the current proposal, Jonas has a more complex
> solution
> > that probably requires bigger changes in the proposed file system API.
> > I'll let Jonas to describe it.
>
> First of all, I'm not sure that multi-file-locking is something that
> we need to solve in this API at all. At least not yet. The google API
> doesn't support locking of any type (as far as I can tell), so I think
> starting with locking on a single-file basis is a better and simpler
> place to start.
>
> The main concern I have with this approach is that it creates a very
> specific API which solves a pretty narrow problem of copying data
> between two files. It doesn't support copying data between a server
> and a file, or between indexedDB and a file.
>
> I can think of two more generic solutions that will solve these problems:
>
> A) Adding something like inotify.
> I.e. add the ability to get notifications about changes to part of a
> filesystem. This way a page could create a lock-file when it wants to
> prevent others from accessing a particular part of the filesystem.
>
> If the lockfile already exists, it could register to be notified when
> the lockfile is removed. Once the file is removed it would create the
> file and start accessing the files in whatever way it wants.
>
> This is how multi-process access to application data is often handled
> in filesystems today.
>
> Adding something like inotify would also enable use cases like having
> a worker synchronize a filesystem to a server. Other parts of the
> application could simply access the filesystem directly and do
> whatever modifications it wants. Those notifications will
> automatically be noticed and synchronized to the server by the worker.
>
> B) Add a generic "cross-window lock" mechanism.
>
> Basically an API for asynchronously requesting a lock. Once the lock
> becomes available the caller would be notified. When the caller is
> done using the lock, he/she calls a function to explicitly release the
> lock, thus enabling other callers to get notified that they are now
> holding the lock.
>
> A lock is never automatically released by the platform on some timeout
> or similar. The only time the lock is forcefully released is when the
> user closes the page that created the lock.
>
> It is the application's responsibility to determine when it is
> appropriate to grab the lock. I.e. there is no platform connection
> between a lock and the resources that it protects. This way a lock
> could represent anything from the whole filesystem, a couple of IDB
> database and some server resources, to just parts of a file.
>
> Darin Fisher has proposed something similar in the past as I recall
> it. Though the details might not exactly match the above.
>
>
> Pros of A
> - Solves more use cases than simply locking-related ones. We've
> discussed adding similar observer mechanisms to IDB because the same
> non-locking-related usecases have come up there.
> - Enables building something like solution B on top of it.
>
> Cons of A
> - More complicated to use correctly. For example it's important to
> register for the notification before checking if the lock file is
> already created. Otherwise there's a risk that the lock file is
> removed between the time when the page checks if its there and the
> notification is registered.
>
> Pros of B
> - Easier to use than A
>
> Cons of B
> - Doesn't help with use cases other than locking. I.e. to enable one
> window to notice when another window modified some data, you would
> either have to use polling, or create custom signaling mechanisms and
> make sure to use those whenever something in the filesystem is
> modified.


I vote for B, it's more generic, simpler to use and can be implemented
without dependency to the FileSystem API (thus it'd have higher probability
to get implemented more quickly by multiple vendors).

Separately from the locking issue, inotify-like feature (e.g. FileWatcher
and DirectoryWatcher) has been also requested in Chrome version's FS API,
and I can imagine the feature would get some popularity.  But my personal
feeling is it might be too rich for 'simple' sandboxed file-storage API (at
least for version 1).

/ Jonas
>
>