Re: Polished FileSystem API proposal

On Nov 6, 2013 3:50 AM, "Jonas Sicking" <jonas@sicking.cc> wrote:
>
> On Tue, Nov 5, 2013 at 11:45 AM, Tim Caswell <tim@creationix.com> wrote:
> > If the backend implementation used something like git's data store then
> > duplicate data would automatically be stored only once without any
security
> > implications.  The keys are the literal sha1 of the values.  If two
websites
> > had the same file tree containing the same files, it would be the same
tree
> > object in the storage.  But only sites who have a reference to the hash
> > would have access to it.
> >
> > Also I like the level of fs support that git's filesystem has.  There
are
> > trees, files, executable files, and symlinks. (there are also gitlinks
used
> > for submodules, but let's ignore those for now)
>
> Sounds like IndexedDB is a better fit than a filesystem for this use case.

I'm not saying that I want a filesystem so I can implement git.  I'm saying
that git is a great way to implement a filesystem and that git's abilities
are a great baseline for web filesystems.

>
> Note that the use case for the filesystem API isn't "storing files",
> IDB is perfectly capable of doing that. The use case for the
> filesystem API is to satisfy people that want a true filesystem with
> directories etc so that they can:
>
> * Sync to a server-side file system. For example when doing web
> development and want to deploy a website.
> * Use hierarchical filesystem: URLs.
> * Support in-place editing of large files.
> * Because filesystems are familiar.
>

Yes.  I completely agree.  Personally in my projects, I wish there was to
create custom filesystems backed by JavaScript code that are exported to
hierarchical URLs.  It would work somewhat like server-side web development
where a js function handles the request for a file.  This is basically what
fuse does for Linux.

I would love to be able to create a custom union filesystem that has
layers.  One would be in-memory and another would be backed by a local bit
database.  It would all be implemented in pure user space js.

> A simple key-value storage, where the values happen to be files,
> doesn't need a filesystem API.

Speaking of, I currently have a git implementation that uses IndexedDB as
the data store.  I don't call IDB simple by any means.  I don't need
transactions or indexes.  The leveldb interface is much closer. To what I
need to implement git.

>
> / Jonas

So in summary:

- The filesystem interface could export symlinks, files, trees, and
executable bit.   (Basically feature parity with git trees)
- using a git style backing store (the content addressable part) is a great
way to store data from multiple domains without duplicating storage or
introducing security issues around sharing data.
- I really want a fuse-like way to create virtual file systems where I
implement the back end using user space code.
- I want a key/value system that's simple like local storage, but async and
supports binary values.

Sorry for conflating so many topics in one email.  They are all related and
I'm new to this list.  I'm not sure of the proper procedure for creating
new topics.

- Tim Caswell