Re: File API: Directories and System

On Wed, Dec 9, 2009 at 10:18 AM, Robin Berjon <robin@robineko.com> wrote:
> Hi all,
>
> as discussed on the call, please find in CVS a rough proposal for the Directories and System parts of the File API:
>
>  http://dev.w3.org/2009/dap/file-system/file-dir-sys.html
>
> It is based on the functionality found in BONDI's file system API, with changes to integrate with FileReader and FileWriter (soon to be updated), and naming that attempts to be more vernacular. The thing that's missing from BONDI is all that relates to mounting and being notified of mounts. I'm still fuzzy about the former (can someone clarify the intended usage?), while the latter is something that's flagged for addition.
>
> There are two entry points. One is through navigator.device.fileSystems(), which upon success lists all available FSs. Naturally that only expected to be exposed in trusted environments.
>
> The other is a localFS attribute of window that is intended to work in the same manner as localStorage and friends. I expect that this is as far as browsers may be interested in going. I can think of some useful applications for it.
>
> Hopefully this is a first step in clarifying how we can integrate all the components in the File API family of specifications, to make it a happy family and all, so that we can have our security cake in browsers and eat it in widgets.

Robin:

Thanks for getting this out so quickly.  I feel like we're really getting some
good momentum going on the various file APIs.  Comments inline.


In 3.1:  "The virtual localFS should have a quota. It also shouldn't
be implemented by storing its contents directly to the disk (even
hidden inside a supposedly hard to find profile directory) as a user
could find such content by chance (for instance through a search) or
just by being nosy, and the files contained therein could potentially
be extremely dangerous. Rather, the virtual file systems should be
hidden from normal access by the user. A simple example of an
implementation strategy would be to use a typical archive format (e.g.
 Zip) without its expected file extension."

I disagree--we should leave it up to the UA.  Let's leave some leeway
for UAs to decide if a saved file is dangerous, either by running a
malware checker, disallowing writing executable files, marking files
as coming from an untrusted zone, etc.

Requiring an invisible filesystem removes the simplest and
highest-performance implementation option: a guarded pass-through to a
local filesystem.  It also keeps the user from easy export of files
[without the help of the app] via Finder or Windows Explorer, keeps
iTunes from indexing downloaded MP3s, Photoshop from editing saved
files, etc.  It turns the browser into a silo.

That being said, if we're really exposing a sandboxed section of the
FS, we then have to worry about a number of other issues beyond
directory separator.  Here's a quick list: max filename length [in
bytes or characters, and in which encoding], max path length, max
directory nesting depth, max number of entries in a directory, illegal
path characters [<>:"/\|?*], illegal file names "foo.prn", "lpt1",
"foo.", the fact that case-insensitivity is locale-dependent.  I have
gathered a list of the constraints of various popular systems, and can
post them when we get to that point if we go this way.

In 3.3.1:

"The localFS attribute points to a Directory object that is the root
of the file system (its parent is null)."

I don't think the localFS attribute is the right way to expose the FS
in the document context.  If we say that the app has to ask for quota
before accessing the FS, or that the user might have blocked FS access
[as if blocking cookies], it would be nicer to have a
FS-access-request function that calls back when access is granted [or
denied].

"Its owner FileSystem has a name of "localFS", and has both
caseSensitive and casePreserving set to true."

Are you saying that all browser-exposed filesystems must be
case-sensitive and case-preserving?  I do like the idea of enforcing
consistency of interface, so that apps that work anywhere work
everywhere.  I'd lean toward making them case-insensitive and
case-preserving, though, as I think that would be easier to emulate
[without race conditions and without obfuscating filenames] on
case-sensitive systems than the other way around.  Also, I think there
are a lot more case-insensitive systems out there, though that should
probably only be a small factor in the decision.

Should we expose case-sensitivity, or just require one or the other in
the interface?  Just passing the buck leads to non-portable apps!
Also, path and filename restrictions.

In 5:

You don't specify whether moveTo or copyTo will fail if they try to
overwrite an existing file.  I think it should be decided by an
optional argument "overwrite", defaulting to false.


In 6:

On Directory, file() and directory() always open for create.
We need:
  * a way to open an existing file for reading;
  * a way to look up an existing sub directory for reading or writing,
and to get a list of the Entries in a directory;
  * a way to test for the existence of a file or directory without
creating it [possibly just opening it for reading];
  * a way to append to a file [maybe just open+read and then open for
writing?  While UAs could be smart about moving the data around
through blob manipulations, it seems like it could be messy if there
are multiple threads using the FS at the same time.  Maybe if you open
for append you get a pre-populated WriteableBlob to which you can
append?].

We will likely also want:
  * a modification time attribute on FileEntry or Entry--very helpful
for letting apps know whether a local file might be out of date with
its server copy...clock skew renders this imperfect, but it may keep
you from needing *both* filesystem and database storage in simple
cases.
  * a size attribute on FileEntry--useful for apps to manage their own
quotas, buffering, and UI.

In 6.2:

All creations+deletions are nicely atomic--excellent.

You've got zip--should we have unzip?  Or is this something that
should be done in a JS library in both directions?  I don't follow the
Widgets stuff, and I'm assuming this is for them, but what's the use
case?

In 7:

This API doesn't look like it quite matches up with
FileReader/FileWriter yet, but since FileWriter's still forming, it's
hard to say.  I'm going to go ahead and suggest how I think they might
fit together.

The FileReader API takes in a File and gives you data.  A File refers
to a particular path on disk [or wherever], so it's a lot like an
Entry.  Let's have FileEntry implement file.

I wrote earlier
[http://lists.w3.org/Archives/Public/public-webapps/2009OctDec/1221.html]
about making FileReader and FileWriter more similar.  Here's what I
had in mind: Have FileWriter's constructor take a File as well, and
then when you want to write, just hand it a Blob or String to write
out.  Then FileEntry doesn't need a WriteableBlob attribute any more,
since it's a File, which is a Blob.  Nor does it need save().  New
Blobs can be constructed however you like, or can be gotten from the
filesystem.  To write to a file multile times, just create a new
FileWriter from your FileEntry, which remains valid as long as the
Document does.

We may want to split Arun's File into separate ReadableFile and
WriteableFile interfaces.  That way the file-open dialog can return
something that's very difficult for UAs to allow to be opened for
writing accidentally, and similarly with the file-save dialog.
FileEntry would implement both interfaces.

Thanks again!

     Eric

Received on Thursday, 10 December 2009 03:10:56 UTC