Re: FileAPI use case: making an image downloading app

On Fri, Dec 17, 2010 at 4:16 PM, Charles Pritchard <chuck@jumis.com> wrote:

> We're actively developing such functionality.
>
> The limit per directory is for the sake of the os file system. If you want
> to create a data store, use indexedDB or use standard file system practices
> (create a subdirectory tree).
>

I think you're missing the point. If I have a folder with 6000 files on some
server and I want to mirror that through the FileAPI to the user's local
machine I'm screwed. I can't **mirror** it if it's not a mirror.



>
> Disk space availability (quota) is an issue no matter what happens. When
> downloading 1000 images, you'll still only be doing so x at a time.
>

I don't see the point you're trying to make here. I don't know the size of
the images before hand. Many internet APIs make getting the sizes
prohibitively expensive. One REST call per file. So before I can download a
single file I'd have to issue 1000 REST XHRs to find the sizes of the files
(waiting the several minutes for that to complete) before I can ask the user
for the space needed.  That's not a good user experience. If on the other
hand the user can give me permission for unlimited space then I can just
start downloading the files without having to find out their ultimate size.

I suppose I can just request 1 terabyte up front. Or query how much space is
free and ask for all of it.



>
> This issues are inherent in the design of any application of scale. At this
> point, the file system API does work for the use case you're describing.
>
> It'd be nice to see Blob storage better integrated with Web Storage Apis.
> Ian has already spoken to this, but no followers yet (afaik).
>
> -Charles
>
>
>
> On Dec 17, 2010, at 3:34 PM, "Gregg Tavares (wrk)" <gman@google.com>
> wrote:
>
> > Sorry if this has been covered before.
> >
> > I've been wanting to write an app to download images from photo sites and
> I'm wondering if this use case has been considered for the FileAPI wrt
> Directories and System.
> >
> > If I understand the current spec it seems like their are some possible
> issues.
> >
> > #1) The spec says there is a 5000 file limit or directory.
> >
> > #2) The spec requires an app to specify how much storage it needs
> >
> > I understand the desire for the limits.  What about an app being able to
> request unlimited storage and unlimited files? The UA can decide how to
> present something to the user to grant permission if they want.
> >
> > Arguments against leaving it as is:
> >
> > The 5000 file limit seems arbitrary. Any app that hits that limit will
> probably require serious re-engineering to work around it. It will not only
> have to some how describe a mapping between files on a server that may not
> have that limit, it also has the issue the user might have something
> organized that way and will require the user to re-organize. I realize that
> 5000 is a large number. I'm sure the author of csh though 1700 entires in a
> glob was a reasonable limit as well. We all know how well that turned out
> :-(   It's easy to imagine a video editing app that edits and composites
> still images. If there are a few layers and 1 image per layer it could
> easily add up to more than 5000 files in a single folder.
> >
> > The size limit also has issues. For many apps the size limit will be no
> problem but for others....  Example: You make ray tracing program, it traces
> each frame and saves the files to disc. When it runs out of room, what is
> its options? (1) fail. (2) request a file system of a larger size. The
> problem is (2) will require user input. Imagine you start your render before
> you leave work expecting it to finish by the time you get in only to find
> that 2 hours after you left the UA popped up a confirmation "this app
> requests more space"
> >
> > The same would be true for an image downloading app. You start the app to
> download a 1000 images, not knowing their sizes before hand. You start it
> and leave. Half way through the UA puts up a conformation asking for more
> space.
> >
> > Have these use cases already come up and been rejected?
> >
> >
> >
>