Re: URLs into IndexedDB databases

On Mon, Jul 15, 2013 at 1:57 AM, Jonas Sicking <jonas@sicking.cc> wrote:

> Hi All,
>
> One of the features that the sandboxed filesystem storage area brings
> over indexedDB is the "filesystem:" URL scheme. Or more precisely, the
> ability to load something from filesystem storage using a URL, without
> having to use the filesystem JS API at all.
>
> There is no technical reason we couldn't fix this, and make indexedDB
> support loads from it using URLs. It's just more complicated to do so.
> The reason is that the storage area in indexedDB is much more
> powerful, but also much more complicated.
>
> While in the filesystem storage, in order to identify a resource, you
> only have to define
> * Origin (valid origin)
> * Persistence ("temporary" or "persistent")
> * Path (valid filename)
>
> In indexedDB you have to define all of the following:
>
> * Origin (valid origin)
> * Database name (arbitrary string)
> * ObjectStore name (arbitrary string)
> * Index name, if any (arbitrary string or void)
> * Key (arbitrary indexedDB key)
> * Keypath, if any (string on the format "foo.bar.baz", or void)
> * Persistence ("temporary" or "persistent") (assuming we add support
> for this in v2).
>
> All of this is relatively straight forward to define. We could simply
> do something like:
>
>
> indexeddb:<origin>/<persistence>/<database>/<objectStore>/<index>/<key>/<keypath>
>
> Each piece above would have to be encoded such that it contains only
> valid, non-'/', URL characters. For things like database name and
> objectStore name this is easy by simply %-encoding characters.
>
> For the key this is significantly messier. We have to encode things
> like Dates and numbers, as well as Arrays. There's no shortage of ways
> of doing this, but there's no particularly clean way to do it either I
> think.
>

Yeah. (Broken record: It's a shame JSON doesn't support Infinities or
Dates. At least NaNs aren't an issue.)


>
> There's a couple of problems with this whole setup though. The first
> one is that it's going to be complex, and thus bug-prone for
> developers to generate these URLs themselves. However that's a pretty
> easy problem to solve. We can simply introduce a function like:
>
> url = indexedDB.getURL({ database: "mydb", objectStore: "...", key:
> [47, new Date(...)]});
>

Nice! I'm 99% certain but want to clarify: the URLs output by thus method
would be specified, meaningful, and not rely on browser state, i.e. they
are not UUIDs opaquely referencing some browser object that contains the
"query", but URLs that could be generated server-side assuming a conforming
implementation of the method. Correct?


>
> The second problem is that this scheme doesn't really support relative
> URLs in any meaningful way. If you have two files stored in two
> different keys in the same objectStore, linking from one file to
> another means having to figure out how to encode the key. We can add
> API here too, and possibly the URL spec will grow API to calculate the
> relative URL needed to link between two specified URLs.
>
> However in any case it's unlikely that two such resources linking to
> each other will have "normal" relative URLs. That means that if the
> two resources were downloaded from somewhere where they had been used
> as http resources, that they would have the correct relative URLs.
>
> Likewise, it's unlikely that if the resources were generated on the
> client, that they could be uploaded to a server and then published
> through http, without having to adjust relative URLs.
>
> A third problem here is that it seems unfortunate to generate a new
> URL scheme for each storage format we have. Google's filesystem API
> introduces the "filesystem" scheme. If indexedDB introduces
> "indexeddb" we might end up with a few schemes for accessing local
> storage areas.
>
> One option for solving this might be to instead use something like:
>
> localdata://<origin>/indexeddb/<persistence>/...
> and
> localdata://<origin>/filesystem/<persistence>/...
>

I'm unsure if this is beneficial. I hope we're not planning to frequently
add more storage formats to the platform. The indirection seems to add
complexity - in the abstract, now you're asking people to read three specs
and defining localdata:// as a new extensibility point in the platform,
since we don't want to constrain <persistence> across future storage types.


> Another advantage with this is that we could more cheaply introduce
> simplified ways of encoding indexedDB URLs. I.e. we could in addition
> to the above scheme support something like
>
> localdata://<origin>/simpleindexeddb/<database>/<objectStore>/<key>
>
> Since the key is at the end, we could be more flexible in how we
> encode it. Especially if we limit keys here to string keys. Then we
> wouldn't need to encode the key at all. This would actually
> additionally help solve the relative-url problem described above.
>
> However I'm not sure if it's worth introducing simplified indexedDB URLs
> yet.
>

Not doing localdata: doesn't preclude simpleindexeddb: from being added. I
agree it makes it less appealing.


>
> What do people think. Does this seems like a sensible approach?
>
> / Jonas
>
>
I'll try and point some of our advanced IDB users at this thread for
additional feedback.