Re: Request to move app: URI to FPWD

Hi Giri!  
Wow, thanks for the detailed review! Took me a little while to get through it, but you raised lots of good issues and I filed a whole bunch of bugs thanks to it! As always, responses inline…  

On Friday, April 19, 2013 at 11:41 PM, Mandyam, Giridhar wrote:

> Comments:
>  
> Section 1
> -----------
> I realize this is non-normative, but there are IMO misleading statements in this section and I think it would be better if some (maybe all) of this material is removed entirely. I think the abstract as written covers the justification for this feature pretty well regarding the desire to leverage XHR for resource retrieval from a package. Rather than focusing on alleged shortcomings of the File URL, it might be good to collapse this section into an overview of requirements to make file retrieval from a package via XHR possible. If you are open to it, I can suggest replacement text.

Sure, that would be great! I'm also working on a Use Cases and Requirements section for the spec. I can have first crack at it, and then we  
   
>  
> a. "Security/privacy issues: on Unix systems, naive implementations expose the user name as part of the path, as well as the full path on the file system to where a file is residing (e.g., "/Users/username/app/index.html"). In addition, the file URL scheme potentially opens up the ability for an attacker to address any file on the file system."
>  
> Knowing the address of the file is different from being able to access the file. Certainly root access would override any file system sandboxing, but I believe it is up to the implementation to protect from unauthorized file access outside of root login.

While it is true that knowing the address of the file is different from being able to access the file, I've seen some pretty funny implementations of widget runtimes using file:// … which allowed anyone to read just about anything on the user's HD (so they go hand in hand a lot of the time). It was a strong motivator for this spec.  
> b. " Undefined security model: The HTML specification lacks a security model definition for when the file URL scheme is used as a document's address, meaning that different user agents behave inconsistently when content is loaded using the file URL scheme (e.g., same origin policy doesn't apply, local storage areas of [WebStorage] don't work as expected if at all, and so on)."
>  
> Agreed, but shouldn't this be addressed eventually in Section 6 in the runtime spec (see http://www.w3.org/TR/2013/WD-runtime-20130321/#data-isolation)?
Sure, but this spec is orthogonal to that spec.   
>  
> c. " An app: URI is a safer alternative to the file URL scheme, as it does not allow addressing outside a sand-boxed environment. Additionally, it does not expose the location of a file on user's local device, nor their user name, as in the case with some Unix-based implementations."


Make an Apple Dashboard Widget, and get it to show you in javascript its "window.location.href". Whoops! :)  
  
> I simply do not understand this. Let us go back to one of the docs cited as examples in the charter - the OMTP Application Security Framework (which is cited as "The BONDI App Security Framework" in the charter).  
>  
> First, Section 2.1.16 of this doc cites examples of when file access outside of the sandbox is necessary. I assume we'd want privileged applications that can bypass sandboxing for WebOS devices too.
>  
> Second, some of the AEE's cited by the OMTP (e.g. BREW, Symbian) have implemented file system sandboxing for 3rd-party applications while leveraging file URL. These environments allow the application to query available roots, and as a result the app can construct the File URL for use in the appropriate file management API. The file system implementation in this way implements the sandbox (e.g. iFile and FileConnection for BREW and J2ME respectively).
>  
> For BREW in particular, I have yet to hear of anyone successfully attacking the file system through a downloaded packaged app. Regarding J2ME (which many in the industry feel has a stricter sandboxing model than BREW), Symantec has a study online (http://www.symantec.com/avcenter/reference/attack.surface.analysis.of.blackberry.devices.pdf) for Blackberry's version of J2ME where the conclusion is " traditional file infector viruses are not feasible for the BlackBerry, short of the discovery of a new vulnerability" (p. 16).  
>  
> Sideload access is a different matter, but I don't think App URI is going to defeat someone who has sideload access to the entire fs.
All the above aside (and no matter how secure they are), you don't have HTTP semantics with any of the above, so whatever file system schemes they use will still break web apps.  

Also, app: layers itself onto of all the sf's above and and adds the HTTP response semantics - so it's all good.   
>  
> d. "An app: URI provides a means to retrieve a file from within a package using similar semantics to performing a GET request over [HTTP]."
>  
> I do not believe you need app:URI to do this. I am going to assume that there is a critical developer need to retrieve resources using XHR as opposed to direct file access (although I am not convinced about this either).
I guess the only way to convince you would be to ask you to build an app using something like jQuery mobile without XHR… and feel the pain :)   
> An on-device proxy can be used to accomplish retrieval of resources as well. The app can send an XHR request to the proxy, the proxy can retrieve files directly, and serve them back. In the early days of the mobile web (when connectivity was intermittent at best), this was one of the solutions used to provide a continuous web experience.

But why would you do that when you can send the file back with a HTTP response? Sounds like a whole bunch of work and middle ware to achieve the same thing that this spec does transparently?  

> Section 2
> -----------
>  
> a. We concluded last week during the review of the Runtime and Security spec that resources could be downloaded (post-install) as well (or at least that was my conclusion as to why " All resources that are commonly used by the application SHOULD be available in the container" from http://www.w3.org/TR/2013/WD-runtime-20130321/#packaged-applications is a SHOULD requirement rather than a MUST) . I think it would be appropriate to leverage FileWriter or FileHandle and provide an example of an app storing a resource on the fs, and retrieving it afterwards. I assume app:URI is still applicable.
Sure, app:URI is still applicable. Of course, you could store it in IndexedDB or maybe even local storage. It really depends on what you are trying to achieve.

Not sure if it helps, but the File API uses synthesized HTTP responses when dereferencing:
  
http://www.w3.org/TR/FileAPI/#processingModel


> Section 5
> -----------
>  
> a. Can I conclude that any locally-stored resources for hosted apps will not be covered by app:URI?  
Yes.   
> For example, locally cached copies of hosted resources will not be accessible from an application, therefore they do not need app:URI.

I believe that is correct. app:URI is only for addressing resources in the zip file - other protocols and schemes can be used for other resources as appropriate.   
>  
>  
> Section 6
> -----------
>  
> a. The mention of UUID in the text preceding Section 6.1 implies that it is required, when Sec. 6.2 makes it clear that it is recommended.
Filed a bug:  
https://github.com/sysapps/app-uri/issues/5
  
> Moreover, (obviously) the ABNF spec does not place any requirements on use of unique identifiers. I would suggest changing " And, and uuid is defined in [UUID]" to "See Section 6.2 for more information on the use of UUID in construction of the authority field."

Good suggestion, filed bug:  
https://github.com/sysapps/app-uri/issues/6
>  
>  
> b. Section 6.1: "The identifier represented by the authority is bound to an instance of an application for the life of that application instance: that is, until that instance is destroyed (e.g., the application is uninstalled from an end-user's device)."
>  
> Does this prevent continuity of the authority for a given application after uninstall? For instance, if the app is uninstalled due to lack of device memory but reinstalled shortly afterwards when memory is freed up, can the authority be retained?
Right now, probably not - but this depends on the use cases. It is assumed that the application's data will be backed up in the cloud and sync'ed, to allow the data to be used across multiple devices simultaneously.  
> Or should the authority be regenerated after reinstall?

For privacy reasons, I think it should generate a new ID. Another issue is that the UUID can be used to track people:  
https://github.com/sysapps/app-uri/issues/7
> c. Section 6.1: " The reason for having a unique authority is, amongst other things, to prevent multiple instances from overriding each other's data."
>  
> I don't think the authority field uniqueness is the mechanism that will prevent this.  
I know what you mean, but on some systems it will be. Having said that, of course implementations will need to block access to all other app: URIs except their own.  
> Rather I believe that runtime security mechanism would control the roots available to the application in order to prevent this (see Jonas's post https://hacks.mozilla.org/2012/07/why-no-filesystem-api-in-firefox/).  

Yes, of course. I read Jonas' post (thanks for this, btw - it was a good read!), but I'm not sure what part in particular you are referring to? My take-away from the post was "use IndexedBD", which is bound to an origin (hence bound to an app: URI I would guess):  

"Each origin has an associated set of databases. A database comprises one or more object stores which hold the data stored in the database."

The concept of "origin" is used throughout the IndexedDB specification to perform various checks.  
> Moreover, depending on how you define an instance's data, there will be other mechanisms for modifying data created by another application instance when both have access to common directories (see https://wiki.mozilla.org/WebAPI/DeviceStorageAPI).

Right, but Device Storage API is orthogonal.  
  
> No point in painting yourself into a corner. Since the intent of the above sentence appears to be informative, I would remove it altogether.

I agree. But I would say that it's application specific. Hence, the runtime spec needs to define the life of the authority (e.g., if it's reconstructed when private data is cleared, when the app is reinstalled, etc.).  

Filed bug:
https://github.com/sysapps/app-uri/issues/8

> d. Section 6.3: I really do not understand how query can ever be used in the context of retrieval of static resources. Seems wasteful to define a feature that has no use to developers.
It just says it has no effect, so it's not harmful. People have asked for clarification about the role of query and fragments in the scheme, which is why it's there.  

The section should remain normative nonetheless, as the text provides the basis for a set of conformance tests. If anything, I should throw a MUST somewhere in there to make sure some tests get made for this. That is, to make sure that two user agents both retrieve the same file when:  

app://c1..66/example.gif?hello
app://c1..66/example.gif?hello=foo&bar=baz

app://c1..66/example.gif?hello#hi-there

Instead of, for instance, throwing an error or not retrieving the file.  
  
Filed a bug:
https://github.com/sysapps/app-uri/issues/9
  
> e. Section 6.3: Similar comment for frag. You certainly can't use XHR for doc frag retrieval, at least that's the way I interpret Step 7 ("Drop <fragment> from url") in  

Yes, absolutely.   
> http://www.w3.org/TR/XMLHttpRequest2/#the-open-method (though there are proprietary web services that propose using URL frags to retrieve parts of an XML doc).

They could do that, sure; if their MIME type allows it.   
> f. Section 6.4: I believe this section covers new behavior required of the UA and could not have been anticipated by a casual reading of the charter.

AFAIK, I thought it was a pretty well-known issue.  
> Can you point me to any one of the references cited in the charter that discusses the dereferencing requirements and required XHR response? I had trouble finding one.

So, any W3C widget engine that's been using widget since September 2011?
http://www.w3.org/TR/2011/WD-widgets-uri-20110927/#dereferencing-and-retrieval-of-files-from-a-container

That includes, all WAC runtimes:
http://specs.wacapps.net/core/index.html#web-standards

"In terms of Web standards, a widget runtime( WRT ) is an implementation of the conformance requirements of this specification, which also supports the following specifications: … [Widgets-URI]". I edited that spec, so I can confidently assert it was a requirement ;)  

Tizen 1.0 also supported the Widget URI scheme.  

> g. Section 6.4: Where does the de-referencing occur? Is it in the UA's XHR implementation? If so, why is this not covered in http://www.w3.org/TR/XMLHttpRequest2/#xmlhttprequest-base-url?
Don't know. We will need to speak to the editor of that spec if there are issues with it.  
> h. Section 6.4: Are request headers ignored?

For this version - they would be, yes. I've filed a bug to clarify that:
https://github.com/sysapps/app-uri/issues/10  
> i. Section 6.4: Section 6.3 mentions that " when dereferencing, the query and fragment components don't play any part in locating a file inside of package". Yet there is no specific text in this section about the UA ignoring the query and fragment fields in the URI. Why?

Because, as you pointed out earlier, XHR drops the fragment. Also, the query component (which would just become a HTTP header) hasn't come up as an interoperability issue.  

> j. Section 6.4: Use of MIME sniffing to set Content-type header. I assume that applications should be able to use File API to read the same files. Is MIME sniffing currently used to set File.type? The spec lists it as an informative reference, but does not seem to have a normative requirement to use it. However, app:URI makes this a normative requirement. If a valid File API implementation does not uses a different MIME sniffing methodology from the one the one used for app:URI, there is a possibility that a file retrieved via XHR could have a different content type from a file retrieved via FileReader.
Sure, we should file a bug on FileReader.   
> Also, some resources are specifically called out in the manifest (icons being the prime example). Can the MIME type indicated by the manifest in such cases be used in place of MIME sniffing?

It could be, but what's the use case for that? SNIFF handles the 80%+ use case.  
> Finally, should there be mention of a requirement for the Blob.type to match the Content-type header if the XHR response.responseType (see https://dvcs.w3.org/hg/xhr/raw-file/tip/Overview.html#dom-xmlhttprequest-response) is "Blob"?

No, IMO. Because no registered MIME type maps to blob, AFAIK.  
>  
> General
> ----------
>  
> a. How does app:URI extend for multiple file system roots (if it does)? For instance, if app resources are stored both on the device fs and removeable memory (e.g. SD Card), does this need to be reflected in app:URI? If so, how does dereferencing occur?
No, it doesn't. If the underlying implementation chooses to span data/files across media, retrieval is still done transparently. I.e., the app: URI is supposed to abstract all that stuff away from the developer.   

Whoa! made it out in one piece! Achievement Unlocked + 100 XP gained :)

Thanks again for the detailed review! Remember, you can track and assist with closing the bugs spawned from this email by "watching" the repo on GitHub.  
https://github.com/sysapps/app-uri/issues   

Kind regards,
Marcos

Received on Monday, 22 April 2013 18:21:17 UTC