Re: CSP reports: `script-sample`

On 19 Oct 2016, at 15:34, Krzysztof Kotowicz <kkotowicz@gmail.com> wrote:
> URLs should not contain sensitive data, precisely because they are often relayed to third parties


Hi Krzysztof,

While I agree, the classic example is a password reset.

As in, a link is sent to the user via email, and that link contains a sensitive token that allows someone to change the account password.

That said, the token should not last long to deduce the risk of it being exposed to 3rd parties (e.g. expiring it after use, or a certain period of time).

Craig






> On 19 Oct 2016, at 15:34, Krzysztof Kotowicz <kkotowicz@gmail.com> wrote:
> 
> 
> 
> 2016-10-19 2:16 GMT+02:00 Artur Janc <aaj@google.com <mailto:aaj@google.com>>:
> On Tue, Oct 18, 2016 at 10:05 AM, Mike West <mkwst@google.com <mailto:mkwst@google.com>> wrote:
> On Tue, Oct 18, 2016 at 1:03 AM, Artur Janc <aaj@google.com <mailto:aaj@google.com>> wrote:
> On Mon, Oct 17, 2016 at 7:15 PM, Devdatta Akhawe <dev.akhawe@gmail.com <mailto:dev.akhawe@gmail.com>> wrote:
> Hey
> 
> In the case of a third-party script having an error, what are example leaks you are worried about?
> 
> The same kinds of issues that lead us to sanitize script errors for things loaded as CORS cross-origin scripts: https://html.spec.whatwg.org/#muted-errors <https://html.spec.whatwg.org/#muted-errors>. If the resource hasn't opted-in to being same-origin with you, script errors leak data you wouldn't otherwise have access to.
>  
> Thanks for the summary, Mike! It's a good overview of the issue, but I'd like to expand on the reasoning for why including the prefix of an inline script doesn't sound particularly scary to me.
> 
> Thanks for fleshing out the counterpoints, Artur!
>  
> Basically, in order for this to be a concern, all of the following conditions need to be met:
> 
> 1. The application has to use untrusted report collection infrastructure. If that is the case, the application is already leaking sensitive data from page/referrer URLs to its collector.
> 
> "trusted" to receive URLs doesn't seem to directly equate to "trusted" to store sensitive data. If you're sure that you don't have sensitive data on your pages, great. But you were also presumably "sure" that you didn't have inline script on your pages, right? :)
> 
> Keep in mind that URLs are sensitive data for most applications and they are currently being sent in violation reports.
> 
> URLs should not contain sensitive data, precisely because they are often relayed to third parties in e.g. referrer. If they do, that's usually considered a vulnerability by e.g. OWASP, especially if the URL is capability bearing (IDOR <https://www.owasp.org/index.php/Top_10_2013-A4-Insecure_Direct_Object_References> is even in OWASP Top 10). I agree that some applications disclose sensitive information in URLs, but that's should not be the majority of them. I think that for a lot of applications URLs reported through e.g. CSP violation reports are still subject to regular access control, whereas we just don't know yet if sensitive tokens are not included in script samples. It's likely some IDs are present in the inline scripts <a href=# oncllick=a.select(123)>
>  
> I'm having a difficult time imagining a case where an application is okay with disclosing their URLs to a third-party for the purpose of debugging violation reports, and is not okay with disclosing script prefixes for the same purpose, given that:
> 1) Almost all applications have sensitive data in URLs, compared to a certainly real, but less specific risk of having inline scripts with sensitive data in its prefix, assuming it's limited to a reasonable length.
> 
> Citation needed (for the "almost all" claim). I agree the risk of leaking sensitive date might be mitigated by adding a reasonable length limit.
>  
> 2) URLs are disclosed much more frequently than script samples would be, because they are sent with every report (not just "inline" script-src violations). In the `referrer` field, the UA is also sending a URL of another, unrelated page, increasing the likelihood that sensitive data will appear in the report.
> 
> Which is why it's a best practice not to have sensitive data in URLs, but instead e.g. using cookies or POST parameters to transfer them. 
>  
> 3) There is no sanitization of URL parameters in violation reports, compared to the prefixing logic we're considering for script samples.
>   
> In fact, I'd be much more worried about URLs than script prefixes, because URLs leak on *any* violation (not just for script-src) and URLs frequently contain PII or authorization/capability-bearing tokens e.g for password reset functionality.
> 
> We've talked a bit about URL leakage in https://github.com/w3c/webappsec-csp/issues/111 <https://github.com/w3c/webappsec-csp/issues/111>. I recall that Emily was reluctant to apply referrer policy to the page's URL vis a vis the reporting endpoint, but I still think it might make sense.
>  
> 2. The application needs to have a script which includes sensitive user data somewhere in the first N characters. FWIW in our small-scale analysis of a few hundred thousand reports we saw ~300 inline script samples sent by Firefox (with N=40) and haven't found sensitive tokens in any of the snippets.
> 
> Yup. I'm reluctant to draw too many conclusions from that data, given the pretty homogeneous character of the sites we're currently applying CSP to at Google, but I agree with your characterization of the data.
> 
> Scott might have more data from a wider sampling of sites, written by a wider variety of engineering teams (though it's not clear that the terms of that site would allow any analysis of the data).
> 
> I completely agree, this data is just what we had readily available -- we can certainly do a much larger analysis of script prefixes based on the search index. That said, if we're worried about the script-sample approach, perhaps not seeing any sensitive data in the first dataset we looked at could be a signal that it's worth pursuing further.
> 
> 3. The offending script needs to cause a CSP violation, i.e. not have a valid nonce, meaning that the application is likely broken if the policy is in enforcing mode.
> 
> 1. Report mode exists.
> 
> 2. Embedded enforcement might make it more likely that XSS on a site could cause policy to be inadvertantly applied to itself or its dependencies. We talked about this briefly last week, and I filed https://github.com/w3c/webappsec-csp/issues/126 <https://github.com/w3c/webappsec-csp/issues/126> to ponder it. :)
> 
> Since CSPs applied by embedded enforcement serve a very different purpose than current policies (they don't try to mitigate script injection), it would very likely be okay to just not include script-sample data for such policies. Also, embedded enforcement is still pretty far off, and the reporting problem is an issue for pretty much every site currently gathering violation reports; we should probably weigh the value of fixing CSP reporting accordingly.
>  
> As a security engineer, I would consider #1 to be the real security boundary -- a developer should use a CSP collector she trusts because otherwise, even without script-sample, reports contain data that can compromise the application.
> 
> That sounds like an argument for reducing the amount of data in reports, not for increasing it. I think it's somewhat rational to believe that reporting endpoints are going to have longer retention times and laxer retention policies than application databases. Data leaking from the latter into the former seems like a real risk. I agree that the URL itself already presents risks, but I don't understand how that's a justification for accepting more risk.
> 
> It is an argument for using trusted infrastructure when building your application ;-) Developers are already accustomed to deciding whether to place trust in various components of their apps, whether it's the hosting platform and OS, server-side modules and libraries, or JS widgets and other embedded resources. A CSP violation endpoint is currently a security-critical part of an application because it receives URLs; people who don't trust their collection infrastructure already have insecure applications and adding script-sample to reports does little to change this. (Note that this wouldn't hold for applications which have nothing sensitive in URLs and embed sensitive data at the beginning of inline scripts, but this doesn't seem like a common pattern.)
> 
> Basically, the reluctance to include relevant debugging information in the violation report seems to be somewhat of a misplaced concern to me, because it ignores the trust relationship the application owner must already have with their report collection endpoint.
> 
> Perhaps it's pertinent to take a step back and think about the reason to have reporting functionality in CSP in the first place -- after all, the mechanism could certainly work only via throwing SecurityPolicyViolation events and requiring developers to write their own logging code. The fact that this capability exists in UAs, and is not restricted to sending reports to the same origin or same "base domain" (contrary to the original proposals, e.g. in http://research.sidstamm.com/papers/csp-www2010.pdf <http://research.sidstamm.com/papers/csp-www2010.pdf>) indicates that CSP wants to be flexible and give developers ultimate control over the reporting functionality. Given this design choice, it seems okay to trust the developer to pick the right report URI for their application and include useful debugging data if the developer wants it; in a way, the status quo is the worst of both worlds, because it already requires the developer to fully trust the collector, but doesn't give her enough useful data to track down causes of violations.
> 
> In case it helps: Lukas ran a quick analysis of the report-uri values we've seen in the wild, and e.g. for the domains with CSP in Alexa 100,000 we see the following:
> - 49% don't set a report-uri 
> - 29% have a report-uri pointing to a relative path (/foo)
> - 10% have a report-uri pointing to the same origin, with another 1% using a sibling subdomain (foo.example.org <http://foo.example.org/> reporting to csp.example.org <http://csp.example.org/>)
> 
> Out of the remaining ~10% which send violations to external URLs, about half point to report-uri.io <http://report-uri.io/> and a couple of other logging services, and the rest seems to use another domain owned by the same person/organization, e.g. vine.co <http://vine.co/> sends reports to twitter.com <http://twitter.com/>. The data for all domains in our set isn't substantially different (66% without report-uri; 24% reporting to own domain; 10% externally). This data doesn't include all the Google ccTLDs and a couple of other big providers, and I'm sure it's missing some other domains, e.g. ones with CSP in parts of the site requiring authentication, but AFAIK it shouldn't have a systematic bias otherwise. 
> 
> I can easily imagine scripts that violate conditions #2 and #3, but at the same time we have not seen many examples of such scripts so far, nor have people complained about the script-sample data already being included by Firefox (AFAIK). 
> 
> People are generally unlikely to complain about getting more data, especially when the data's helpful and valuable. That can justify pretty much anything, though: lots of people think CORS is pretty restrictive, for instance, and probably wouldn't be sad if we relaxed it in various ways.
>  
> Overall, I don't see the gathering of script samples as qualitatively different to the collection of URLs. However, if we are indeed particularly worried about script snippets, we could make this opt-in and enable the functionality only in the presence of a new keyword (report-uri /foo 'report-script-samples') and add warnings in the spec to explain the pitfalls. This way even if I'm wrong about all of the above we would not expose any data from existing applications.
> 
> I suspect that such an option would simply be copy-pased into new policies, but yes, it seems like a reasonable approach.
>  
> For some background about why we're even talking about this: currently violation reports are all but useless for both debugging and detection of the exploitation of XSS due to the noise generated by browser extensions.
> 
> I agree that this is a problem that we should solve. One way of solving it is to add data to the reports. Another is to invest more in cleaning up the reports that you get so that there's less noise. I wish browser vendors (including Chrome) spent more time on the latter, as we're actively harming users by not doing so.
> 
> Yes, fixing the blocking and reporting of extension-injected scripts would certainly help (although note that "less noise" likely isn't sufficient, it really has to be zero noise), but IIRC prior discussions we've had about the topic indicated that this is an almost intractable problem, so it would be great to find alternative solutions.
> 
> The script sample approach also has several important advantages because even without extension-related false positives, developers would have very little information about the actual cause of inline script violations (which are the majority of possible CSP problems in nonce-based policies). Sending some of the script text not only makes it possible to discard all spurious reports, but also gives the developer the crucial bit of data to find and fix actual site errors; it seems like a workable solution to the current reporting problems faced by many sites.
> 
> Cheers,
> -Artur
> 
> 
> 
> -- 
> Best regards,
> Krzysztof Kotowicz