Re: HSTS, mixed content, and priming

On Mon, Aug 24, 2015 at 1:31 PM, Eric Mill <eric@konklone.com> wrote:

>
>
> On Mon, Aug 24, 2015 at 12:13 PM, Brad Hill <hillbrad@gmail.com> wrote:
>
>> It might provide the greatest compatibility to specify that the probing
>> request always fetches the server root rather than the specific resource.
>>
>
> I agree, and I think that's what Richard's original proposal says as well.
>

Yeah, there are a few different reasonable options.  My preference would be
to use "/" or something under ".well-known"



> I do generally agree that removing mixed content warnings for successful
>> HSTS upgrades is a good thing.
>>
>
> I'll add that this proposed approach would provide an additional powerful
> incentive for major third-party resource providers (e.g. code.jquery.com,
> ajax.googleapis.com, etc.) to turn on the HSTS flag, knowing that that
> action alone will resolve mixed content problems for all of their
> integrating sites.
>

Thanks.  This is a major point that I totally forgot to inculde.



> I assume the proposal would not apply to CORS requests, where redirects
> (even "internal" redirects triggered by HSTS policy) affect Safari and some
> older clients in more problematic ways than includes through <link> or
> <script> tags.
>
> https://github.com/konklone/cdns-to-https/#results
>

The interaction, if any, with CORS, would be something we would need to
think through.  There might also be some lessons to learn from CORS about
how to build the priming query.

--Richard



>
>
> -- Eric
>
>
>> -Brad
>>
>> On Mon, Aug 24, 2015 at 8:05 AM Richard Barnes <rbarnes@mozilla.com>
>> wrote:
>>
>>> Hey all,
>>>
>>> This is a circle we've been around a few times, but I wanted to see if
>>> we might be able to break out of it this time, using a new-ish idea.  The
>>> goal of this message is to lay out the idea and an initial discussion of
>>> the trade-offs, to see if this is an idea worth pursuing.
>>>
>>> tl;dr: If we add priming requests for HSTS, we can allow HSTS-upgraded
>>> reqeusts from HTTPS pages, and avoid the need for scheme changes.
>>>
>>>
>>> # Background
>>>
>>> In order to maximize the amount of the web that is protected with HTTPS,
>>> we want to fetch resources using HTTPS even if the URL uses the "http:"
>>> scheme -- but only when the "http:" and "https:" versions of the URL are
>>> equivalent.
>>>
>>> Currently, we have two ways of determining that HTTP and HTTPS resources
>>> are equivalent:
>>>
>>>   1. HSTS - equivalence asserted by the resource owner
>>>   2. upgrade-insecure-requests - equivalence asserted by linking site
>>>
>>> Now we have a funny asymmetry, though: Requests that are upgrade through
>>> u-i-r are not mixed content, but those upgraded through HSTS are.  This
>>> seems silly, given that they both result in content being loaded over
>>> HTTPS.  In fact, HSTS is a better signal than u-i-r of when the upgrade
>>> should be done.  How does the linking site know that the HTTP and HTTPS
>>> URLs reference the same resource?
>>>
>>> It seems like if we could get to the point where we could give
>>> HSTS-upgraded resource loads the same mixed-content treatment as other
>>> scheme-upgraded loads, it could reduce the friction of moving to HTTPS.
>>>
>>>
>>> # HSTS Priming
>>>
>>> The proposal here has two major parts:
>>>
>>> 1. Discover HSTS support with "priming requests":
>>>   * When the browser encounters http://example.com/foo/bar.js on an
>>> HTTPS page...
>>>   * And the example.com is not an HSTS host...
>>>   * Send a HEAD request https://example.com/ with no cookies, etc.
>>>   * See if the query returns HSTS headers
>>>   * If so, the browser loads https://example.com/foo/bar.js
>>>   * ... and don't consider it mixed content
>>> 2. Do not treat HSTS-upgraded requests as mixed content
>>>
>>> This is basically a CORS preflight, but looking for HSTS instead of
>>> ACAO.  In either case, you're taking a request that would not be allowed by
>>> the default policy and checking to see whether it can be done in a way that
>>> is allowed.  (Yes, you would have to do both before an upgraded CORS
>>> request.)
>>>
>>> In past discussions of allowing HSTS-upgraded loads, there have been two
>>> main objections:
>>>
>>>   * Indeterminacy: Whether the upgrade happens depends on whether the
>>> browser has encountered the HSTS header in earlier browsing.
>>>   * Silently upgrading requests hides potential breakage
>>>
>>> I think that adding priming addresses both of these concerns.  Priming
>>> obviously removes indeterminacy, since if the browser doesn't know the HSTS
>>> state of a site, it goes and checks.  As far as hiding breakage, well,
>>> there's no breakage to hide for browsers that implement priming, and as
>>> always, if you want to support older browsers, you need to test with them.
>>>
>>>
>>> # Some Issues
>>>
>>> ## What value does priming add?
>>>
>>> As mentioned above, the primary value is to remove the indeterminacy
>>> around HSTS upgrades, so that it's safe to treat HSTS ugprades as not mixed
>>> content.
>>>
>>> Allowing HSTS loads also addresses some of the inherent deficiencies of
>>> upgrade-insecure-requests:
>>>
>>> * As noted above, it's difficult for the linking site to actually
>>> determine whether the HTTP and HTTPS URLs actually reference the same
>>> content (except by looking at the linked site's HSTS headers).  So there's
>>> a risk of breakage if the linking site turns on u-i-r and the resource
>>> owner does not maintain the equivalence.
>>>
>>> * Priming provides a softer upgrade path than u-i-r.  Mixed content that
>>> would not be blocked can still load, and will smoothly upgrade to HTTPS as
>>> the resource server is able.
>>>
>>> So relative to u-i-r, this reduces uncertainty for site operators, and
>>> gets more HTTPS faster (since it's a partial ugprade).  It seems like these
>>> two are complementary in much the same way that HTTPS and HSTS are -- you
>>> can turn on HTTPS for some parts of your site, then turn on HSTS to lock it
>>> in.  Relying on priming to upgrade what can be upgraded of your site on day
>>> 0, then once you're sure that all your sub-resources can upgrade properly,
>>> turn on u-i-r.
>>>
>>>
>>> ## Is this something developers will understand?
>>>
>>> Developers have gotten used to these sorts of dynamic changes to
>>> resource loads before.  CORS obviously comes to mind, as does IPv4 / IPv6
>>> selection.
>>>
>>>
>>> ## Is HSTS priming an expensive hack to paper over a temporary problem?
>>>
>>> In terms of "expense": It's worth noting that HSTS priming would only be
>>> done for potentially mixed-content requests, in cases where the HSTS state
>>> of the remote host is unknown.  Current Firefox telemetry indicates that
>>> around 2/25% of page loads have mixed content, which places an upper bound
>>> on the number of additional queries.  If you load 10 pages, each of which
>>> has 100 links to the same insecure host, you still only get one priming
>>> query.
>>>
>>> In terms of "hack": Any solution for upgrading things opportunistically
>>> is going to look messy, since you need a way to probe for when you can
>>> opportunistically upgrade.  This version seems minimally messy, since (1)
>>> it only probes when needed, and (2) the probe relies on existing technology
>>> (HSTS) for indicating when the upgrade is possible.
>>>
>>> In terms of "temporary": The cost scales as the need.  As more of the
>>> web is labeled with "https:" URIs, and as there's preloaded HSTS, there
>>> will be no more need for priming queries, and they will not be sent.  So
>>> the "http: and unknown HSTS" problem might be temporary, but we can see in
>>> telemetry when it starts to disappear, and remove the feature when it's not
>>> needed.  I expect the "http: links" problem to stay around longer, possibly
>>> indefinitely, but that doesn't require priming, just allowing HSTS upgrades.
>>>
>>>
>>> ## Is there privacy risk from the priming request?
>>>
>>> The priming request must be HTTPS -- it's looking for HSTS, and HSTS can
>>> only be sent over HTTPS.  So to the network, it only leaks the hostname
>>> that the browser is considering connecting to.  To the website, we can
>>> strip context (cookies, referer, etc.), so all the website learns is that a
>>> given browser/IP is attempting an upgrade.
>>>
>>
>
>
> --
> konklone.com | @konklone <https://twitter.com/konklone>
>