Re: MACing HTTP requests/responses (Re: Content-Integrity header)

I have lost track of which tendril of this thread has gone where, but put
me down as another +1 for Phillip's suggestion.

-stephen

On 7/12/12 10:40 AM, "James M Snell" <jasnell@gmail.com> wrote:

>On Thu, Jul 12, 2012 at 9:32 AM, Phillip Hallam-Baker <hallam@gmail.com>
>wrote:
>> I suspected as much, but is 're-invent chunked in HTTP/1.1' any more
>> practical than 'make chunked worked in HTTP/1.1'.
>>
>
>Unfortunately no. For 1.1 it would appear that the only practical
>approach would be An Ugly Hack at either the content or
>content-encoding level, neither of which are great options for many
>obvious reasons.
>
>> This might be something that we just need to make happen in HTTP/2.0
>> as part of multiplexing. I am willing to spend some of my personal
>> time on that.
>>
>
>+1 ... I would very much like to see integrity checking added to
>http/2.0 as well as a stronger requirement to support trailers in
>general.
>
>
>
>
>> From a Web Services point of view, I think we can manage with an
>> integrity header for cases where we need to support intermediaries and
>> restrict trailers to cases where we don't. In most cases we are going
>> to be layering over TLS in any case.
>>
>>
>> I am not really proposing this header to compete with TLS. The reason
>> I want it is to enable a closer binding of the authentication layer to
>> the content than I can get in TLS. In particular I observe that:
>>
>> HTTP + JSON + Integrity Header ~= WS-*
>>
>>
>> I can't run TLS on the type of processor typical in embedded systems
>> for all sorts of reasons. But I can get a JSON stack and a frame
>> around it.
>>
>>
>> On Thu, Jul 12, 2012 at 12:19 PM, James M Snell <jasnell@gmail.com>
>>wrote:
>>> On Wed, Jul 11, 2012 at 6:51 PM, Phillip Hallam-Baker
>>><hallam@gmail.com> wrote:
>>>> What are the problems inherent in using Content-Integrity as a
>>>>Trailer?
>>>>
>>>> Do the legacy clients botch chunked encoding?
>>>>
>>>>
>>>
>>> Well, it appears that many existing 1.1 client, server and
>>> intermediary implementations have taken the "trailers are optional"
>>> idea to the extreme and either (a) provide no means of appending
>>> trailers to the output, (b) provide no means of accessing trailers
>>> included in the input or make it quite a bit more difficult to get to
>>> them or (c) do not properly forward trailers on as a chunked message
>>> flows from hop-to-hop. It's not clear at this point exactly how
>>> widespread this problem is but it appears pervasive enough to make a
>>> trailer-based content-integrity option problematic at best.
>>>
>>> - James
>>>
>>>> On Wed, Jul 11, 2012 at 7:44 PM, James M Snell <jasnell@gmail.com>
>>>>wrote:
>>>>> Phillip, just want to make sure that I'm keeping up with the
>>>>> conversation thus far... Because of the problems inherent in using
>>>>> Content-Integrity as a Trailer, is the idea then that
>>>>> Content-Integrity would be a standard Header and that a new Transfer
>>>>> or Content Encoding would be defined that supports an incremental
>>>>> integrity check as a component of the encoding?
>>>>>
>>>>>   GET /some/uri HTTP/1.1
>>>>>   Host: example.org
>>>>>   TE: integrity
>>>>>
>>>>> For instance... something like...
>>>>>
>>>>>   HTTP/1.1 200 OK
>>>>>   Content-Type: application/octet-stream
>>>>>   Content-Integrity: SHA-256; modifier=123...; param="..."
>>>>>   Transfer-Encoding: integrity
>>>>>
>>>>>   10
>>>>>   {chunk of bytes}
>>>>>   {digest}
>>>>>   10
>>>>>   {chunk of bytes}
>>>>>   {digest}
>>>>>   0
>>>>>
>>>>> - James
>>>>>
>>>>> On Wed, Jul 11, 2012 at 4:07 PM, Phillip Hallam-Baker
>>>>><hallam@gmail.com> wrote:
>>>>>> On Wed, Jul 11, 2012 at 2:43 PM, Nico Williams
>>>>>><nico@cryptonector.com> wrote:
>>>>>>> I agree that we need something better, and in particular that we
>>>>>>>ought
>>>>>>> to have a MAC instead of a plain hash.  The problem with a MAC is:
>>>>>>> whence the key?  Also, what should the MAC be applied to?
>>>>>>
>>>>>> The MAC should be applied to the 8-bit clean message content (i.e.
>>>>>> precisely that which is bounded by Content-Length)
>>>>>>
>>>>>> If we are talking about Web Services then the key would be
>>>>>>established
>>>>>> through some application layer key exchange (TBS).
>>>>>>
>>>>>> The key requirement from a performance standpoint as I see it is
>>>>>>that
>>>>>> the server has to be able to operate in a stateless fashion which
>>>>>> means using a ticket like approach.
>>>>>>
>>>>>>
>>>>>>> Using a MAC, having a shared session key, ties into HTTP
>>>>>>> authentication.  We can definitely have a generic MAC for HTTP and
>>>>>>>say
>>>>>>> that HTTP authentication mechanisms that can should output session
>>>>>>> keys.  And the HTTP authentication would also have to take care of
>>>>>>>MAC
>>>>>>> algorithm negotiation.  I'd be quite happy with this approach.
>>>>>>
>>>>>> I think that there is definitely an opportunity to make use of a
>>>>>> ticket mode to tie the HTTP authentication to the HTTP channel.
>>>>>>
>>>>>>
>>>>>>> One issue with this approach is: if we always use TLS (but we might
>>>>>>> not), why do the extra session crypto?  What do we gain?  Do we
>>>>>>>need
>>>>>>> to worry about content re-writing proxies, say, as in some 3G
>>>>>>> networks?  If we always use TLS then it suffices to ensure that a)
>>>>>>>TLS
>>>>>>> provides confidentiality protection, b) the server cert remains the
>>>>>>> same for the length of the login session, c) we have a unique,
>>>>>>> unpredictable session ID in the headers (what we might call a
>>>>>>>cookie,
>>>>>>> though we don't want it to be a cookie as such).
>>>>>>
>>>>>> TLS is very large, very complex and was engineered from the
>>>>>>assumption
>>>>>> that there would be public key credentials on the client. Yes,
>>>>>>people
>>>>>> can train it to do other tricks, but doing that is a lot more
>>>>>>complex
>>>>>> than doing what we need in HTTP.
>>>>>>
>>>>>> In my particular Web Service I am using TLS but still want to have a
>>>>>> transport layer authentication protection because I don't want to
>>>>>>do a
>>>>>> TLS public key negotiation on each transaction and I don't want to
>>>>>>be
>>>>>> bound to TLS session expiry.
>>>>>>
>>>>>> In a large commercial environment the TLS processing is often
>>>>>> completely offloaded onto an accelerator that strips out the TLS and
>>>>>> hands clean IP packets to the Web Service. Another frequent screw
>>>>>>case
>>>>>> is TLS proxies like bluecoat devices.
>>>>>>
>>>>>>
>>>>>> But even in the simplest TLS use case, the TLS security context is
>>>>>> really not exposed to the Web Server or the client in the way you
>>>>>> would need to use it for Web Services authentication in the commonly
>>>>>> used APIs. The problem is that TLS is designed to conceal all the
>>>>>> complexity of crypto from the application. That is why it was called
>>>>>> SSL at the start.
>>>>>>
>>>>>>
>>>>>>> In one post you talked about sequencing and replay protection for
>>>>>>> chunks.  Adding that to the MAC really gets us close to the MIC
>>>>>>>token
>>>>>>> features/design from RFCs 1964/4121 (Kerberos GSS mech).  We're
>>>>>>> talking about having a sequence number.  As you say: this isn't
>>>>>>> difficult; we've been doing this for a loooong time in Kerberos
>>>>>>>land.
>>>>>>
>>>>>> Heh, you could use a Kerberos token in my Omnibroker protocol if you
>>>>>> wanted to. But since it is an opaque string of bytes as far as the
>>>>>> client is concerned, well there is no reason to tie it to any one
>>>>>> approach.
>>>>>>
>>>>>> People have been using kerberized cookies for years. The problem
>>>>>>being
>>>>>> that the cookie is not at all bound to the requests or responses.
>>>>>>
>>>>>>> Note that there's no need for sequence numbers to randomized given
>>>>>>> that we have session keys, but sequence number windows add to the
>>>>>>> state to be kept on the server side -- can we tolerate that? Note
>>>>>>>that
>>>>>>> while session key state might be kept on the client in an encrypted
>>>>>>> state ticket, session number windows cannot safely be kept that
>>>>>>>way --
>>>>>>> they must be kept locally.  I tend to think that sequencing and
>>>>>>>replay
>>>>>>> protection are the responsibility of the application -- all it
>>>>>>>needs
>>>>>>> to do is add a sequence number to the chunks and manage its own
>>>>>>> [per-resource] sequence number windows.
>>>>>>
>>>>>> The way I was thinking of helping the application was to provide a
>>>>>> feature that allows Content-Integrity header to specify a key
>>>>>>modifier
>>>>>> as well as a ticket. The key used to calculate the MAC would then be
>>>>>> the XOR of the modifier and the authentication key associated with
>>>>>>the
>>>>>> ticket.
>>>>>>
>>>>>>
>>>>>>> Altogether we need: a session key identifier in the headers (this
>>>>>>> should imply algorithm selection), a direction identifier (or
>>>>>>>separate
>>>>>>> keys for each direction), a sequence number if we need sequencing
>>>>>>> and/or replay detection, what content to MAC, and the MAC itself.
>>>>>>
>>>>>> Direction is already implicit in HTTP requests/responses.
>>>>>>
>>>>>> I don't think we need sequencing, we do need a modifier capability
>>>>>>though.
>>>>>>
>>>>>>> Regarding what to MAC: the direction flag (unless we have diff keys
>>>>>>> for each direction), the channel binding for the TLS channel (if we
>>>>>>> have it), the URL?, some subset of headers? and the body.  Note
>>>>>>>that
>>>>>>> applying the MAC to any headers requires that we say something
>>>>>>>about
>>>>>>> canonicalization (e.g., "use the headers exactly as sent") and
>>>>>>> canonical order (if a subset of headers) (e.g., "in the relative
>>>>>>>order
>>>>>>> of appearance").  Header and body content need to be unambiguously
>>>>>>> separated in the MAC input.  Obviously we can't MAC all headers:
>>>>>>>some
>>>>>>> might be added by proxies, for example.
>>>>>>
>>>>>> I don't see the need to MAC any headers for a Web Service
>>>>>>application.
>>>>>> Put all the information in the content block.
>>>>>>
>>>>>> Otherwise we would have to do the sort of thing that DKIM does to
>>>>>>sign
>>>>>> headers and copy them.
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Website: http://hallambaker.com/
>>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Website: http://hallambaker.com/
>>
>>
>>
>> --
>> Website: http://hallambaker.com/
>

Received on Thursday, 12 July 2012 22:18:28 UTC