Re: MACing HTTP requests/responses (Re: Content-Integrity header)

What are the problems inherent in using Content-Integrity as a Trailer?

Do the legacy clients botch chunked encoding?

On Wed, Jul 11, 2012 at 7:44 PM, James M Snell <> wrote:
> Phillip, just want to make sure that I'm keeping up with the
> conversation thus far... Because of the problems inherent in using
> Content-Integrity as a Trailer, is the idea then that
> Content-Integrity would be a standard Header and that a new Transfer
> or Content Encoding would be defined that supports an incremental
> integrity check as a component of the encoding?
>   GET /some/uri HTTP/1.1
>   Host:
>   TE: integrity
> For instance... something like...
>   HTTP/1.1 200 OK
>   Content-Type: application/octet-stream
>   Content-Integrity: SHA-256; modifier=123...; param="..."
>   Transfer-Encoding: integrity
>   10
>   {chunk of bytes}
>   {digest}
>   10
>   {chunk of bytes}
>   {digest}
>   0
> - James
> On Wed, Jul 11, 2012 at 4:07 PM, Phillip Hallam-Baker <> wrote:
>> On Wed, Jul 11, 2012 at 2:43 PM, Nico Williams <> wrote:
>>> I agree that we need something better, and in particular that we ought
>>> to have a MAC instead of a plain hash.  The problem with a MAC is:
>>> whence the key?  Also, what should the MAC be applied to?
>> The MAC should be applied to the 8-bit clean message content (i.e.
>> precisely that which is bounded by Content-Length)
>> If we are talking about Web Services then the key would be established
>> through some application layer key exchange (TBS).
>> The key requirement from a performance standpoint as I see it is that
>> the server has to be able to operate in a stateless fashion which
>> means using a ticket like approach.
>>> Using a MAC, having a shared session key, ties into HTTP
>>> authentication.  We can definitely have a generic MAC for HTTP and say
>>> that HTTP authentication mechanisms that can should output session
>>> keys.  And the HTTP authentication would also have to take care of MAC
>>> algorithm negotiation.  I'd be quite happy with this approach.
>> I think that there is definitely an opportunity to make use of a
>> ticket mode to tie the HTTP authentication to the HTTP channel.
>>> One issue with this approach is: if we always use TLS (but we might
>>> not), why do the extra session crypto?  What do we gain?  Do we need
>>> to worry about content re-writing proxies, say, as in some 3G
>>> networks?  If we always use TLS then it suffices to ensure that a) TLS
>>> provides confidentiality protection, b) the server cert remains the
>>> same for the length of the login session, c) we have a unique,
>>> unpredictable session ID in the headers (what we might call a cookie,
>>> though we don't want it to be a cookie as such).
>> TLS is very large, very complex and was engineered from the assumption
>> that there would be public key credentials on the client. Yes, people
>> can train it to do other tricks, but doing that is a lot more complex
>> than doing what we need in HTTP.
>> In my particular Web Service I am using TLS but still want to have a
>> transport layer authentication protection because I don't want to do a
>> TLS public key negotiation on each transaction and I don't want to be
>> bound to TLS session expiry.
>> In a large commercial environment the TLS processing is often
>> completely offloaded onto an accelerator that strips out the TLS and
>> hands clean IP packets to the Web Service. Another frequent screw case
>> is TLS proxies like bluecoat devices.
>> But even in the simplest TLS use case, the TLS security context is
>> really not exposed to the Web Server or the client in the way you
>> would need to use it for Web Services authentication in the commonly
>> used APIs. The problem is that TLS is designed to conceal all the
>> complexity of crypto from the application. That is why it was called
>> SSL at the start.
>>> In one post you talked about sequencing and replay protection for
>>> chunks.  Adding that to the MAC really gets us close to the MIC token
>>> features/design from RFCs 1964/4121 (Kerberos GSS mech).  We're
>>> talking about having a sequence number.  As you say: this isn't
>>> difficult; we've been doing this for a loooong time in Kerberos land.
>> Heh, you could use a Kerberos token in my Omnibroker protocol if you
>> wanted to. But since it is an opaque string of bytes as far as the
>> client is concerned, well there is no reason to tie it to any one
>> approach.
>> People have been using kerberized cookies for years. The problem being
>> that the cookie is not at all bound to the requests or responses.
>>> Note that there's no need for sequence numbers to randomized given
>>> that we have session keys, but sequence number windows add to the
>>> state to be kept on the server side -- can we tolerate that? Note that
>>> while session key state might be kept on the client in an encrypted
>>> state ticket, session number windows cannot safely be kept that way --
>>> they must be kept locally.  I tend to think that sequencing and replay
>>> protection are the responsibility of the application -- all it needs
>>> to do is add a sequence number to the chunks and manage its own
>>> [per-resource] sequence number windows.
>> The way I was thinking of helping the application was to provide a
>> feature that allows Content-Integrity header to specify a key modifier
>> as well as a ticket. The key used to calculate the MAC would then be
>> the XOR of the modifier and the authentication key associated with the
>> ticket.
>>> Altogether we need: a session key identifier in the headers (this
>>> should imply algorithm selection), a direction identifier (or separate
>>> keys for each direction), a sequence number if we need sequencing
>>> and/or replay detection, what content to MAC, and the MAC itself.
>> Direction is already implicit in HTTP requests/responses.
>> I don't think we need sequencing, we do need a modifier capability though.
>>> Regarding what to MAC: the direction flag (unless we have diff keys
>>> for each direction), the channel binding for the TLS channel (if we
>>> have it), the URL?, some subset of headers? and the body.  Note that
>>> applying the MAC to any headers requires that we say something about
>>> canonicalization (e.g., "use the headers exactly as sent") and
>>> canonical order (if a subset of headers) (e.g., "in the relative order
>>> of appearance").  Header and body content need to be unambiguously
>>> separated in the MAC input.  Obviously we can't MAC all headers: some
>>> might be added by proxies, for example.
>> I don't see the need to MAC any headers for a Web Service application.
>> Put all the information in the content block.
>> Otherwise we would have to do the sort of thing that DKIM does to sign
>> headers and copy them.
>> --
>> Website:


Received on Thursday, 12 July 2012 01:52:19 UTC