A brief history of HTTP Message Signatures (was: Re: Call for Adoption: draft-richanna-http-message-signatures)

Hi all,

I've been the primary author of the draft-cavage-http-signatures-* I-D
for the past 8+ years. That draft is what
draft-richanna-http-message-signatures is based on. I thought it might
be helpful to give this group a brief history on the last 8 years of
work that led up to this recent Call for Adoption.

The work started back in the pre-2010s when Mark Cavage was at Amazon
and involved in securing HTTP API calls in their infrastructure and
between their customers and their infrastructure (AWS). The problem that
he was dealing with at the time was all of the massive amounts of shared
secrets (API keys) that companies were flinging around the Internet and
all of the security concerns (and break ins) related to the practice of
managing millions of customer shared secrets to secure your
infrastructure. The most straightforward solution was to use asymmetric
keys and digital signatures of some kind. He put together a library that
did this and Digital Bazaar (my company) discovered the library and
started using it. His (and others) work at Amazon would eventually
become the way AWS customers sign AWS API requests[1] to this day.

Digital Bazaar was interested in solving the same problem for our
customers and software systems. We felt like shared secrets were a dead
end and were already looking into building what would become W3C
Verifiable Credentials and W3C Decentralized Identifiers. That is, a new
decentralized PKI system that would need a way for clients and servers
to authenticate with each other without necessarily depending on the CA
system. This was also back in the day when Let's Encrypt didn't exist
and we had a need to digitally sign content travelling over plain 'ol
HTTP (no TLS)... because some our customers (individuals in developing
nations) couldn't afford TLS certs at scale. Fundamentally, we were
looking for a way to secure an ad-hoc peer to peer network built on top
of HTTP without the cost associated with TLS.

Around 2012, I approached Mark Cavage and asked him if he'd be okay with
us putting together an I-D for more or less what his software library
did at the time. He agreed, noting that he didn't want any part of the
standards making process due to the glacial speed at which it
progressed. It only took us 8 years to get here, so I have no idea why
Mark had such a skeptical view on the speed of the standards making
process! :P

In 2013, I put together the first I-D for HTTP Message Signatures[2] and
did my best to keep it in sync w/ the two implementations at the time.
Mark subsequently moved onto Joyent and then Oracle deploying the HTTP
Message Signature solution at those organizations. He noted that whether
the organization picked up the technology or not had far more to do with
internal politics than technical merit. That said, those organizations
still use the mechanism to this day.

Then, not much happens between 2013-2017. Most of the folks I end up
speaking to at W3C and IETF note things like "TLS already exists, why do
you need this?" and "Let's Encrypt makes it cheap to own a certificate,
why do you need this?" and "Why don't you just use a JWT payload"...
which have become the standard off-the-cuff responses to why not to do
HTTP Message Signatures. So, for four years, we just learned from our
implementations, updated the specification, addressed attack vectors
that we (and others) were concerned about and would pick up the
standards track stuff if people ever started pulling on the specification.

... and then some European Banking API standards groups picked it up and
deployed it to millions of people in production because it met a need
that they had - they wanted to secure client to server communication
using digital signatures at the HTTP layer. There was a sharp uptick in
implementations. We pleaded with folks to not deploy to production, to
help us with the spec and test suites and getting it through IETF...
which was met with responses to the effect of: "Nah, we've tested it, it
works for us, we're deploying it". Due to the secretive nature of
banking in general, it was impossible to get more information without
being in the inner circle.

I made a trip to IETF 98 in Chicago (late 2017) to see what we could get
started, had a few chats with Mark Nottingham, Martin Thomson, Lucas
Pardue, and Jim Schaad about the possibility of bringing it into the
HTTP WG. Each was supportive (and I'm thankful to each of them for being
welcoming. A small minority of others (from large organizations) were
not, due to some of the design decisions the specification made, and I
didn't personally have the bandwidth to fend off the inevitable deluge
of attacks from those others while trying to take the various
specifications that we were involved in at W3C through that process. So
the decision was made to just sit on the HTTP Message Signatures
specification (and keep updating it) until there was a stronger pull on
the spec. There were around 12 implementations of the spec at this
point... no one had any clue if they interoperated with one another
because we still hadn't written a test suite.

In this time, the W3C Verifiable Credentials work and the W3C
Decentralized Identifier (DID) work was well under way. We were using
HTTP Message Signatures to perform authorization to these software
systems called Encrypted Data Vaults. We were also using HTTP Message
Signatures to do DID-based authentication. We were also using HTTP
Message Signatures to secure HTTP APIs. The approach worked and was
simpler than pulling in the entire JOSE/COSE stack or using JWTs or JWS.
Implementations were fairly easy to put together. Developers seemed to
be happy with the design choices for HTTP Message Signatures.

Fast forward to late 2019 - Mastodon and Pleorma implements and pushes
some variant of HTTP Message Signatures out to its million plus users
and uses it for federated messaging, there are now 20 implementations,
there is a test suite, but only a handful of implementers care to run
against it because many of them are just using their library internally
(while other companies are using their library externally). People are
creating their own bespoke things at W3C that kinda sorta does what HTTP
Message Signatures does but in an incompatible way... there are multiple
Proof of Possession specs popping up... and many are yelling at us to
finish the work at IETF (unpaid, of course). :)

Annabelle, Justin, and I ended up on a random call in some random
standards group late last year to provide some updates on doing proof of
possession. We started talking and here we are. Justin presented the
spec to where he felt it needed to be presented at IETF (and clearly did
a great job, because here we are). Annabelle volunteered to be Editor,
and I'm very thankful for that because the spec has suffered from having
a part-time, overworked editor for years. I'm happy to support by
providing spec text, a communications channel to all the implementers,
and an explanation for why the spec is constructed in the way that it is
and the design decisions that have gone into the spec up to this point.

Hope that helps give folks some context wrt. where the HTTP Message
Signatures spec came to be, some of the history we'll have to be aware
of, and potentially where it might go as a result.

-- manu

PS: I just typed this out and fired it off, no proof reading, my
apologies for it's length and any errors. I'm pressed for time but
wanted to get /something/ out to the group.

[1]https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html
[2]https://datatracker.ietf.org/doc/draft-cavage-http-signatures/00/

-- 
Manu Sporny (skype: msporny, twitter: manusporny)
Founder/CEO - Digital Bazaar, Inc.
blog: Veres One Decentralized Identifier Blockchain Launches
https://tinyurl.com/veres-one-launches

Received on Tuesday, 21 January 2020 16:01:54 UTC