Re: A brief history of HTTP Message Signatures (was: Re: Call for Adoption: draft-richanna-http-message-signatures)

Manu, thanks very much for telling your story!

To add just a bit to the story from my perspective:

The working group call mentioned below was in fact the Financial-Grade API Working Group in the OpenID Foundation, which was looking at the landscape of different signature mechanisms out there for their use cases. As you can see from Manu’s story below, the financial industry has been using various forms of HTTP Message Signing over the years as a point solution, and the family of incompatible approaches is concerning to a group like FAPI that’s trying to standardize across different groups. That’s why the group invited Manu, Annabelle (who represented AWS Sigv4, their proprietary internal successor to Cavage’s original work), myself (representing the signature method we developed in the OAuth WG years ago), and others on this call. In parallel, I had already been pursuing the goal of a general purpose HTTP Signature Method  around the time of the call, and subsequent conversations with Manu, Mark N, Annabelle, and others gave us the current approach to the path forward with the HTTP WG that I think has the best chance of succeeding.

My goal in my presentation at SECDISPATCH was not just to bring the existing Cavage doc into a working group, but to hopefully solve HTTP Message Signatures for a wide array of use cases using all of the different tools and experience at hand today. I am going to need this work for security token presentation and key binding, both in OAuth 2.0 and in the new work that’s happening in TxAuth. Annabelle’s draft is, I believe, a good starting point for a lot of work going forward. 

I’m really excited to be part of the editorial team and think we’ve got a great opportunity to make something that is useful and impactful for many people on the internet today and in the future. 

 — Justin

> On Jan 21, 2020, at 11:01 AM, Manu Sporny <msporny@digitalbazaar.com> wrote:
> 
> Hi all,
> 
> I've been the primary author of the draft-cavage-http-signatures-* I-D
> for the past 8+ years. That draft is what
> draft-richanna-http-message-signatures is based on. I thought it might
> be helpful to give this group a brief history on the last 8 years of
> work that led up to this recent Call for Adoption.
> 
> The work started back in the pre-2010s when Mark Cavage was at Amazon
> and involved in securing HTTP API calls in their infrastructure and
> between their customers and their infrastructure (AWS). The problem that
> he was dealing with at the time was all of the massive amounts of shared
> secrets (API keys) that companies were flinging around the Internet and
> all of the security concerns (and break ins) related to the practice of
> managing millions of customer shared secrets to secure your
> infrastructure. The most straightforward solution was to use asymmetric
> keys and digital signatures of some kind. He put together a library that
> did this and Digital Bazaar (my company) discovered the library and
> started using it. His (and others) work at Amazon would eventually
> become the way AWS customers sign AWS API requests[1] to this day.
> 
> Digital Bazaar was interested in solving the same problem for our
> customers and software systems. We felt like shared secrets were a dead
> end and were already looking into building what would become W3C
> Verifiable Credentials and W3C Decentralized Identifiers. That is, a new
> decentralized PKI system that would need a way for clients and servers
> to authenticate with each other without necessarily depending on the CA
> system. This was also back in the day when Let's Encrypt didn't exist
> and we had a need to digitally sign content travelling over plain 'ol
> HTTP (no TLS)... because some our customers (individuals in developing
> nations) couldn't afford TLS certs at scale. Fundamentally, we were
> looking for a way to secure an ad-hoc peer to peer network built on top
> of HTTP without the cost associated with TLS.
> 
> Around 2012, I approached Mark Cavage and asked him if he'd be okay with
> us putting together an I-D for more or less what his software library
> did at the time. He agreed, noting that he didn't want any part of the
> standards making process due to the glacial speed at which it
> progressed. It only took us 8 years to get here, so I have no idea why
> Mark had such a skeptical view on the speed of the standards making
> process! :P
> 
> In 2013, I put together the first I-D for HTTP Message Signatures[2] and
> did my best to keep it in sync w/ the two implementations at the time.
> Mark subsequently moved onto Joyent and then Oracle deploying the HTTP
> Message Signature solution at those organizations. He noted that whether
> the organization picked up the technology or not had far more to do with
> internal politics than technical merit. That said, those organizations
> still use the mechanism to this day.
> 
> Then, not much happens between 2013-2017. Most of the folks I end up
> speaking to at W3C and IETF note things like "TLS already exists, why do
> you need this?" and "Let's Encrypt makes it cheap to own a certificate,
> why do you need this?" and "Why don't you just use a JWT payload"...
> which have become the standard off-the-cuff responses to why not to do
> HTTP Message Signatures. So, for four years, we just learned from our
> implementations, updated the specification, addressed attack vectors
> that we (and others) were concerned about and would pick up the
> standards track stuff if people ever started pulling on the specification.
> 
> ... and then some European Banking API standards groups picked it up and
> deployed it to millions of people in production because it met a need
> that they had - they wanted to secure client to server communication
> using digital signatures at the HTTP layer. There was a sharp uptick in
> implementations. We pleaded with folks to not deploy to production, to
> help us with the spec and test suites and getting it through IETF...
> which was met with responses to the effect of: "Nah, we've tested it, it
> works for us, we're deploying it". Due to the secretive nature of
> banking in general, it was impossible to get more information without
> being in the inner circle.
> 
> I made a trip to IETF 98 in Chicago (late 2017) to see what we could get
> started, had a few chats with Mark Nottingham, Martin Thomson, Lucas
> Pardue, and Jim Schaad about the possibility of bringing it into the
> HTTP WG. Each was supportive (and I'm thankful to each of them for being
> welcoming. A small minority of others (from large organizations) were
> not, due to some of the design decisions the specification made, and I
> didn't personally have the bandwidth to fend off the inevitable deluge
> of attacks from those others while trying to take the various
> specifications that we were involved in at W3C through that process. So
> the decision was made to just sit on the HTTP Message Signatures
> specification (and keep updating it) until there was a stronger pull on
> the spec. There were around 12 implementations of the spec at this
> point... no one had any clue if they interoperated with one another
> because we still hadn't written a test suite.
> 
> In this time, the W3C Verifiable Credentials work and the W3C
> Decentralized Identifier (DID) work was well under way. We were using
> HTTP Message Signatures to perform authorization to these software
> systems called Encrypted Data Vaults. We were also using HTTP Message
> Signatures to do DID-based authentication. We were also using HTTP
> Message Signatures to secure HTTP APIs. The approach worked and was
> simpler than pulling in the entire JOSE/COSE stack or using JWTs or JWS.
> Implementations were fairly easy to put together. Developers seemed to
> be happy with the design choices for HTTP Message Signatures.
> 
> Fast forward to late 2019 - Mastodon and Pleorma implements and pushes
> some variant of HTTP Message Signatures out to its million plus users
> and uses it for federated messaging, there are now 20 implementations,
> there is a test suite, but only a handful of implementers care to run
> against it because many of them are just using their library internally
> (while other companies are using their library externally). People are
> creating their own bespoke things at W3C that kinda sorta does what HTTP
> Message Signatures does but in an incompatible way... there are multiple
> Proof of Possession specs popping up... and many are yelling at us to
> finish the work at IETF (unpaid, of course). :)
> 
> Annabelle, Justin, and I ended up on a random call in some random
> standards group late last year to provide some updates on doing proof of
> possession. We started talking and here we are. Justin presented the
> spec to where he felt it needed to be presented at IETF (and clearly did
> a great job, because here we are). Annabelle volunteered to be Editor,
> and I'm very thankful for that because the spec has suffered from having
> a part-time, overworked editor for years. I'm happy to support by
> providing spec text, a communications channel to all the implementers,
> and an explanation for why the spec is constructed in the way that it is
> and the design decisions that have gone into the spec up to this point.
> 
> Hope that helps give folks some context wrt. where the HTTP Message
> Signatures spec came to be, some of the history we'll have to be aware
> of, and potentially where it might go as a result.
> 
> -- manu
> 
> PS: I just typed this out and fired it off, no proof reading, my
> apologies for it's length and any errors. I'm pressed for time but
> wanted to get /something/ out to the group.
> 
> [1]https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html
> [2]https://datatracker.ietf.org/doc/draft-cavage-http-signatures/00/
> 
> -- 
> Manu Sporny (skype: msporny, twitter: manusporny)
> Founder/CEO - Digital Bazaar, Inc.
> blog: Veres One Decentralized Identifier Blockchain Launches
> https://tinyurl.com/veres-one-launches
>

Received on Tuesday, 21 January 2020 17:23:24 UTC