Re: the intersection between Swagger-style APIs and SSI/decentralized identity

Thank you, Dmitri and Markus and Justin for the thoughtful responses. This
is exactly the sort of dialog I was hoping for.

Let me ACK some important points so the conversation can narrow.

   1. Thanks for the correction, @Justin. I did indeed misunderstand your
   proposal, and I agree that the vision you articulated avoids some of the
   concerns I raised.
   2. Likewise, thank you @Markus for the clarification. Knowing that the
   issuer APIs included external stuff only by accident, I can cross a lot off
   my worry list.

More granular responses follow, inline.

>[Dmitri] I don’t quite get why having a REST interface, which covers a lot
of important use cases, precludes also having methods similar to didcomm...
why does it have to be XOR? Why not AND?

In short, I am worried about misspending our momentum. Faced with easy
problems (how to help institutions play a role in SSI) and hard problems
(how to make SSI work peer-to-peer for the marginalized, the offline, and
the private individuals), and faced with urgent customer pain from
institutions that have with real money behind them, we are allowing the
solution space to diverge, and applying the customer pain only to the part
of the solution that's easy (make everybody who wants to talk to an
institution speak REST controlled by that institution). And we are
justifying this choice in exactly those terms ([Justin]: "like it or not,
HTTP/JSON is really, really, really easy to write to, and it’s what
developers today are familiar with") When we get around to addressing the
part that's hard, the institutional pain will be gone, and there will be an
incumbent standard that only works for the non-peer-perpetuating
institutional needs. Then, the lack of institutional pain/money plus an
incumbent standard that ignores the corner cases will guarantee that the
hard part of the problem never gets solved--or that it gets solved far, far
more slowly than it should.

"AND" means two standards. Two standards with overlapping purpose is not
automatically the wrong answer, but we should be suspicious of it. First,
because implementing and supporting and evolving two standards is more
expensive than doing it with one. And second, because this usually leads to
market forces picking one over the other. (In a lot of cases, I think
that's a good thing. However, SSI purports to empower the farmers in
Bolivia as a peer of big corp. If the two modes of interacting use
different standards, and only one of the standards is implemented by big
corp, then there's really only one standard that matters, and SSI's an
empty promise.)

>[Markus]: I imagine that in a typical deployment, calling these internal
Issuer or Verifier APIs would then trigger one or more of the credential
exchange protocols

And therein is a problem. Baked into this assumption is a worldview where
the workflow always starts with the institution--because the web servers
hosting these internal APIs never begin a workflow--they only react, and
then only to a client that's inside the institution. But so-called
"ephemeral proving" imagines a previously unknown/unsessioned Alice
approaching an institutional verifier that's published a standing request
for a presentation; the interaction initiates with the holder, not the
verifier. And how does Alice start issuance on her side, asking the DMV for
a new driver's license because her address has changed?

You could say, "Oh, Alice's intent is out of scope. She uses some
non-standard way to signal her desire, and then the institution begins the
workflow." That "don't call us, we'll call you" answer just means that all
initiation of workflow outside an institution is custom/proprietary and
marginalized. We're back to users adapting themselves to institutional
convenience again. Note Justin's observation: "Abstractions are incredibly
difficult to form from a single use case, and impossible to get right from
zero points. I have seen this attempted so many times, usually to awkward
results at best." When all the use cases for these internal APIs assume
institutions decide when and how interactions begin, we effectively have a
monolithic use case and a single data point.

"Who starts the interaction?" isn't the only deep issue here. Another is
the assumption that there are only two parties. Multiparty issuance (issuer
threshold signatures) and issuance or verification that involve payment are
also square pegs in the client/server round hole. The overarching
phenomenon that I'm trying to get us to grapple with is that institutions
need to stop assuming they can host--and therefore dictate for all other
parties who show up--whatever process we're talking about. Swagger APIs say
a lot about where a host will listen, and about the data payloads that
accompany particular URLs, either as inputs or outputs. They're pretty weak
on sequencing, and they're really hard to adapt to N-party workflows where
N > 2. When they do adapt to N > 2, the server assumes a privileged
position relative to all other parties.

This need to model interactions more richly is not an unsolvable problem
(e.g., an HTTP spec for internal APIs could complement REST with
standardized web hooks and/or SSE)--but now we're talking about something
beyond Swagger, and web developers are probably going to beg for it to be
optional since it complicates the security model (that pesky "servers are
identified with certs" thing again...). Then products will shrug their
shoulders and ignore it, implementing only the workflows where the
institution is in charge. Now the effort won't achieve its interop goal.
(This is not a theoretical concern. Evernym's own issuing product is
implementing both aspects of the functionality, but the complete solution
alters the behavior of each subset, so I fear Evernym would have real
trouble shoehorning its solution into APIs that only contemplate the
institution being in charge. Even if we did, you'd still have vendor
lock-in unless other vendors agree to support the full feature matrix.)

>[Justin]: I don’t expect whatever we come out of TxAuth to :stay:
HTTP-only. I am proposing that we build our protocol on top of HTTP, but
that protocol and its structures can, and arguably should be, ported over
to other non-HTTP stacks (probably by future working groups). However, to
me that doesn’t mean “don’t depend on HTTP” in building things in TxAuth.

I predict that if it starts out HTTP-only, it will definitely stay that
way. I've given several examples from Markus's problem domain where
starting with HTTP assumptions could bake into a spec ideas about the roles
of participants that make porting the structures to another context useless
(because it won't materially change the addressable use cases). These
specific examples may not apply to the TxAuth problem, so perhaps shrug
that off. However, the principle behind my examples is still true: if you
start out by refusing to generalize, later generalization will be painful,
not easy. And independent of that reasoning, my concern about misspending
momentum applies to an HTTP TxAuth, too.

The core assumption that folks want to make is that HTTP is *plenty*
general as a starting point. After all, we have the web. It works and it's
awesome and ubiquitous. I get it. But that's the very mindset that I
consider dangerous. Yes, the web DOES work and it IS awesome in its way,
and it IS ubiquitous. But it's also perpetuating a power imbalance between
individuals and institutions that we can't handwave around, and we can't
afford to solve that problem later. It is now or never. We won't get the
chance to move the needle toward SSI twice. Moving it only in directions
that institutions like is anesthesia that hides the pain, which makes the
deeper problems more insidious.

On Tue, Dec 31, 2019 at 6:02 AM Justin Richer <jricher@mit.edu> wrote:

> To be clear, the proposal on the table for TxAuth isn’t to create "HTTP
> APIs for delegation", but rather to create delegation for HTTP APIs. So,
> the opposite direction. I’m currently also proposing that the process to do
> that will also be HTTP-based (but not an API so much as a protocol). So
> what does that mean for SSI-style work? A couple things that I can think of:
>
> First, this can offer a bridge between HTTP-based applications and APIs
> into non-HTTP (or not-just-HTTP) systems by allowing an HTTP-speaking
> client to kick things off. From that transaction, you can gather claims or
> use the token in all kinds of non-HTTP ways. But like it or not, HTTP/JSON
> is really, really, really easy to write to, and it’s what developers today
> are familiar with. Yes, it has its drawbacks and limitations, many of which
> are listed below. But it still has value as a starting point for many, and
> so I think that a protocol with the ability to bridge these worlds is
> valuable.
>
> Second, I don’t expect whatever we come out of TxAuth to :stay: HTTP-only.
> I am proposing that we build our protocol on top of HTTP, but that protocol
> and its structures can, and arguably should be, ported over to other
> non-HTTP stacks (probably by future working groups). However, to me that
> doesn’t mean “don’t depend on HTTP” in building things in TxAuth. I would
> rather have those translations happen from a firmly rooted protocol than
> from a well-intentioned interlingua. Abstractions are incredibly difficult
> to form from a single use case, and impossible to get right from zero
> points. I have seen this attempted so many times, usually to awkward
> results at best. SOAP comes to mind, which invented many things to allow it
> to go over non-HTTP stacks but ended up being used, in practice, over HTTP.
> This leads to a lot of redundancy and conflict between the layers. That was
> the real value of REST style APIs and HATEOS as a principle.
>
> Note that this approach is very different from what’s being described
> below. Both have benefits, and I’m not suggesting that this group take the
> same tactic that I’m proposing for TxAuth, just explaining what I’m hoping
> to see there.
>
>  — Justin
>
> On Dec 30, 2019, at 2:54 PM, Daniel Hardman <daniel.hardman@evernym.com>
> wrote:
>
> Markus has recently proposed a work item for the CCG to develop
> Swagger-style APIs for issuers and verifiers. Justin has recently proposed
> a charter for a TxAuth group to start working on HTTP-based APIs to
> accomplish delegation. Digital Bazaar has advocated their RESTful
> Credential Handler API (CHAPI) in CCG and other circles as well. No doubt
> many others on this thread are aware of efforts to standardize APIs with
> similar style and similar intersection to the SSI/decentralized identity
> space.
>
> I would like to raise a red flag about such efforts, and trigger a
> thoughtful follow-up conversation. I love Swagger-style APIs and have
> advocated them extensively at past junctures of my career, but I am
> concerned that they are exactly the wrong thing to standardize right now.
> I'll explain my concerns below, in red, and then offer an alternative
> path that has most of the same benefits, in blue.
>
> 1. RESTful APIs are web-only.
>
> How do two farmers with cheap android devices in the highlands of Bolivia
> (or a Canadian Mounty and a speeder on a lonely highway in Yukon Territory,
> or friends in Frankfurt whose cell service has a brownlout, or two
> anti-government protesters) use RESTful APIs to transact business? If the
> answer is, "they subscribe to a service in the cloud so they can talk
> device-to-device," I hope we are embarrassed. We need something that also
> works over BlueTooth and email and other transports where a web server
> isn't a component.
>
> By standardizing a solution that doesn't think about these scenarios, we
> are further marginalizing them, and we are enthroning a
> *you-must-be-connected-and-you-can-be-surveilled* model that guarantees
> *it-isnt-a-standard* FUD for any other efforts to fix the problem.
>
> 2. RESTful APIs perpetuate the PKI model that we claim to be replacing
> with DIDs.
>
> Servers are authenticated in these APIs with a cert. Clients are
> authenticated with a session that follows from an OAuth token or an API key
> or basic auth material. It is possible to imagine "DID Auth" being used for
> a client of a RESTful API, and there have been several efforts to describe
> and standardize such a thing. So far, none has meaningful traction, so all
> APIs in the SSI space that use APIs are also allowing non-DID
> authentication for clients. But even if we solve the problem for the client
> side, nobody is proposing to solve the problem for the server side.
> Institutions in these APIs don't use DIDs for anything meaningful. Thus
> none of the decentralized properties of DIDs are brought to bear for the
> server side of the interaction, and any decentralized qualities of DIDs are
> relegated to minor, optional status for clients.
>
> 3. RESTful APIs foster a power imbalance
>
> What if there's a standard way for institutions to be a verifier or
> issuer, but no way for ordinary people to be a verifier or issuer? Or a
> standard way for institutions to delegate, but no way for ordinary people
> to do it? That's effectively what APIs like the ones I mentioned above
> guarantee. There are multiple reasons why, including:
>
>    - Institutions have web servers; farmers in Bolivia don't. (Saying
>    that they *could* is not helpful; we're just creating more adoption burden
>    for SSI by making the tech harder and more expensive for them.)
>    - Servers can't make the first move in RESTful APIs; everything begins
>    when a client initiates the interaction. This makes it natural for an
>    institution (or a regulatory regime) to impose terms of service or
>    reputation criteria on a client, and unnatural for a client to do the
>    opposite. It's also convenient for hackers and surveillers, since they know
>    they can catch all interactions at inception by simply listening on the
>    server.
>    - Because these APIs are online-only, and because the server always
>    waits for the client to make the first move, they can only be operated by
>    those who have a 24x7x365 cloud presence. Institutions and ordinary people
>    don't have equal access to 24x7x365 cloud capabilities.
>    - Because these APIs are secured on the server side by a cert, they
>    can only be operated by those who have access to expensive, premium,
>    centralized reputation. Again, institutions and people don't have equal
>    access to this.
>
> This is not an exhaustive list of my concerns, but I think it's enough to
> trigger a conversation.
>
> Proposed Alternative
>
> We create web APIs, but we think about them differently. We conceive of
> all of them as exchanges of messages that could also be accomplished over
> BlueTooth, email, etc. HTTP(S) is just another transport, where messages
> happen to be passed by HTTP POST (or GET, as appropriate). Security
> properties associated with the exchange are based on the control of DIDs
> and embodied in the messages themselves (e.g., through encryption/signing),
> not in a transport layer. All semantics for the interaction are conveyed by
> the message content. The traditional URL namespacing of Swagger can still
> exist, but it becomes less interesting, since the message content must be
> enough to convey semantics on its own (so the messages are enough in other
> transports). Either party can initiate an interaction. Institutions and
> people are actually peers.
>
> This is the world of didcomm and application-level protocols built atop
> it; it's described in Aries RFC 0003
> <https://github.com/hyperledger/aries-rfcs/blob/master/concepts/0003-protocols/README.md>.
> But before folks get their hackles up about not wanting to work with Aries
> or DIF, note that I didn't propose that Aries protocols have to be the
> basis for this approach. Instead, I proposed some characteristics we need
> to avoid. Can we explore that assertion on its merits, without getting
> immediately entangled in politics?
>
> There are already maybe 8-10 software stacks, some independent from top to
> bottom, that implement an "API" for issuing credentials and an "API" for
> verifying presentations based on the model I just articulated. These
> implementations are demonstrably interoperable with one another. By
> proposing a new work item for a Swagger API for issuance and verification,
> we are walking away from interoperability with these implementations, and
> we are incurring all of the architectural disadvantages I highlighted in
> red above.
>
> What if we did this instead?
>
>    - Agree that for issuance and verification, the goal about payloads
>    and sequencing for HTTP calls should be alignment between those defined in
>    the Aries protocol and those used by people who aren't Aries-centric. This
>    could involve give and take in either direction; I'm not proposing that it
>    has to be done by simply adopting the Aries work.
>    - Agree not to depend on HTTP-specific constructs (e.g., HTTP headers,
>    HTTP status codes) to signal important semantics--so the payloads could be
>    exchanged over BlueTooth or email just as easily.
>    - Agree that, while URL namespacing gives us a nice hook into
>    Swagger-oriented tools, all semantics required for the interaction are
>    detectable from the payloads themselves.
>
> Then we'd have an HTTP solution that is Swagger-compatible but not limited
> to HTTP. The "API" we created would be interoperable on a much broader
> canvas than simple web.
>
> If we further agreed to this:
>
>    - Authentication of both parties in the interaction will be done on
>    the basis of DID control, not on the basis of certs.
>
> Then we'd also eliminate the dependence on the web's flawed PKI model, and
> the power imbalance of today's web. But I know this is more controversial.
>
>
>