Re: the intersection between Swagger-style APIs and SSI/decentralized identity

Daniel,

I feel some of your arguments are still based on the assumption that
HTTP REST is being proposed for interaction between Issuers, Verifiers,
Holders. I completely agree HTTP REST is not great for that because of
the client/server architecture which leads to power imbalance etc. But
that's not what is being proposed in the Issuer and Verifier API Work Item.

In a typical enterprise system, if internal component A (e.g. a database
server) and internal component B (e.g. a logging service) want to talk
to each other, what's wrong with doing that with an HTTP REST API? Those
internal components don't need DID-to-DID connections between each
other, they don't need DID Auth, JOSE, forwarding messages, etc.

There are many people in our community who strongly believe in the SSI
ideals, yet they work on ways of making certain specialized use cases
easier to implement. For example, the BCGov/Mattr project has developed
an intermediary component that sits between the Holder and Verifier and
makes it possible to use your SSI agent to log in to a website which
understands only OIDC. Would you also say that a dependency on OIDC is
problematic in this SSI use case, in the same way as you argue that a
dependency on HTTP REST is problematic in another SSI use case? And
would you say that since this SSI-OIDC project makes a certain
institution-centric work flow easy (authenticating to a website), it
marginalizes other SSI workflows such as peer-to-peer authentication via
Bluetooth?

I actually do sympathize with your argument that if we publish APIs that
only support (or emphasize) the institution-initiated work flows, then
that will likely create a power imbalance. I also agree that the current
initial draft of the Issuer API only supports this type of work flow and
nothing else.

To address that, as far as I am concerned, supporting the "full feature
matrix" in this Work Item would definitely be a good thing! Modifying
the Work Item to start from a non-HTTP worldview also sounds fine to me
(although I am not convinced at this point that purely internal HTTP
REST APIs necessarily have to result in a power imbalance in the outside
world).

If a company today wants to integrate the Evernym issuing product into
their infrastructure, what do they have to do? Is there a DIDComm
connection between their internal component and the Evernym component,
or is there some other interface? And how hard would it be for them to
switch, if they later decide that they want to use another issuing
product instead of the Evernym one? This is basically what the proposed
Work Item is meant to address.

Markus

On 12/31/19 8:24 PM, Daniel Hardman wrote:
> Thank you, Dmitri and Markus and Justin for the thoughtful responses.
> This is exactly the sort of dialog I was hoping for.
>
> Let me ACK some important points so the conversation can narrow.
>
>  1. Thanks for the correction, @Justin. I did indeed misunderstand
>     your proposal, and I agree that the vision you articulated avoids
>     some of the concerns I raised.
>  2. Likewise, thank you @Markus for the clarification. Knowing that
>     the issuer APIs included external stuff only by accident, I can
>     cross a lot off my worry list.
>
> More granular responses follow, inline.
>
> >[Dmitri] I don’t quite get why having a REST interface, which covers
> a lot of important use cases, precludes also having methods similar to
> didcomm... why does it have to be XOR? Why not AND?
>
> In short, I am worried about misspending our momentum. Faced with easy
> problems (how to help institutions play a role in SSI) and hard
> problems (how to make SSI work peer-to-peer for the marginalized, the
> offline, and the private individuals), and faced with urgent customer
> pain from institutions that have with real money behind them, we are
> allowing the solution space to diverge, and applying the customer pain
> only to the part of the solution that's easy (make everybody who wants
> to talk to an institution speak REST controlled by that institution).
> And we are justifying this choice in exactly those terms ([Justin]:
> "like it or not, HTTP/JSON is really, really, really easy to write to,
> and it’s what developers today are familiar with") When we get around
> to addressing the part that's hard, the institutional pain will be
> gone, and there will be an incumbent standard that only works for the
> non-peer-perpetuating institutional needs. Then, the lack of
> institutional pain/money plus an incumbent standard that ignores the
> corner cases will guarantee that the hard part of the problem never
> gets solved--or that it gets solved far, far more slowly than it should.
>
> "AND" means two standards. Two standards with overlapping purpose is
> not automatically the wrong answer, but we should be suspicious of it.
> First, because implementing and supporting and evolving two standards
> is more expensive than doing it with one. And second, because this
> usually leads to market forces picking one over the other. (In a lot
> of cases, I think that's a good thing. However, SSI purports to
> empower the farmers in Bolivia as a peer of big corp. If the two modes
> of interacting use different standards, and only one of the standards
> is implemented by big corp, then there's really only one standard that
> matters, and SSI's an empty promise.)
>
> >[Markus]: I imagine that in a typical deployment, calling these
> internal Issuer or Verifier APIs would then trigger one or more of the
> credential exchange protocols
>
> And therein is a problem. Baked into this assumption is a worldview
> where the workflow always starts with the institution--because the web
> servers hosting these internal APIs never begin a workflow--they only
> react, and then only to a client that's inside the institution. But
> so-called "ephemeral proving" imagines a previously
> unknown/unsessioned Alice approaching an institutional verifier that's
> published a standing request for a presentation; the interaction
> initiates with the holder, not the verifier. And how does Alice start
> issuance on her side, asking the DMV for a new driver's license
> because her address has changed?
>
> You could say, "Oh, Alice's intent is out of scope. She uses some
> non-standard way to signal her desire, and then the institution begins
> the workflow." That "don't call us, we'll call you" answer just means
> that all initiation of workflow outside an institution is
> custom/proprietary and marginalized. We're back to users adapting
> themselves to institutional convenience again. Note Justin's
> observation: "Abstractions are incredibly difficult to form from a
> single use case, and impossible to get right from zero points. I have
> seen this attempted so many times, usually to awkward results at
> best." When all the use cases for these internal APIs assume
> institutions decide when and how interactions begin, we effectively
> have a monolithic use case and a single data point.
>
> "Who starts the interaction?" isn't the only deep issue here. Another
> is the assumption that there are only two parties. Multiparty issuance
> (issuer threshold signatures) and issuance or verification that
> involve payment are also square pegs in the client/server round hole.
> The overarching phenomenon that I'm trying to get us to grapple with
> is that institutions need to stop assuming they can host--and
> therefore dictate for all other parties who show up--whatever process
> we're talking about. Swagger APIs say a lot about where a host will
> listen, and about the data payloads that accompany particular URLs,
> either as inputs or outputs. They're pretty weak on sequencing, and
> they're really hard to adapt to N-party workflows where N > 2. When
> they do adapt to N > 2, the server assumes a privileged position
> relative to all other parties.
>
> This need to model interactions more richly is not an unsolvable
> problem (e.g., an HTTP spec for internal APIs could complement REST
> with standardized web hooks and/or SSE)--but now we're talking about
> something beyond Swagger, and web developers are probably going to beg
> for it to be optional since it complicates the security model (that
> pesky "servers are identified with certs" thing again...). Then
> products will shrug their shoulders and ignore it, implementing only
> the workflows where the institution is in charge. Now the effort won't
> achieve its interop goal. (This is not a theoretical concern.
> Evernym's own issuing product is implementing both aspects of the
> functionality, but the complete solution alters the behavior of each
> subset, so I fear Evernym would have real trouble shoehorning its
> solution into APIs that only contemplate the institution being in
> charge. Even if we did, you'd still have vendor lock-in unless other
> vendors agree to support the full feature matrix.)
>
> >[Justin]: I don’t expect whatever we come out of TxAuth to :stay:
> HTTP-only. I am proposing that we build our protocol on top of HTTP,
> but that protocol and its structures can, and arguably should be,
> ported over to other non-HTTP stacks (probably by future working
> groups). However, to me that doesn’t mean “don’t depend on HTTP” in
> building things in TxAuth.
>
> I predict that if it starts out HTTP-only, it will definitely stay
> that way. I've given several examples from Markus's problem domain
> where starting with HTTP assumptions could bake into a spec ideas
> about the roles of participants that make porting the structures to
> another context useless (because it won't materially change the
> addressable use cases). These specific examples may not apply to the
> TxAuth problem, so perhaps shrug that off. However, the principle
> behind my examples is still true: if you start out by refusing to
> generalize, later generalization will be painful, not easy. And
> independent of that reasoning, my concern about misspending momentum
> applies to an HTTP TxAuth, too.
>
> The core assumption that folks want to make is that HTTP is *plenty*
> general as a starting point. After all, we have the web. It works and
> it's awesome and ubiquitous. I get it. But that's the very mindset
> that I consider dangerous. Yes, the web DOES work and it IS awesome in
> its way, and it IS ubiquitous. But it's also perpetuating a power
> imbalance between individuals and institutions that we can't handwave
> around, and we can't afford to solve that problem later. It is now or
> never. We won't get the chance to move the needle toward SSI twice.
> Moving it only in directions that institutions like is anesthesia that
> hides the pain, which makes the deeper problems more insidious.
>
> On Tue, Dec 31, 2019 at 6:02 AM Justin Richer <jricher@mit.edu
> <mailto:jricher@mit.edu>> wrote:
>
>     To be clear, the proposal on the table for TxAuth isn’t to create
>     "HTTP APIs for delegation", but rather to create delegation for
>     HTTP APIs. So, the opposite direction. I’m currently also
>     proposing that the process to do that will also be HTTP-based (but
>     not an API so much as a protocol). So what does that mean for
>     SSI-style work? A couple things that I can think of:
>
>     First, this can offer a bridge between HTTP-based applications and
>     APIs into non-HTTP (or not-just-HTTP) systems by allowing an
>     HTTP-speaking client to kick things off. From that transaction,
>     you can gather claims or use the token in all kinds of non-HTTP
>     ways. But like it or not, HTTP/JSON is really, really, really easy
>     to write to, and it’s what developers today are familiar with.
>     Yes, it has its drawbacks and limitations, many of which are
>     listed below. But it still has value as a starting point for many,
>     and so I think that a protocol with the ability to bridge these
>     worlds is valuable.
>
>     Second, I don’t expect whatever we come out of TxAuth to :stay:
>     HTTP-only. I am proposing that we build our protocol on top of
>     HTTP, but that protocol and its structures can, and arguably
>     should be, ported over to other non-HTTP stacks (probably by
>     future working groups). However, to me that doesn’t mean “don’t
>     depend on HTTP” in building things in TxAuth. I would rather have
>     those translations happen from a firmly rooted protocol than from
>     a well-intentioned interlingua. Abstractions are incredibly
>     difficult to form from a single use case, and impossible to get
>     right from zero points. I have seen this attempted so many times,
>     usually to awkward results at best. SOAP comes to mind, which
>     invented many things to allow it to go over non-HTTP stacks but
>     ended up being used, in practice, over HTTP. This leads to a lot
>     of redundancy and conflict between the layers. That was the real
>     value of REST style APIs and HATEOS as a principle. 
>
>     Note that this approach is very different from what’s being
>     described below. Both have benefits, and I’m not suggesting that
>     this group take the same tactic that I’m proposing for TxAuth,
>     just explaining what I’m hoping to see there. 
>
>      — Justin
>
>>     On Dec 30, 2019, at 2:54 PM, Daniel Hardman
>>     <daniel.hardman@evernym.com <mailto:daniel.hardman@evernym.com>>
>>     wrote:
>>
>>     Markus has recently proposed a work item for the CCG to develop
>>     Swagger-style APIs for issuers and verifiers. Justin has recently
>>     proposed a charter for a TxAuth group to start working on
>>     HTTP-based APIs to accomplish delegation. Digital Bazaar has
>>     advocated their RESTful Credential Handler API (CHAPI) in CCG and
>>     other circles as well. No doubt many others on this thread are
>>     aware of efforts to standardize APIs with similar style and
>>     similar intersection to the SSI/decentralized identity space.
>>
>>     I would like to raise a red flag about such efforts, and trigger
>>     a thoughtful follow-up conversation. I love Swagger-style APIs
>>     and have advocated them extensively at past junctures of my
>>     career, but I am concerned that they are exactly the wrong thing
>>     to standardize right now. I'll explain my concerns below, in red,
>>     and then offer an alternative path that has most of the same
>>     benefits, in blue.
>>
>>     1. RESTful APIs are web-only.
>>
>>     How do two farmers with cheap android devices in the highlands of
>>     Bolivia (or a Canadian Mounty and a speeder on a lonely highway
>>     in Yukon Territory, or friends in Frankfurt whose cell service
>>     has a brownlout, or two anti-government protesters) use RESTful
>>     APIs to transact business? If the answer is, "they subscribe to a
>>     service in the cloud so they can talk device-to-device," I hope
>>     we are embarrassed. We need something that also works over
>>     BlueTooth and email and other transports where a web server isn't
>>     a component.
>>
>>     By standardizing a solution that doesn't think about these
>>     scenarios, we are further marginalizing them, and we are
>>     enthroning a
>>     /you-must-be-connected-and-you-can-be-surveilled/ model that
>>     guarantees /it-isnt-a-standard/ FUD for any other efforts to fix
>>     the problem.
>>
>>     2. RESTful APIs perpetuate the PKI model that we claim to be
>>     replacing with DIDs.
>>
>>     Servers are authenticated in these APIs with a cert. Clients are
>>     authenticated with a session that follows from an OAuth token or
>>     an API key or basic auth material. It is possible to imagine "DID
>>     Auth" being used for a client of a RESTful API, and there have
>>     been several efforts to describe and standardize such a thing. So
>>     far, none has meaningful traction, so all APIs in the SSI space
>>     that use APIs are also allowing non-DID authentication for
>>     clients. But even if we solve the problem for the client side,
>>     nobody is proposing to solve the problem for the server side.
>>     Institutions in these APIs don't use DIDs for anything
>>     meaningful. Thus none of the decentralized properties of DIDs are
>>     brought to bear for the server side of the interaction, and any
>>     decentralized qualities of DIDs are relegated to minor, optional
>>     status for clients.
>>
>>     3. RESTful APIs foster a power imbalance
>>
>>     What if there's a standard way for institutions to be a verifier
>>     or issuer, but no way for ordinary people to be a verifier or
>>     issuer? Or a standard way for institutions to delegate, but no
>>     way for ordinary people to do it? That's effectively what APIs
>>     like the ones I mentioned above guarantee. There are multiple
>>     reasons why, including:
>>
>>       * Institutions have web servers; farmers in Bolivia don't.
>>         (Saying that they *could* is not helpful; we're just creating
>>         more adoption burden for SSI by making the tech harder and
>>         more expensive for them.)
>>       * Servers can't make the first move in RESTful APIs; everything
>>         begins when a client initiates the interaction. This makes it
>>         natural for an institution (or a regulatory regime) to impose
>>         terms of service or reputation criteria on a client, and
>>         unnatural for a client to do the opposite. It's also
>>         convenient for hackers and surveillers, since they know they
>>         can catch all interactions at inception by simply listening
>>         on the server.
>>       * Because these APIs are online-only, and because the server
>>         always waits for the client to make the first move, they can
>>         only be operated by those who have a 24x7x365 cloud presence.
>>         Institutions and ordinary people don't have equal access to
>>         24x7x365 cloud capabilities.
>>       * Because these APIs are secured on the server side by a cert,
>>         they can only be operated by those who have access to
>>         expensive, premium, centralized reputation. Again,
>>         institutions and people don't have equal access to this.
>>
>>     This is not an exhaustive list of my concerns, but I think it's
>>     enough to trigger a conversation.
>>
>>     Proposed Alternative
>>
>>     We create web APIs, but we think about them differently. We
>>     conceive of all of them as exchanges of messages that could also
>>     be accomplished over BlueTooth, email, etc. HTTP(S) is just
>>     another transport, where messages happen to be passed by HTTP
>>     POST (or GET, as appropriate). Security properties associated
>>     with the exchange are based on the control of DIDs and embodied
>>     in the messages themselves (e.g., through encryption/signing),
>>     not in a transport layer. All semantics for the interaction are
>>     conveyed by the message content. The traditional URL namespacing
>>     of Swagger can still exist, but it becomes less interesting,
>>     since the message content must be enough to convey semantics on
>>     its own (so the messages are enough in other transports). Either
>>     party can initiate an interaction. Institutions and people are
>>     actually peers.
>>
>>     This is the world of didcomm and application-level protocols
>>     built atop it; it's described in Aries RFC 0003
>>     <https://github.com/hyperledger/aries-rfcs/blob/master/concepts/0003-protocols/README.md>.
>>     But before folks get their hackles up about not wanting to work
>>     with Aries or DIF, note that I didn't propose that Aries
>>     protocols have to be the basis for this approach. Instead, I
>>     proposed some characteristics we need to avoid. Can we explore
>>     that assertion on its merits, without getting immediately
>>     entangled in politics?
>>
>>     There are already maybe 8-10 software stacks, some independent
>>     from top to bottom, that implement an "API" for issuing
>>     credentials and an "API" for verifying presentations based on the
>>     model I just articulated. These implementations are demonstrably
>>     interoperable with one another. By proposing a new work item for
>>     a Swagger API for issuance and verification, we are walking away
>>     from interoperability with these implementations, and we are
>>     incurring all of the architectural disadvantages I highlighted in
>>     red above.
>>
>>     What if we did this instead?
>>
>>       * Agree that for issuance and verification, the goal about
>>         payloads and sequencing for HTTP calls should be alignment
>>         between those defined in the Aries protocol and those used by
>>         people who aren't Aries-centric. This could involve give and
>>         take in either direction; I'm not proposing that it has to be
>>         done by simply adopting the Aries work.
>>       * Agree not to depend on HTTP-specific constructs (e.g., HTTP
>>         headers, HTTP status codes) to signal important semantics--so
>>         the payloads could be exchanged over BlueTooth or email just
>>         as easily.
>>       * Agree that, while URL namespacing gives us a nice hook into
>>         Swagger-oriented tools, all semantics required for the
>>         interaction are detectable from the payloads themselves.
>>
>>     Then we'd have an HTTP solution that is Swagger-compatible but
>>     not limited to HTTP. The "API" we created would be interoperable
>>     on a much broader canvas than simple web.
>>
>>     If we further agreed to this:
>>
>>       * Authentication of both parties in the interaction will be
>>         done on the basis of DID control, not on the basis of certs.
>>
>>     Then we'd also eliminate the dependence on the web's flawed PKI
>>     model, and the power imbalance of today's web. But I know this is
>>     more controversial.
>>
>