An attempt to clean up the protocol architecture

I have had an ongoing debate with a number of people about the correct way
to architect the protocol layering for Interledger. This has been an
excellent discussion and I've made some valuable realizations (and conceded
some mistakes on my part) along the way but I am still convinced that we
have some errors in our architecture which are making the whole stack
unwieldy and making it difficult to progress on designs for other layers.

There is a sense that we should not be changing the core packet definitions
any more but I'm not sure what justifies this. We have a limited number of
implementations and the specifics of the packet formats and encapsulation
are confined to a single library in both. Furthermore, the fundamentals of
the protocol don't change, just the packet formats and layering to better
reflect the protocol and not be influenced so strongly by the
implementation.

This is my attempt to explain my rationale and solicit feedback from the
community. Hopefully we can come to consensus on the right design (ignoring
concerns about X is frozen; or "we said we aren't going to change Y
anymore" or this will break Z). Once we have a design we consider correct,
let's count the cost and decide if it's a v2 or we can implement it now.


*Protocol Layering: Request/Response at the core*
Protocol layering is a mature pattern. The layering is achieved through
encapsulation. A packet contains headers and a payload, and the payload is
the packet of the next layer up.

In the Internet stack the internetworking layer is the IP layer. Packets
are addressed using IP addresses and layers above rely on these addresses
to introduce messaging semantics. Layers below are host-to-host protocols
that carry the IP packet from one host to another. *All data in the
host-to-host protocol is discarded* when the packet is de-framed and
re-framed for the next hop.

The internetworking layer in the ILP stack is the Interledger layer. At
this layer packets are used to carry payments, quotes and errors and are
addressed using an Interledger Address. Similar to IP there are
host-to-host protocols that exist below ILP and carry the ILP packets over
each hop and higher layer protocols that carry end-to-end data encapsulated
in the payload of the ILP packet.

Currently, ILP requires that data outside the packet be passed along with
the packet instead of inside the packet, requiring special treatment of
specific hots-to-host data and making it very difficult to add/change this
data in future.

A critical difference between IP and ILP is in the messaging semantics. IP
has none. A packet simply contains a source and destination address and
higher layers use these (and introduce additional properties such as ports)
to establish reliable connections, sessions etc. In contrast, ILP only
works if ILP packets exist as request/response pairs and the response
packets follow exactly the same route as the request packets.

As such an ILP request packet has no source address, as this is useless. A
response packet can't simply be addressed to the source of the request and
sent. *A response packet must retrace the route of the request.*

One may argue (as I did before) that a request/response pair should then
have a unique end-to-end identifier that nodes can use to associate a
request with it's original response, however there is a danger in relying
on this identifier as it could be manipulated to cause responses to fail.
If you can't rely on it, why have it at all?

As such, a node at the Interledger layer MUST maintain state and be able
to, not only match request/response pairs at the host-to-host layer but
also,
*match an incoming request/response pair to an outgoing request/response
pair.*

How a host does this not defined in the protocol but it means that at a
minimum the host-to-host protocols must allow request/response pairs to be
matched AND uniquely identified.

Example:

When Host B receives Response A from Host C it must pass that on as
Response 1 to Host A. This would have to be recorded during the routing of
Request 1 to Request A.

Host A                    Host B                    Host C
       --- Request 1 -->         --  Request A -->
       <-- Response 1 --         <-- Response A --

In protocol stacks like IP and ILP the internetworking layer is the glue
between lower host-to-host layers and higher end-to-end layers. To avoid
overloading the internetworking packets themselves it should be possible to
encapsulate higher level protocol packets in the internetworking layer
payload.

Another unique feature of ILP (where it differs from IP) is that some
Interledger packets carry value. You can look at this two ways:
1. The ILP payment packet carries value end-to-end.
2. The ILP payment packet carries data end-to-end but the sender pays for
this.

There are two basic request types in ILP, those that carry data and those
that carry both data and value.

For all request types there is a valid response type but the response could
also be an error. Example: the response to a payment is a fulfillment, the
response to a quote by source amount request is a quote by source amount
response.

In summary:
- ILP depends on request/response semantics at it's core (these are a
requirement of the internetworking layer itself, not just higher level
protocols).
- As such, the internetworking layer packets must be either a request or a
response and the protocol should define which responses are valid for which
requests and all requests should have a response.
- There are two primary request packet types (a request carrying data and a
request carrying data and value).
- There are two primary response types, a success and failure response. The
success responses are sub-typed to match the sub-type of the request,
however the error response could be returned for any request.

Proposal:
- All interledger layer messages should be ILP packets (including
fulfillments) and be capable of carrying higher layer protocol payloads.
- While it may make sense to split the interledger payment and interledger
quoting protocols into new higher level protocols that seems like an
unnecessary abstraction. Instead the packet definitions should just have
some consistency and probably a common base/header.
- We should define two base ILP packet types: request and response.
- A payment packet should extend the request type and fulfillment and error
should extend response.
- The ILQP packets already roughly follow this pattern but perhaps a
generic message packet is also required.

*Ledgers and Trustlines*

For some time there has been a debate about what constitutes the nodes and
what the edges in the Interledger graph. As we've begun to appreciate the
various different forms a line of trust between two peers can take it's
become clear that ledgers are not actually core to the protocol (despite
the name). If you consider senders, receivers and connectors to be the
nodes, then edges are simply lines of trust between any two nodes.

In other words the Interledger is made of nodes that are connected via
financial agreements of some form that allow for two nodes to transfer
value between one another. This agreement may or may not be underwritten by
a ledger that supports conditional transfers. Where there is a ledger that
both parties trust to perform conditional transfers correctly the parties
no longer need to trust one another (they trust the ledger).

The Interledger works because not only does it define a universal address
space for this network, it also defines a universal signature scheme for
payment requests and receipts that can be leveraged by individual
trustlines on the route of a payment, for conditional transfers between two
peers.

When two disconnected nodes wish to setup a payment they exchange data so
that the sender has, at a minimum, the address of the receiver, a condition
that the receiver can fulfill and the amount (in the receiver's currency)
that the receiver must receive. Next the sender finds a peer to send a
local transfer to, to initiate the end-to-end payment. They package all of
the end-to-end data in an ILP packet and make the local transfer (attaching
the packet or sending it to the peer separately).

The terms of the local transfer are: "Show me the signature from the
receiver that proves they were paid and you get the proceeds of the local
transfer."

These terms can be enforced in a variety of ways, one of which is by making
the transfer on a ledger that understands the condition and fulfillment
format and supports conditional payments. This protects the sender from
their upstream peer in the case where they are unable to produce the
fulfillment but still want to claim the proceeds of the transfer. i.e. It
simply changes who the sender trusts (the ledger instead of the peer)

How the ILP packet is transferred to the peer and how the transfer is setup
on the ledger *are very different concerns*. It is a mistake to assume that
the communication mechanism between peers is the creation of the transfer
on the ledger itself. I believe that some of the confusion around this
comes from the design of 5-bells ledger where these are the same thing.

Consider the following setup where the ledger supports conditional
transfers but does not support notifications as an example.

Peer A
Ledger                                              Peer B

  | -- LocalTransfer(LocalAmount,Condition) -->
|                                                   |
  | <----------- Prepared --------------------- |
                                             |
  | -------------- ILP Payment Request (ILP Address, RemoteAmount,
Condition) --------------------> |
  |                                             | <--------- Check for
pending transfer------------ |
  | <----------------------------------------- Ack
------------------------------------------------ |
  | - Poll for completed transfer (Optional) ->
|                                                   |
  |                                             |
                                             | --> Transfer
  |                                             |
                                             | <-- Fulfillment
  |                                             | <---------- Submit
fulfillment------------------- |
  | <------------------------- ILP Payment Response(Fulfillment)
---------------------------------- |
  | - Confirm completed transfer (Optional) -->
|                                                   |

Note how the communication between peers, carrying the ILP packet, does not
go through the ledger. Also, note how the ledger doesn't need to understand
anything about ILP other than supporting the universal signature format
(conditions and fulfillments). For simplicity this example excluded expiry
but these could just as easily be included alongside the condition in the
direct communication between the peers (i.e. in the ILP packet).

The expiry in the packet is very similar to the TTL in an IP packet.
Theoretically it can be modified at each hop to a value that the local
sender considers safe for the outgoing local transfer and the trustline on
which it is sent.

Summary:

- ILP is not about ledgers, it is about trustlines between nodes/hosts.
- A trustline may be underwritten by a ledger to allow nodes to establish a
trustline-by-proxy where they trust the ledger and not their peer. This is
possible if the ledger supports conditional payments using ILP-style
conditions.
- The protocol should differentiate between the operation of preparing a
transfer on a ledger and the operation of passing an ILP packet from one
peer to another.

Proposal:
- The condition and timeout should be included in the ILP payment packet.
- Where the fulfillment is a signature over the packet these can be zero'd
out first as with the checksum in an IP packet.

Look forward to your thoughts!