Re: [Ledger] AGENDA - Interledger Call - 23 January - 3pm UTC

Hey Evan,

Thanks for sharing. Some thoughts on your blog post, we can discuss further
on the call:

Many of the issues you highlight with BTP are not issues with WebSockets
themselves but more with how we have used it. E.g. We should be hosting
WebSocket servers that listen on a single port and path and then use a
proper auth during the handshake. If you do that, the subsequent messages
sent over the underlying TCP connection are very light on framing and
meta-data.

From my research WebSockets are one of the most efficient and well
standardised framing protocols available over a raw TCP socket. Binary
WebSocket frames have very little overhead and all of the session context
is dealt with up front in the handshake. If you compare this to HTTP (and
HTTP2) you have a lot of overhead in the headers (i.e. sent with every
message).

Request/reply matching is a key feature requirement of bilateral protocols
in ILP but I think we've overblown the complexity required in
implementations when you consider that an ILP request/reply pair should be
very short lived and that by simply using TCP you already have many of the
delivery guarantees you need.

HTTP is only capable of request/reply matching because it only allows a
request to be sent after a previous reply has been received on the same
connection. i.e. You can only send requests serially. Any ILP connections
using HTTP 1 or 1.1 will be very inefficient unless they open multiple
connections.

HTTP/2 gets over this by mutiplexing the connection through streams. But
under the hood this is just another framing protocol (like WebSockets) and
giving those frames a stream id. Users of the HTTP/2 connection then still
need to explicitly create new streams for each request/reply pair if they
want to avoid head-of-line blocking. It's likely that the way this is
implemented in clients is likely to be highly optimised given the
popularity of HTTP/2, but given that streams have a complex state machine
that the client must manage it seems unlikely that it can be optimised
further than a simple request/reply protocol using a correlation id. (i.e.
simply adding a correlation id to the header of a packet and sending in a
binary WebSocket frame is definitely more efficient in theory but in
practice will come down to implementations).

The one aspect of WebSockets that may be hurting performance is the masking
which means that payloads need to be pre-processed before being sent. That
said, it's a very efficient algorithm so we'd need to test it to be sure
it's an issue.

A concern I have with the cluster of stateless connectors is the latency
you'll introduce if the connector must do a balance check against a DB for
each packet. This can be optimised to a point but in reality the most
efficient way to do this will be to ensure the stickiness at the load
balancer routes packets from the same peer to the same connector. Not sure
this will be useful in a scenario where both peers are routing high volumes
of packets (e.g. Coil and Strata).

That said, I think decoupling the messaging from the clearing and
settlement (FYI: forwarding ILP packets isn't clearing, clearing involves
reconciliation and netting, but that's a different issue) will be a huge
improvement to the architecture. As I've mentioned before, this is the
common pattern in most payments. We've been discussing how we could
implement this in the reference connector and hope to do some experiments
in this regard when we are back from the Mojaloop convening next week.

Lastly, I don't think hosting an HTTP server should be treated as a light
requirement. I'd still like to be able to accept payment on a connection
that I establish to my ILSP (i.e. I am the client) and have some
third-party host my SPSP server. Assuming that the ILP receiver and SPSP
server are the same thing is a mistake in my opinion and imposes
unnecessary limitations on the architecture.

Question: Why not transmit the ILP packets with the packet headers as HTTP
headers and the data payload as the body? In place byte replacement for
packet forwarding is not going to help much if you are framing and
de-framing the packets off the wire anyway.

All in, there are some great ideas in the blog but I don't think they all
depend on changing the bilateral protocol (although I do still want to
propose some better options than BTP).

I'm keen to get agreement on a basic JS interface that is agnostic to the
underlying wire protocol and then we can ship HTTP, WebSocket and any other
implementation (I'll try and implement your HTTP protocol in JS as soon as
it's stable).

Adrian




On Wed, 23 Jan 2019 at 01:11, Evan Schwartz <evan@ripple.com> wrote:

> I just wrote up this blog post describing my proposal for making
> connectors stateless and switching bilateral communication to HTTP: Thoughts
> on Scaling Interledger Connectors
> <https://medium.com/@emschwartz/thoughts-on-scaling-interledger-connectors-7e3cad0dab7f>
> :
> <https://medium.com/@emschwartz/thoughts-on-scaling-interledger-connectors-7e3cad0dab7f>
> How I Learned to Stop Worrying and Love HTTP
> <https://medium.com/@emschwartz/thoughts-on-scaling-interledger-connectors-7e3cad0dab7f>
>
> Looking forward to hearing your thoughts and discussing on tomorrow's call!
> Evan
>
> On Jan 22 2019, at 2:17 pm, Adrian Hope-Bailie <adrian@hopebailie.com>
> wrote:
>
> Hi all,
>
> Evan has been working on a new HTTP-based bilateral protocol which he'll
> be chatting about on the call tomorrow. Look out for a follow up mail from
> him with some details.
>
> As usual, if you have any other topics to bring to the call please do!
>
> Join from PC, Mac, Linux, iOS or Android: https://zoom.us/s/377506344
> <http://email.zoom.us/track/click/30854053/zoom.us?p=eyJzIjoiM0w2aVdDeVIxQS1QMWFUY2dOWDNqRF9FXzg4IiwidiI6MSwicCI6IntcInVcIjozMDg1NDA1MyxcInZcIjoxLFwidXJsXCI6XCJodHRwczpcXFwvXFxcL3pvb20udXNcXFwvc1xcXC8zNzc1MDYzNDRcIixcImlkXCI6XCI1NzY5ZmNlY2IyMTA0YjJlOGY5NTdjZDNkMzU3MDQxMlwiLFwidXJsX2lkc1wiOltcImY3NTEyMWVkZGI2NDQ2YjA4YzI1ODAxNjdiMWQ5ODlhM2MyY2FmMTZcIl19In0>
>
> Or iPhone one-tap:
>     US: +14086380968,,377506344# or +16468769923,,377506344#
>
> Or Telephone:
>     Dial(for higher quality, dial a number based on your current
> location):
>     US: +1 408 638 0968 <+1%20408%20638%200968> or +1 646 876 9923
> <+1%20646%20876%209923> or +1 669 900 6833 <+1%20669%20900%206833>
>     Meeting ID: 377 506 344
>     International numbers available: https://zoom.us/u/cbrUAzE3W
> <http://email.zoom.us/track/click/30854053/zoom.us?p=eyJzIjoiTkhxZ1pWcWZEbDlWZy1VaTBYTnFqX2RwcUs0IiwidiI6MSwicCI6IntcInVcIjozMDg1NDA1MyxcInZcIjoxLFwidXJsXCI6XCJodHRwczpcXFwvXFxcL3pvb20udXNcXFwvdVxcXC9jYnJVQXpFM1dcIixcImlkXCI6XCI1NzY5ZmNlY2IyMTA0YjJlOGY5NTdjZDNkMzU3MDQxMlwiLFwidXJsX2lkc1wiOltcImY3NTEyMWVkZGI2NDQ2YjA4YzI1ODAxNjdiMWQ5ODlhM2MyY2FmMTZcIl19In0>
>
> _______________________________________________
> Ledger mailing list
> Ledger@ietf.org
> https://www.ietf.org/mailman/listinfo/ledger
>
>