Re: [Ledger] AGENDA - Interledger Call - 23 January - 3pm UTC

An interesting project that could be undertaken if we have a solid
HTTP-based protocol would be a native module for nginx or a similar
webserver that operates as a simple connector.

It could simply route packets based on an in-memory routing table that is
updated by another process (which is also the target of peer.* packets)

On Wed, 23 Jan 2019 at 16:36, Evan Schwartz <evan@ripple.com> wrote:

> Most likely. No matter what messaging technologies we use, attackers will
> figure out how to DDoS connectors and receivers. Another reason I like the
> idea of using HTTP is that it puts us in the same boat as every website on
> the internet and would make it easier for connectors to use off-the-shelf
> DDoS protection tools or services.
>
> On Jan 23 2019, at 9:34 am, Jeff Martinez <jmar42@gmail.com> wrote:
>
> Will there be any issues with cyber attacks such as DDoS attacks which can
> affect the stream?
>
>
>
>
>
> On Wed, Jan 23, 2019, 5:42 AM Evan Schwartz <evan@ripple.com> wrote:
>
> Here are a couple of points that I meant to emphasize in the piece:
>
>    - I do not think we need to (or are necessarily going to be able to)
>    get rid of BTP. I think BTP is fine for the client-to-server use case.
>    However, it seems less appropriate for the server-to-server use case and I
>    think something else is needed for that specific case.
>    - One part I really like about the idea of using HTTP2 instead of
>    WebSockets is that it would allow us to decouple the connection management
>    logic from the application logic. Right now, our actual implementations
>    need to figure out if they have an open WebSocket with a specific peer and
>    route to the correct socket. If we were using HTTP with Keep-Alive or
>    HTTP2, we'd be letting the client libraries figure out for us who to keep
>    connections open with.
>
>
> > however, http is the « substrate » for everything on Internet nowadays
> and therefore one benefits of all services available through cloud
> providers that are added every day. Therefore, by using http and that
> offering, you can really scale up much more easily.
>
> I wholeheartedly agree!
>
> > Users of the HTTP/2 connection then still need to explicitly create new
> streams for each request/reply pair if they want to avoid head-of-line
> blocking.
>
> I don't think that's true. Even if the language supports a lower-level
> interface, you wouldn't program to that directly. For example, in Node.js
> you'd probably use something like http2-client
> <https://www.npmjs.com/package/http2-client> and in Rust you'd use reqwest
> <https://github.com/seanmonstar/reqwest>.
>
> > A concern I have with the cluster of stateless connectors is the
> latency you'll introduce if the connector must do a balance check against a
> DB for each packet. This can be optimised to a point but in reality the
> most efficient way to do this will be to ensure the stickiness at the load
> balancer routes packets from the same peer to the same connector. Not sure
> this will be useful in a scenario where both peers are routing high volumes
> of packets (e.g. Coil and Strata).
>
> This is worth digging into more. A couple of thoughts:
>
>    - I don't think we're anywhere near the throughput where we'd max out
>    Redis
>    - Connectors with very high volume and trust may not track balance
>    limits at all and may just settle up the amount after based on the packet
>    log (I was envisioning the balance limits being optional)
>
> > Lastly, I don't think hosting an HTTP server should be treated as a
> light requirement. I'd still like to be able to accept payment on a
> connection that I establish to my ILSP (i.e. I am the client) and have some
> third-party host my SPSP server.
>
>
>    - Practically speaking, which third party are you going to have
>    hosting your SPSP server? It's important to realize that anyone hosting
>    your SPSP server could steal money from you by changing the ILP address and
>    shared secret they return
>    - I'm not saying the SPSP server and receiver *must* be the same
>    entity. However, even in the case you describe, I'd guess you'd be running
>    that receiver on a server anyway so you could similarly accept HTTP
>    requests there
>    - Even if some power users might have separate SPSP servers and STREAM
>    receivers, I think most people will end up having that infrastructure
>    hosted by the same entity
>
> > Question: Why not transmit the ILP packets with the packet headers as
> HTTP headers and the data payload as the body? In place byte replacement
> for packet forwarding is not going to help much if you are framing and
> de-framing the packets off the wire anyway.
>
> That was the direction I went with ILP3.
> <https://github.com/emschwartz/ilp3> Stefan hates that idea so I'll let
> you duke it out with him.
>
> In short, I think you could but it would mean somewhat needlessly
> retransmitting the strings "ILP-Destination", "ILP-Amount", etc. Because of
> the way HTTP2 header compression works, you get most of the benefit when
> the header keys *and* values are exactly the same each time (not true for
> the expiry and condition).
>
> > I'm keen to get agreement on a basic JS interface that is agnostic to
> the underlying wire protocol and then we can ship HTTP, WebSocket and any
> other implementation (I'll try and implement your HTTP protocol in JS as
> soon as it's stable).
>
> Don't we already have a JS interface (the plugin)? I don't see any value
> in having a separate format for correlating requests and responses if we
> are going to put the messages on a protocol that provides that correlation.
> It seems more useful for something like WebSockets or raw TLS sockets. As I
> said at the top of this email though, I would advocate against making
> another WebSocket-based protocol since we already have one.
> On Jan 23 2019, at 8:15 am, Marc Blanchet <marc.blanchet@viagenie.ca>
> wrote:
>
>
>
> On 23 Jan 2019, at 8:08, Adrian Hope-Bailie wrote:
>
> Hey Evan,
>
> Thanks for sharing. Some thoughts on your blog post, we can discuss
> further
> on the call:
>
> Many of the issues you highlight with BTP are not issues with
> WebSockets
> themselves but more with how we have used it. E.g. We should be
> hosting
> WebSocket servers that listen on a single port and path and then use a
> proper auth during the handshake. If you do that, the subsequent
> messages
> sent over the underlying TCP connection are very light on framing and
> meta-data.
>
> From my research WebSockets are one of the most efficient and well
> standardised framing protocols available over a raw TCP socket. Binary
> WebSocket frames have very little overhead and all of the session
> context
> is dealt with up front in the handshake. If you compare this to HTTP
> (and
> HTTP2) you have a lot of overhead in the headers (i.e. sent with every
> message).
>
> Request/reply matching is a key feature requirement of bilateral
> protocols
> in ILP but I think we've overblown the complexity required in
> implementations when you consider that an ILP request/reply pair
> should be
> very short lived and that by simply using TCP you already have many of
> the
> delivery guarantees you need.
>
> HTTP is only capable of request/reply matching because it only allows
> a
> request to be sent after a previous reply has been received on the
> same
> connection. i.e. You can only send requests serially. Any ILP
> connections
> using HTTP 1 or 1.1 will be very inefficient unless they open multiple
> connections.
>
> HTTP/2 gets over this by mutiplexing the connection through streams.
> But
> under the hood this is just another framing protocol (like WebSockets)
> and
> giving those frames a stream id. Users of the HTTP/2 connection then
> still
> need to explicitly create new streams for each request/reply pair if
> they
> want to avoid head-of-line blocking. It's likely that the way this is
> implemented in clients is likely to be highly optimised given the
> popularity of HTTP/2, but given that streams have a complex state
> machine
> that the client must manage it seems unlikely that it can be optimised
> further than a simple request/reply protocol using a correlation id.
> (i.e.
> simply adding a correlation id to the header of a packet and sending
> in a
> binary WebSocket frame is definitely more efficient in theory but in
> practice will come down to implementations).
>
>
>
> however, http is the « substrate » for everything on Internet
> nowadays and therefore one benefits of all services available through
> cloud providers that are added every day. Therefore, by using http and
> that offering, you can really scale up much more easily.
>
> my 2 cents,
>
> Marc.
>
>
> The one aspect of WebSockets that may be hurting performance is the
> masking
> which means that payloads need to be pre-processed before being sent.
> That
> said, it's a very efficient algorithm so we'd need to test it to be
> sure
> it's an issue.
>
> A concern I have with the cluster of stateless connectors is the
> latency
> you'll introduce if the connector must do a balance check against a DB
> for
> each packet. This can be optimised to a point but in reality the most
> efficient way to do this will be to ensure the stickiness at the load
> balancer routes packets from the same peer to the same connector. Not
> sure
> this will be useful in a scenario where both peers are routing high
> volumes
> of packets (e.g. Coil and Strata).
>
> That said, I think decoupling the messaging from the clearing and
> settlement (FYI: forwarding ILP packets isn't clearing, clearing
> involves
> reconciliation and netting, but that's a different issue) will be a
> huge
> improvement to the architecture. As I've mentioned before, this is the
> common pattern in most payments. We've been discussing how we could
> implement this in the reference connector and hope to do some
> experiments
> in this regard when we are back from the Mojaloop convening next week.
>
> Lastly, I don't think hosting an HTTP server should be treated as a
> light
> requirement. I'd still like to be able to accept payment on a
> connection
> that I establish to my ILSP (i.e. I am the client) and have some
> third-party host my SPSP server. Assuming that the ILP receiver and
> SPSP
> server are the same thing is a mistake in my opinion and imposes
> unnecessary limitations on the architecture.
>
> Question: Why not transmit the ILP packets with the packet headers as
> HTTP
> headers and the data payload as the body? In place byte replacement
> for
> packet forwarding is not going to help much if you are framing and
> de-framing the packets off the wire anyway.
>
> All in, there are some great ideas in the blog but I don't think they
> all
> depend on changing the bilateral protocol (although I do still want to
> propose some better options than BTP).
>
> I'm keen to get agreement on a basic JS interface that is agnostic to
> the
> underlying wire protocol and then we can ship HTTP, WebSocket and any
> other
> implementation (I'll try and implement your HTTP protocol in JS as
> soon as
> it's stable).
>
> Adrian
>
>
>
>
> On Wed, 23 Jan 2019 at 01:11, Evan Schwartz <evan@ripple.com> wrote:
>
> I just wrote up this blog post describing my proposal for making
> connectors stateless and switching bilateral communication to HTTP:
> Thoughts
> on Scaling Interledger Connectors
> <
> https://medium.com/@emschwartz/thoughts-on-scaling-interledger-connectors-7e3cad0dab7f
> >
> :
> <
> https://medium.com/@emschwartz/thoughts-on-scaling-interledger-connectors-7e3cad0dab7f
> >
> How I Learned to Stop Worrying and Love HTTP
> <
> https://medium.com/@emschwartz/thoughts-on-scaling-interledger-connectors-7e3cad0dab7f
> >
>
> Looking forward to hearing your thoughts and discussing on tomorrow's
> call!
> Evan
>
> On Jan 22 2019, at 2:17 pm, Adrian Hope-Bailie
> <adrian@hopebailie.com>
> wrote:
>
> Hi all,
>
> Evan has been working on a new HTTP-based bilateral protocol which
> he'll
> be chatting about on the call tomorrow. Look out for a follow up mail
> from
> him with some details.
>
> As usual, if you have any other topics to bring to the call please
> do!
>
> Join from PC, Mac, Linux, iOS or Android: https://zoom.us/s/377506344
> <
> http://email.zoom.us/track/click/30854053/zoom.us?p=eyJzIjoiM0w2aVdDeVIxQS1QMWFUY2dOWDNqRF9FXzg4IiwidiI6MSwicCI6IntcInVcIjozMDg1NDA1MyxcInZcIjoxLFwidXJsXCI6XCJodHRwczpcXFwvXFxcL3pvb20udXNcXFwvc1xcXC8zNzc1MDYzNDRcIixcImlkXCI6XCI1NzY5ZmNlY2IyMTA0YjJlOGY5NTdjZDNkMzU3MDQxMlwiLFwidXJsX2lkc1wiOltcImY3NTEyMWVkZGI2NDQ2YjA4YzI1ODAxNjdiMWQ5ODlhM2MyY2FmMTZcIl19In0
> >
>
> Or iPhone one-tap:
> US: +14086380968,,377506344# or +16468769923,,377506344#
>
> Or Telephone:
> Dial(for higher quality, dial a number based on your current
> location):
> US: +1 408 638 0968 <+1%20408%20638%200968> or +1 646 876 9923
> <+1%20646%20876%209923> or +1 669 900 6833 <+1%20669%20900%206833>
> Meeting ID: 377 506 344
> International numbers available: https://zoom.us/u/cbrUAzE3W
> <
> http://email.zoom.us/track/click/30854053/zoom.us?p=eyJzIjoiTkhxZ1pWcWZEbDlWZy1VaTBYTnFqX2RwcUs0IiwidiI6MSwicCI6IntcInVcIjozMDg1NDA1MyxcInZcIjoxLFwidXJsXCI6XCJodHRwczpcXFwvXFxcL3pvb20udXNcXFwvdVxcXC9jYnJVQXpFM1dcIixcImlkXCI6XCI1NzY5ZmNlY2IyMTA0YjJlOGY5NTdjZDNkMzU3MDQxMlwiLFwidXJsX2lkc1wiOltcImY3NTEyMWVkZGI2NDQ2YjA4YzI1ODAxNjdiMWQ5ODlhM2MyY2FmMTZcIl19In0
> >
>
> _______________________________________________
> Ledger mailing list
> Ledger@ietf.org
> https://www.ietf.org/mailman/listinfo/ledger
>
>