- From: Jeff Martinez <jmar42@gmail.com>
- Date: Wed, 23 Jan 2019 06:34:18 -0800
- To: Evan Schwartz <evan@ripple.com>
- Cc: Marc Blanchet <marc.blanchet@viagenie.ca>, Adrian Hope-Bailie <adrian@hopebailie.com>, Interledger Community Group <public-interledger@w3.org>, Interledger Mailing List - IETF <ledger@ietf.org>, Hyperledger Quilt Mailing List <hyperledger-quilt@lists.hyperledger.org>
- Message-ID: <CANz1PS60ps7bquTnjLCsXAMKuOfxGL8k6sCBCm_0=21G2h1dAw@mail.gmail.com>
Will there be any issues with cyber attacks such as DDoS attacks which can affect the stream? On Wed, Jan 23, 2019, 5:42 AM Evan Schwartz <evan@ripple.com> wrote: > Here are a couple of points that I meant to emphasize in the piece: > > - I do not think we need to (or are necessarily going to be able to) > get rid of BTP. I think BTP is fine for the client-to-server use case. > However, it seems less appropriate for the server-to-server use case and I > think something else is needed for that specific case. > - One part I really like about the idea of using HTTP2 instead of > WebSockets is that it would allow us to decouple the connection management > logic from the application logic. Right now, our actual implementations > need to figure out if they have an open WebSocket with a specific peer and > route to the correct socket. If we were using HTTP with Keep-Alive or > HTTP2, we'd be letting the client libraries figure out for us who to keep > connections open with. > > > > however, http is the « substrate » for everything on Internet nowadays > and therefore one benefits of all services available through cloud > providers that are added every day. Therefore, by using http and that > offering, you can really scale up much more easily. > > I wholeheartedly agree! > > > Users of the HTTP/2 connection then still need to explicitly create new > streams for each request/reply pair if they want to avoid head-of-line > blocking. > > I don't think that's true. Even if the language supports a lower-level > interface, you wouldn't program to that directly. For example, in Node.js > you'd probably use something like http2-client > <https://www.npmjs.com/package/http2-client> and in Rust you'd use reqwest > <https://github.com/seanmonstar/reqwest>. > > > A concern I have with the cluster of stateless connectors is the > latency you'll introduce if the connector must do a balance check against a > DB for each packet. This can be optimised to a point but in reality the > most efficient way to do this will be to ensure the stickiness at the load > balancer routes packets from the same peer to the same connector. Not sure > this will be useful in a scenario where both peers are routing high volumes > of packets (e.g. Coil and Strata). > > This is worth digging into more. A couple of thoughts: > > - I don't think we're anywhere near the throughput where we'd max out > Redis > - Connectors with very high volume and trust may not track balance > limits at all and may just settle up the amount after based on the packet > log (I was envisioning the balance limits being optional) > > > Lastly, I don't think hosting an HTTP server should be treated as a > light requirement. I'd still like to be able to accept payment on a > connection that I establish to my ILSP (i.e. I am the client) and have some > third-party host my SPSP server. > > > - Practically speaking, which third party are you going to have > hosting your SPSP server? It's important to realize that anyone hosting > your SPSP server could steal money from you by changing the ILP address and > shared secret they return > - I'm not saying the SPSP server and receiver *must* be the same > entity. However, even in the case you describe, I'd guess you'd be running > that receiver on a server anyway so you could similarly accept HTTP > requests there > - Even if some power users might have separate SPSP servers and STREAM > receivers, I think most people will end up having that infrastructure > hosted by the same entity > > > Question: Why not transmit the ILP packets with the packet headers as > HTTP headers and the data payload as the body? In place byte replacement > for packet forwarding is not going to help much if you are framing and > de-framing the packets off the wire anyway. > > That was the direction I went with ILP3. > <https://github.com/emschwartz/ilp3> Stefan hates that idea so I'll let > you duke it out with him. > > In short, I think you could but it would mean somewhat needlessly > retransmitting the strings "ILP-Destination", "ILP-Amount", etc. Because of > the way HTTP2 header compression works, you get most of the benefit when > the header keys *and* values are exactly the same each time (not true for > the expiry and condition). > > > I'm keen to get agreement on a basic JS interface that is agnostic to > the underlying wire protocol and then we can ship HTTP, WebSocket and any > other implementation (I'll try and implement your HTTP protocol in JS as > soon as it's stable). > > Don't we already have a JS interface (the plugin)? I don't see any value > in having a separate format for correlating requests and responses if we > are going to put the messages on a protocol that provides that correlation. > It seems more useful for something like WebSockets or raw TLS sockets. As I > said at the top of this email though, I would advocate against making > another WebSocket-based protocol since we already have one. > On Jan 23 2019, at 8:15 am, Marc Blanchet <marc.blanchet@viagenie.ca> > wrote: > > > > On 23 Jan 2019, at 8:08, Adrian Hope-Bailie wrote: > > Hey Evan, > > Thanks for sharing. Some thoughts on your blog post, we can discuss > further > on the call: > > Many of the issues you highlight with BTP are not issues with > WebSockets > themselves but more with how we have used it. E.g. We should be > hosting > WebSocket servers that listen on a single port and path and then use a > proper auth during the handshake. If you do that, the subsequent > messages > sent over the underlying TCP connection are very light on framing and > meta-data. > > From my research WebSockets are one of the most efficient and well > standardised framing protocols available over a raw TCP socket. Binary > WebSocket frames have very little overhead and all of the session > context > is dealt with up front in the handshake. If you compare this to HTTP > (and > HTTP2) you have a lot of overhead in the headers (i.e. sent with every > message). > > Request/reply matching is a key feature requirement of bilateral > protocols > in ILP but I think we've overblown the complexity required in > implementations when you consider that an ILP request/reply pair > should be > very short lived and that by simply using TCP you already have many of > the > delivery guarantees you need. > > HTTP is only capable of request/reply matching because it only allows > a > request to be sent after a previous reply has been received on the > same > connection. i.e. You can only send requests serially. Any ILP > connections > using HTTP 1 or 1.1 will be very inefficient unless they open multiple > connections. > > HTTP/2 gets over this by mutiplexing the connection through streams. > But > under the hood this is just another framing protocol (like WebSockets) > and > giving those frames a stream id. Users of the HTTP/2 connection then > still > need to explicitly create new streams for each request/reply pair if > they > want to avoid head-of-line blocking. It's likely that the way this is > implemented in clients is likely to be highly optimised given the > popularity of HTTP/2, but given that streams have a complex state > machine > that the client must manage it seems unlikely that it can be optimised > further than a simple request/reply protocol using a correlation id. > (i.e. > simply adding a correlation id to the header of a packet and sending > in a > binary WebSocket frame is definitely more efficient in theory but in > practice will come down to implementations). > > > > however, http is the « substrate » for everything on Internet > nowadays and therefore one benefits of all services available through > cloud providers that are added every day. Therefore, by using http and > that offering, you can really scale up much more easily. > > my 2 cents, > > Marc. > > > The one aspect of WebSockets that may be hurting performance is the > masking > which means that payloads need to be pre-processed before being sent. > That > said, it's a very efficient algorithm so we'd need to test it to be > sure > it's an issue. > > A concern I have with the cluster of stateless connectors is the > latency > you'll introduce if the connector must do a balance check against a DB > for > each packet. This can be optimised to a point but in reality the most > efficient way to do this will be to ensure the stickiness at the load > balancer routes packets from the same peer to the same connector. Not > sure > this will be useful in a scenario where both peers are routing high > volumes > of packets (e.g. Coil and Strata). > > That said, I think decoupling the messaging from the clearing and > settlement (FYI: forwarding ILP packets isn't clearing, clearing > involves > reconciliation and netting, but that's a different issue) will be a > huge > improvement to the architecture. As I've mentioned before, this is the > common pattern in most payments. We've been discussing how we could > implement this in the reference connector and hope to do some > experiments > in this regard when we are back from the Mojaloop convening next week. > > Lastly, I don't think hosting an HTTP server should be treated as a > light > requirement. I'd still like to be able to accept payment on a > connection > that I establish to my ILSP (i.e. I am the client) and have some > third-party host my SPSP server. Assuming that the ILP receiver and > SPSP > server are the same thing is a mistake in my opinion and imposes > unnecessary limitations on the architecture. > > Question: Why not transmit the ILP packets with the packet headers as > HTTP > headers and the data payload as the body? In place byte replacement > for > packet forwarding is not going to help much if you are framing and > de-framing the packets off the wire anyway. > > All in, there are some great ideas in the blog but I don't think they > all > depend on changing the bilateral protocol (although I do still want to > propose some better options than BTP). > > I'm keen to get agreement on a basic JS interface that is agnostic to > the > underlying wire protocol and then we can ship HTTP, WebSocket and any > other > implementation (I'll try and implement your HTTP protocol in JS as > soon as > it's stable). > > Adrian > > > > > On Wed, 23 Jan 2019 at 01:11, Evan Schwartz <evan@ripple.com> wrote: > > I just wrote up this blog post describing my proposal for making > connectors stateless and switching bilateral communication to HTTP: > Thoughts > on Scaling Interledger Connectors > < > https://medium.com/@emschwartz/thoughts-on-scaling-interledger-connectors-7e3cad0dab7f > > > : > < > https://medium.com/@emschwartz/thoughts-on-scaling-interledger-connectors-7e3cad0dab7f > > > How I Learned to Stop Worrying and Love HTTP > < > https://medium.com/@emschwartz/thoughts-on-scaling-interledger-connectors-7e3cad0dab7f > > > > Looking forward to hearing your thoughts and discussing on tomorrow's > call! > Evan > > On Jan 22 2019, at 2:17 pm, Adrian Hope-Bailie > <adrian@hopebailie.com> > wrote: > > Hi all, > > Evan has been working on a new HTTP-based bilateral protocol which > he'll > be chatting about on the call tomorrow. Look out for a follow up mail > from > him with some details. > > As usual, if you have any other topics to bring to the call please > do! > > Join from PC, Mac, Linux, iOS or Android: https://zoom.us/s/377506344 > < > http://email.zoom.us/track/click/30854053/zoom.us?p=eyJzIjoiM0w2aVdDeVIxQS1QMWFUY2dOWDNqRF9FXzg4IiwidiI6MSwicCI6IntcInVcIjozMDg1NDA1MyxcInZcIjoxLFwidXJsXCI6XCJodHRwczpcXFwvXFxcL3pvb20udXNcXFwvc1xcXC8zNzc1MDYzNDRcIixcImlkXCI6XCI1NzY5ZmNlY2IyMTA0YjJlOGY5NTdjZDNkMzU3MDQxMlwiLFwidXJsX2lkc1wiOltcImY3NTEyMWVkZGI2NDQ2YjA4YzI1ODAxNjdiMWQ5ODlhM2MyY2FmMTZcIl19In0 > > > > Or iPhone one-tap: > US: +14086380968,,377506344# or +16468769923,,377506344# > > Or Telephone: > Dial(for higher quality, dial a number based on your current > location): > US: +1 408 638 0968 <+1%20408%20638%200968> or +1 646 876 9923 > <+1%20646%20876%209923> or +1 669 900 6833 <+1%20669%20900%206833> > Meeting ID: 377 506 344 > International numbers available: https://zoom.us/u/cbrUAzE3W > < > http://email.zoom.us/track/click/30854053/zoom.us?p=eyJzIjoiTkhxZ1pWcWZEbDlWZy1VaTBYTnFqX2RwcUs0IiwidiI6MSwicCI6IntcInVcIjozMDg1NDA1MyxcInZcIjoxLFwidXJsXCI6XCJodHRwczpcXFwvXFxcL3pvb20udXNcXFwvdVxcXC9jYnJVQXpFM1dcIixcImlkXCI6XCI1NzY5ZmNlY2IyMTA0YjJlOGY5NTdjZDNkMzU3MDQxMlwiLFwidXJsX2lkc1wiOltcImY3NTEyMWVkZGI2NDQ2YjA4YzI1ODAxNjdiMWQ5ODlhM2MyY2FmMTZcIl19In0 > > > > _______________________________________________ > Ledger mailing list > Ledger@ietf.org > https://www.ietf.org/mailman/listinfo/ledger > >
Received on Wednesday, 23 January 2019 14:34:57 UTC