RE: Raw sockets feedback from Mozilla Network team

In order to facilitate further work and tracking on the issues from Patrick I have created separate issues at the SysApps Raw Socket Github repository:

https://github.com/sysapps/raw-sockets/issues/36

https://github.com/sysapps/raw-sockets/issues/37

https://github.com/sysapps/raw-sockets/issues/38

https://github.com/sysapps/raw-sockets/issues/39

https://github.com/sysapps/raw-sockets/issues/40

https://github.com/sysapps/raw-sockets/issues/41

https://github.com/sysapps/raw-sockets/issues/42

https://github.com/sysapps/raw-sockets/issues/43

https://github.com/sysapps/raw-sockets/issues/44

https://github.com/sysapps/raw-sockets/issues/45

https://github.com/sysapps/raw-sockets/issues/46


Patrick's comment on security was added as a comment in https://github.com/sysapps/raw-sockets/issues/10.


I suggest that further discussion on these issues take place at the Github issue list.

Best regards
 Claes

> -----Original Message-----
> From: Nilsson, Claes1 [mailto:Claes1.Nilsson@sonymobile.com]
> Sent: den 26 augusti 2013 18:16
> To: 'Ke-Fong Lin'; Jonas Sicking; public-sysapps@w3.org; Patrick
> McManus
> Subject: RE: Raw sockets feedback from Mozilla Network team
>
> Hi,
>
> Thanks a lot Patrick for reviewing this specification and Jonas and Ke-
> Fong for commenting. Your feedback is very valuable.
>
> Patrick, are you located in Toronto and able to participate in the F2F-
> meeting? We are discussion this API at 11 tomorrow Aug 27 and probably
> also in a "task force session" on Wednesday 28th 14-18.
>
> See my replies inline below.
>
> BR
>   Claes
>
> > -----Original Message-----
> > From: Ke-Fong Lin [mailto:ke-fong.lin@4d.com]
> > Sent: den 26 augusti 2013 14:17
> > To: Jonas Sicking; public-sysapps@w3.org; Patrick McManus
> > Subject: RE : Raw sockets feedback from Mozilla Network team
> >
> > Hi everyone,
> >
> > See my comments inside the text.
> >
> >
> >
> > Regards,
> >
> >
> >
> >
> >
> > Ke-Fong Lin
> > Développeur Senior
> >
> > 4D SAS
> > 60, rue d'Alsace
> > 92110 Clichy
> > France
> >
> > Standard :
> > Email :    Ke-Fong.Lin@4d.com
> > Web :      www.4D.com
> >
> >
> > ________________________________________
> > De : Jonas Sicking [jonas@sicking.cc]
> > Date d'envoi : lundi 26 août 2013 01:26 À : public-sysapps@w3.org;
> > Patrick McManus Objet : Raw sockets feedback from Mozilla Network
> team
> >
> > Hi All,
> >
> > I asked Patrick McManus from the Mozilla Network team to have a look
> > over the Raw Sockets draft. Here's his feedback (please keep Patrick
> > on cc for the replies since he's not subscribed to this list):
> >
> > * the concept of an isolated "default local interface" (used in a few
> > different places) doesn't really align with networking.. generally
> > when a local interface isn't specified for a socket the one it is
> > assigned is derived from looking up the remote address in the routing
> > table and taking the address of the interface with the most preferred
> > route to the remote address.. This is equally true of TCP and UDP.
> >
> > think about a case where you've got 3 interfaces defined on your
> > machine - 192.168.16.1 which is a natted address used to connect to
> > the internet, 130.215.21.5 which is an address assigned to you while
> > you're connected to your university's VPN, and 127.0.0.1 (localhost).
> >
> > Without additional context - none of those qualify as the default
> > local interface. What generally happens is that when you ask to
> > connect to
> > 8.8.8.8 your local address is assigned to be 192.168.16.1 because
> your
> > Internet route will be used for 8.8.8.8.. but if you ask to connect
> to
> > 130.215.21.1 your local address is assigned to be 130.215.21.5.. and
> > if you want to connect to 127.0.0.1 your local address is also
> 127.0.0.1.
> > So the remote address and the routing table matter - there really
> > isn't a default local address outside of that context.
> >
> > so in general whenever you want a local interface (and you did not
> > explicitly provide one) it can only be determined after your provide
> > the remote address and a system call is made to consult the routing
> > table.
> >
> > you specifically asked about
> > https://github.com/sysapps/raw-sockets/issues/24 .. I'm not concerned
> > about blocking IO here.. the address lookup will require a system
> call
> > but its just another kernel service with a quick response.. no
> > different than gettimeofday() or something really. To me the issue is
> > really just that the concept of assigning a local address is
> > nonsensical until you have assigned the remote one.
> >
> > >>>
> >
> > [Ke-Fong]: When you bind with INADDR_ANY, that is indeed the
> behavior:
> > Use the network interface according to routing.
> > We're a bit higher level from this, having a specified default
> network
> > interface makes sense and would be needed.
> > Suppose my smartphone has 3G, wifi, and bluetooth, they can all act
> as
> > network interfaces.
> > Yet, I may don't want to use 3G because it's usually expensive and
> use
> > wifi instead.
>
> [Claes]  So this means that for TCP a better description on the process
> for allocating a local address is needed in the case when a specific
> local address is not defined by the web application but I don't think
> that the API itself has to change?
>
> For the case when a local interface needs to be stated by the web
> application I refer to the latest comments in
> https://github.com/sysapps/raw-sockets/issues/24, i.e. that the phase 2
> SysApps API, Network Interface API, should be used to list the
> available networks so that a specific interface can be selected by the
> application.
>
> However, your description above on how the routing table is used to set
> the local address based on the remote address to connect to works for
> TCP. What applies then to UDP? With the constructor's options argument
> we can set an optional "default remote address" for subsequent send()
> calls. So if this option is used I assume that the routing table can be
> used in the same manner as for TCP. However, which is the procedure if
> we don't set this option? Does that mean that the local address can't
> be determined until we issue a send() call?
>
> Regarding setting the localAddress attribute I assume that Patrick
> means that for TCP the localAddress attribute immediately could be set
> to the actual value during the execution of the constructor, not to
> null, even if it is not explicitly specified in the constructor.
> Correct?
>
> >
> > >>>
> >
> > * "bind the socket to any available randomly selected local port" -
> > its not clear you want to say randomly here. Sometimes local ports
> are
> > assigned sequentially according to availability.
> >
> > >>>
> >
> > [Ke-Fong]: The intended meaning : Use standard behavior of socket,
> > bind to one of those ephemeral ports they way bind() does it when
> given port as zero.
> > By "random", it rather meant let the system decide.
>
> [Claes] Yes, this is basically what I mean so I will improve the
> wording here.
> >
> > >>>
> >
> > * I don't really understand the loopback attribute. What does it mean
> > to set it to true but connect to 8.8.8.8? What does it mean to set it
> > to false but connect to 15.15.15.15 which your OS has bound to the
> > localhost interface? What purpose does it serve at all?
> >
> > >>>
> >
> > [Ke-Fong]: Loopback is only relevant for UDP multicast.  A note
> should
> > be added about that.
>
> [Claes] Yes, the use case for loopback is multicast. By setting the
> attribute the developer can define whether a sent multicast message
> should be received by yourself or not. I will clarify that this
> attribute is only applicable for multicast.
> >
> > >>>
> >
> > * I don't understand the onHalfClose event.. how do you know if the
> > server called half close or if it hung up completely? (they look the
> > same on the wire)
> >
> > >>>
> >
> > [Ke-Fong]: To "really" close a TCP connection, both peers have to
> > exchange FIN packets along with ACK packets.
> > It is indeed impossible to know if the peer has called shutdown() or
> > close() as both sends a FIN packet (which is part of the four steps
> > closing handshake).
> > But in the case of shutdown(), the peer can still receive data but
> not
> > send, whereas close() kills its socket descriptor.
> > Just imagine a client sending a single request to the server and then
> > doing a shutdown(), it can still read the answer of its request.
> > This is a case allowed by TCP protocol (and probably well used), it
> > ought to be supported.
> >
> > Indeed, a host never calling close() or halfclose(), will always
> > receive an "onhalfclosed" event instead of "closed". This can be
> > confusing.
> > A "usage note" section and a state diagram of the possible states
> > along with transitions would be helpful for understanding.
>
> [Claes] I am considering if we really need the onHalfClose event? Use
> cases:
> 1. A web app (Client) sends a request and expects a reply. Then the
> connection should be closed. So this means send() + halfclose(). The
> server ACKs the FIN and then sends the reply to the request that is
> ACKed by the Client. Then the Server sends FIN that is ACKed by the
> Client and the close event is issued to the web application.
> 2. A server sends FIN but the Client has more data in the send buffer.
> The Client ACKs the FIN, sends the data + FIN and issues the close
> event to the web application.
>
> >
> > >>>
> >
> > * there doesn't seem to be any discussion of the nagle algorithm or a
> > mapping to TCP_NODELAY anywhere in here and its an important topic
> for
> > TCP applications. I would suggest that you provide a TCP attribute
> > called sendCoalescing which defaults to false. Have the documentation
> > point out that this corresponds to the nagle algorithm, which in most
> > TCP APIs defaults to true/on, but because it is often the source of
> > performance problems we have changed the traditional default.
> > Applications that do a lot of small sends that aren't expecting
> > replies to each one (e.g. a ssh application) should enable nagle for
> > networking performance but most applications will not want to. A bit
> > more radically you could just disable nagle all the time without an
> > attribute, but if you do that the API document should really mention
> > it and the ssh client is an example of somewhere where such a config
> > is not optimal.
> >
> > >>>
> >
> > [Ke-Fong]: Ok, that needs to be addressed.
>
> [Claes] So would it be enough to have an additional Boolean field in
> the TCPOptions dictionary (type of the TCPSocket constructor's options
> argument) stating nagle true/false with default set to false?
>
> >
> > >>>
> >
> > * the TCP onMessage event should be called onData or something. A
> > message, at least in network parlance, is data with a preserved
> length..
> > UDP is like that - if you send a 500 byte message the receiver either
> > gets 500 bytes or nothing.. but TCP is all about data streams.. so if
> > you send 500 bytes in one call the receiver could end up with
> anywhere
> > from the first 1 to 500 bytes in its first read and TCP doesn't
> > provide any way to tell if it is just a partial down payment.. folks
> > used to TCP APIs will be used to that - its just the term "message"
> is
> > confusing.
> >
> > >>>
> >
> > [Ke-Fong]: I agree with you regarding the significance of "message"
> > in network parlance.
> > NodeJS has the "data" event instead.
> >
> > Web Worker or Web Socket specs use the onmessage callback to signify
> > that data has been received for reading.
> > Choice for using the "onmessage" name was to conform to the same kind
> > of naming.
> >
> > It can indeed be argued that in cas of websocket it is indeed a
> > message according to the websocket protocol.
> > Same for webworker, as onmessage is the result of a postMessage().
> >
> > In the case of UDPSocket, our spec is very specific that "onmessage"
> > is when a datagram (a "message") is received, so that can work. But
> > for TCPSocket, yes it doesn't make much sense and "data" would be
> better.
> >
>
> [Claes] Ok, if no objection  will change the name to onData for TCP.
>
> > >>>
> >
> > * While we're talking nomenclature please don't use the term "raw"
> > anywhere in this document. That is a well known networking term and
> it
> > doesn't mean access to TCP and UDP interfaces - this was brought up
> to
> > me by several folks at the IETF meeting who were confused about the
> > applicability of this spec because of the use of the term raw sockets.
> > (raw sockets generally give access to ethernet level framing in
> normal
> > networking parlance) These are "transport level socket interfaces" or
> > "tcp/udp sockets" or so on..
> >
> > >>>
> >
> > [Ke-Fong]: I agree with you. "Transport level socket" would the most
> > appropriate term.
>
> [Claes] Ok, yes I can see that "Transport level socket API" would
> probably be a better name but I guess that there will be many options
> on the appropriate name for this API :-).
>
> >
> > >>>
> >
> > * for the server socket API it should be called "onAccept" instead of
> > "onConnect" to match the commonly understood sockets API - accept()
> is
> > the system call you used to take an incoming connection. There
> doesn't
> > seem to be a compelling reason to invent new lingo for well
> understood
> > operations.
> >
> > >>>
> >
> > [Ke-Fong]: I'd rather think "onconnect" is good enough. It can match
> > other W3C's APIs and it is the correct term (contrary to "onmessage").
> > onaccept would definitely ring a bell for seasoned socket programmers,
> > but I'd rather think we should keep onconnect.
>
> [Claes] I don't have a strong opinion here but listening to Jonas and
> Ke-Fong "connect" seems to have slightly more support than "accept" so
> I propose to stay with "connect".
>
> >
> > >>>
> >
> > * the server socket API doesn't need an onOpen event.. there is
> > nothing that happens in between the constructor and onOpen that could
> > block
> >
> > >>>
> >
> > [Ke-Fong]: socket() + bind() + listen() have "immediate" non blocking
> > effect indeed.
>
> [Claes] [Claes] The reason for the open event for TCPServerSocket and
> UDPSocket was the concern that Jonas expressed in
> https://github.com/sysapps/raw-sockets/issues/24 but if there is no
> time consuming actions, e.g. allocating a local interface, we should
> remove the open event.
>
> >
> > >>>
> >
> > * some folks will question why the server socket API doesn't contain
> a
> > backlog attribute that corresponds to the listen() system call that
> is
> > traditionally part of the socket API
> >
> > >>>
> >
> > [Ke-Fong]: Yes, it should be added.
>
> [Claes] If I remember correctly it was said that the maximum number of
> connection requests that can be queued before the system starts
> rejecting incoming requests should not be controllable by web
> applications. But this might be a mistake so the attribute can easily
> be added if required.
> >
> > >>>
> >
> > * on security - we need to think about this a little harder. What
> does
> > it mean to be priv'd enough to use this API? Simply being an
> installed
> > app or being an audited/signed one? The security implications are
> > pretty staggering here and I'm pretty sure the answer needs to be
> more
> > than "unprivd js off a webpage can't do this". Our user's privacy is
> > pretty much undermined by allowing this.. I know this is desired as a
> > backwards looking bridge, but the truth is it brings new
> functionality
> > to the mobile platform and that platform ought to at least be dealing
> > only in TLS and DTLS as table stakes.. While I think TLS and DTLS
> > ought to be mandatory - at the very least they ought to possible and
> > it doesn't really look like that use case has been fully baked into
> > the API yet.
> >
> > >>>
> >
> > [Ke-Fong]: I'm supposed to help write the proposal in that regard.
> > The problem is the TLS and DTLS specs are rather complicated.
> > And there are quite a few questions about what to include or not.
> >
> > Having basic client capabilities (to allow secure connection to SMTPS
> > or HTTPS, etc) will be pretty easy.
> > Problem is mostly server.
>
> [Claes] Patrick, please see the discussion at
> https://github.com/sysapps/raw-sockets/issues/10 .
> >
> > >>>
> >
> > * I guess I'm also concerned about TCPSocket.send().. the definition
> > of it says that if it exceeds an internal buffer of unknowable size
> it
> > must close the socket and throw an error. How can an application use
> > that safely if it doesn't know what value will overrun the socket and
> > trigger the exception and a close?
> >
> > Rather than the true/false semantic being used as a return value here
> > (which requires the whole send be buffered) it would be traditional
> to
> > let the send accept 0->N bytes of the N bytes being sent and have
> that
> > (0->N value) be the return code. Partial sends are part and parcel of
> > stream APIs. That way if I have 4MB to send but you've only got 1MB
> of
> > buffers I don't have to magically guess that - I do a 4MB write, 1MB
> > gets buffered - 1MB is returned and I come back later to try and
> write
> > the next 3MB. (either immediately which probably returns 0 or after
> an
> > ondrain event).
> >
> > >>>[Ke-Fong]:
> >
> > The closing on buffer overflow is too much, and should be changed to
> a
> > (transient) error instead.
> > At previous F2F meeting, this matter has been discussed and it was
> > agreed that the buffer should have an implementation defined size.
> > I agree with that, except that spec should probably specify a minimum.
> >
> > The goal was to make the API easier to use. Partial sends (and reads)
> > are indeed everyday's life with stream API. Yet, this is a JavaScript
> > API, not low-level C.
> > The kernel is buffering the send anyway, send() rather means "queue
> > data to socket's kernel-side write buffer" than actual "send and
> > return me control when you've done it".
> > So it's better to let the API do the buffering, especially when
> you'll
> > have to handle it with ArrayBuffer otherwise. JavaScript is not that
> > good for dealing with raw binary data.
> > Also JavaScript functions are not to be blocking.
> >
> > NodeJS's network API works in a somewhat similar way, and it has a
> > proven track record.
> >
> > For all these reasons, I believe letting the API do the buffering +
> > ondrain when buffer is flushed, is the good way to proceed.
> > The current specification is perhaps a little vague and the ondrain
> > and can be enhanced.[Claes]
>
> [Claes] I don't have so much to add in addition to what Jonas and Ke-
> Fong says but as editor I am happy for tangible proposals for wording
> to clarify vague stuff.
> >
> > >>>

Received on Tuesday, 27 August 2013 18:39:46 UTC