Re: delta encoding and state management

On Tue, Jan 22, 2013 at 04:51:29PM -0800, William Chan (?????????) wrote:
> > I must say I'm a bit shocked by this behaviour which is very inefficient
> > from a TCP point of view. This means you have two possibilities for sending
> > your requests then :
> >   - either you keep Nagle enabled and your requests wait in the kernel's stack
> >     for some time (typically 40 ms) before leaving, even if the request is
> >     the last one ;
> >
> >   - or you disable Nagle to force them to leave immediately, but then each
> >     request leaves with a TCP push flag, and then your TCP stack will not
> >     send anything else over the same socket for a full RTT (until its pending
> >     data are ACKed), which is worse.
> 
> We disable Nagle on our sockets. I must be missing something. Why
> would the TCP stack only send one packet per roundtrip when you
> disable Nagle? I do not believe TCP stacks only allow one packet's
> worth of unack'd data.

I'm sorry William, I was wrong. I had memories from debugging a TCP stack
on a mobile network where each PUSH caused a stall waiting for an RTT,
which we attributed to the way they used nagle with the Minshall's
modification which consists in waiting for all small packets to be ACKed.
But this does not apply when NODELAY is set :

>From linux:net/ipv4/tcp_output.c:

    /* Return false, if packet can be sent now without violation Nagle's rules:
     * 1. It is full sized.
     * 2. Or it contains FIN. (already checked by caller)
     * 3. Or TCP_CORK is not set, and TCP_NODELAY is set.
     * 4. Or TCP_CORK is not set, and all sent packets are ACKed.
     *    With Minshall's modification: all sent small packets are ACKed.
     */
    static inline bool tcp_nagle_check(const struct tcp_sock *tp,
                                      const struct sk_buff *skb,
                                      unsigned int mss_now, int nonagle)
    {
            return skb->len < mss_now &&
                    ((nonagle & TCP_NAGLE_CORK) ||
                     (!nonagle && tp->packets_out && tcp_minshall_check(tp)));
    }

That said, the points remains that you don't necessarily want to push many
small packets over the wire!

> > I did not understand you meant delay between objects while parsing, I
> > thought you meant delay between groups.
> 
> Indeed, there's a delay between objects. There must be. As previously
> stated, we have no means of predicting the future. We do not know that
> there are more objects in the document yet to be parsed. When we
> encounter an object during parsing, we request it immediately.

Got it.

> >> > Concerning response headers, I'd say that you emit a first response
> >> > group with the headers from the first response, followed by the
> >> > response. When another response comes in, you have two possibilities,
> >> > either it shares the same headers and you can add a response to the
> >> > existing group, or it does not and you open a new group.
> >>
> >> Wait, is this the critical misunderstanding? Are you maintaining state
> >> across requests and responses? Isn't this a minor modification on the
> >> "simple" compressor? I was assuming you were trying to be stateless.
> >
> > I'm having a hard time following you, I'm sorry. What state across requests
> > and responses do you mean ? The only "state" I'm talking about is the list
> > of common headers between the current message and the previous one in fact.
> > This is true both for requests and responses.
> 
> Yes, this is what I refer to as "simple" compression, as coined by
> Mark (see http://www.mnot.net/blog/2013/01/04/http2_header_compression):
> """
> ?simple? - Omitting headers repeated from the last message, tokenising
> common field names, and a few other tweaks. Otherwise, it looks like
> HTTP/1.
> """

Not exactly. The difference is that if we want to compress based on what
the previous request was, we just have an optimistic compression mode
which depends a lot on how requests are ordered. If we consider that a
browser sends grouped requests, then we can hope they're much better
ordered because its job will precisely be to be smart about how to group
them (eg: take care to avoid minor header variations that are not
indispensable and which might happen from time to time without this
consideration).

> I consider this a form of stateful compression. If you are fine with
> that, then great, I think we've made significant progress in our
> discussions here, and it's a good starting point for discussing
> connection state requirements.

As I said, I think that if the state itself is never larger than a request
and substitutes for the request, it's not that big of a deal. What we must
absolutely avoid is to cumulate states on intermediaries (and to a less
extent, servers). But considering that part of the request will be reused
for subsequent requests might be OK. In order to avoid undesired memory
allocations, we could even state that if a request holds data, then it
must be the last one in the group so that we don't have to keep the state
when we need the space to receive a message body.

Willy

Received on Wednesday, 23 January 2013 07:34:30 UTC