Re: Proposal: New Frame Size Text (was: Re: Design Issue: Frame Size Items)

On Fri, May 10, 2013 at 2:58 PM, James M Snell <jasnell@gmail.com> wrote:

> On Fri, May 10, 2013 at 10:40 AM, William Chan (陈智昌)
> <willchan@chromium.org> wrote:
> > Personally, I'm not sure if there's enough "rough" consensus here yet to
> > commit anything. Just my two cents there.
> >
>
> Agreed. I created a branch and the pull request to track to proposal.
> I will keep the branch up to date as we go until there is either a
> consensus to commit or an alternative is accepted.
>
> > On Fri, May 10, 2013 at 2:07 PM, James M Snell <jasnell@gmail.com>
> wrote:
> >>
> >> This is now pull request #92:
> https://github.com/http2/http2-spec/pull/92
> >>
> >> On Thu, May 9, 2013 at 2:20 PM, James M Snell <jasnell@gmail.com>
> wrote:
> >> > On Thu, May 9, 2013 at 2:01 PM, William Chan (陈智昌)
> >> > <willchan@chromium.org> wrote:
> >> >> I discussed this with Roberto and Hasan. Here are some thoughts on
> >> >> including
> >> >> header blocks within flow control:
> >> >>
> >> >
> >> > Note that my proposal does not explicitly state that the header blocks
> >> > would be put covered by flow control, only that the current window
> >> > size be used to limit the maximum allowed size of those frames.
> >> > Whether or not headers count towards the flow control limits is a
> >> > separate (but definitely related) question... with that in mind, some
> >> > responses below...
> >
> >
> > Thanks for the clarification, I did indeed misunderstand that. What's
> > supposed to happen when the initial per-stream window is set to be
> smaller
> > than what's necessary for a HEADERS+PRIORITY frame? We just never send
> the
> > HTTP message then? Or are we going to move forward with the frame
> > continuation bit?
> >
>
> The limit only effects individual frames, so the option here would be
> to go with the continuation bit.
>
> >>
> >> >
> >> >> * In the proposed text, it appears that only the compressed header
> >> >> block is
> >> >> counting towards the window. Is that really what's desirable in order
> >> >> to
> >> >> properly control memory? Processing the header block requires
> >> >> decompressing
> >> >> it, and that's what will be kept in memory buffers until drained by
> the
> >> >> sink.
> >> >
> >> > Yes, I think it is desirable. The memory management for the
> >> > decompressor state is separate from the notion of how much data the
> >> > receiver is willing to accept over the network. It's likely, for
> >> > instance, that the data in a DATA frame could be compressed as well.
> >> > It would not really make sense to count the decompressed bytes there
> >> > towards the flow control limits.
> >
> >
> > I'm trying to figure out if I'm misunderstanding you or you're
> > misunderstanding the concern. Let me give a concrete example, an HTTP/2
> > proxy. When forwarding DATA frames, it doesn't have to decompress unless
> it
> > wants to inspect the payload for some reason. It just forwards onward to
> the
> > sink, unless the sink is not ready, in which case it buffers the frame
> > (agnostic of whether or not it's compressed). On the contrary, for a
> > HEADERS+PRIORITY frame, and IIUC, it *has* to decompress the header
> block to
> > update the compression context appropriately without introducing HoL
> > blocking with other things using the compression context. And if the
> sink is
> > not ready to accept the headers, and then it most likely has to keep the
> > decompressed header block around. That's my understanding at least, what
> did
> > I miss?
> >
>
> The decompressor state is going to have it's own limit that is
> independent of the frame size. I'm understanding what you're saying
> but what I'm saying is that what you're describing is a different
> issue.
>

I read Roberto's email and this mostly makes sense to me now (I didn't
realize that decompressing would use pointers into the compression context
rather than copying, that was the missing context leading to
misunderstanding here). I think there are some adversarial cases where this
doesn't work, but that's OK.


>
> - James
>