- From: Jeff Pinner <jpinner@twitter.com>
- Date: Tue, 8 Jul 2014 19:17:02 -0700
- To: William Chan (陈智昌) <willchan@chromium.org>
- Cc: Mike Belshe <mike@belshe.com>, Greg Wilkins <gregw@intalio.com>, HTTP Working Group <ietf-http-wg@w3.org>
To be clear, my issue is that this change makes determining valid frame lengths dependent on the session state which to me seems a step backwards both in terms of protocol layering and in terms of the achievable decoding throughput based on the simple frame layout. On Tue, Jul 8, 2014 at 7:13 PM, Jeff Pinner <jpinner@twitter.com> wrote: > Reading through I am -1 on the frame layout changes. > > I would be fine with un-reserving the first two bits and simply > requiring 64kb frames, but I feel like needing to compare allowed > sizes against SETTINGS that may change dynamically is a more > complicated change. > > On Tue, Jul 8, 2014 at 5:37 PM, William Chan (陈智昌) > <willchan@chromium.org> wrote: >> On Tue, Jul 8, 2014 at 5:36 PM, William Chan (陈智昌) <willchan@chromium.org> >> wrote: >>> >>> On Tue, Jul 8, 2014 at 3:09 PM, Mike Belshe <mike@belshe.com> wrote: >>>> >>>> Thanks, Greg. >>>> >>>> I believe your proposal is preferable to what we have today in H2. >>>> >>>> >>>> On Mon, Jul 7, 2014 at 8:14 PM, Greg Wilkins <gregw@intalio.com> wrote: >>>>> >>>>> >>>>> Mike, >>>>> >>>>> I think the settings are needed with the ability to send larger frame >>>>> sizes. If I understand the feedback from SPDY correctly, the larger frame >>>>> sizes there were some endpoints that were lazy with their frame sizes and >>>>> sent overly large ones that hurt multiplexing. >>>> >>>> >>>> OK - thanks for clarifying. I don't recall any such learning from SPDY. >>>> It certainly wasn't learning while I was at Google. >>> >>> >>> Yes, Patrick identified these issues after you stopped working on SPDY. >> >> >> Clarification: Patrick identified actual abuses live in the wild. Not >> theoretical. We indeed talked about the theoretical issue back when you were >> still at Google. >> >>> >>> >>>> >>>> >>>> BTW, some history: >>>> * SPDY originally used large frames too - 24bits - and let you use it as >>>> you wished. Some people worried that these large frame sizes would be >>>> abused someday and felt compelled to shrink them. But, as far as I know, >>>> those arguments were only theoretical. The fact is that you're really only >>>> going to hurt yourself; and 'you' is the server. Proxies are a different >>>> story. >>>> >>>> * SPDY's final field sizes were not arbitrary - they were an evolution >>>> based on lessons of what is valuable and what works. Initially all headers >>>> were limited to 16k-ish. This was because we didn't "like" large headers. >>>> But, when put to practice there are edge cases which needed to be dealt >>>> with, and some apps out there, wisely or not, use ridiculously sized >>>> headers. It just happens. Increasing the size was always preferable >>>> because: >>>> a) we learned that the constraint itself wasn't very valuable and >>>> didn't make anything materially better. >>>> b) there were edge cases where the constraint just didn't work, and >>>> the solutions were worse than the initial problem. >>>> >>>> >>>>> >>>>> >>>>> The intention of the proposal is to allow large frame sizes IF NEEDED, >>>>> but to use the settings to constrain implementations to the 16KB that has >>>>> been selected as a reasonable one-size-fits-all for todays traffic. The >>>>> settings can then be adjusted without needing to rev the spec or deploy >>>>> extensions as experience is gained, traffic changes, networks change etc. >>>>> It may well be that they are adjusted down initially as much as they are >>>>> adjusted up. >>>>> >>>>> >>>>> So I don't see it as a compromise - it is simply moving the limit that >>>>> must exist from being a fixed capability of the framing layer to be an >>>>> explicitly managed parameter of the protocol. >>>>> >>>> >>>> Ok. I don't see any real need for these either. They may introduce new >>>> oddities if someone goes crazy with them. I'd lean toward not adding these >>>> if possible. >>>> >>>> Mike >>>> >>>> >>>> >>>> >>>> >>>>> >>>>> cheers >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On 8 July 2014 10:07, Mike Belshe <mike@belshe.com> wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Jul 7, 2014 at 12:50 AM, Greg Wilkins <gregw@intalio.com> >>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> https://github.com/http2/http2-spec/pull/548 >>>>>>> >>>>>>> Background >>>>>>> >>>>>>> The HTTP2 protocol has a requirement to be able to transport large >>>>>>> headers, that exceed the payload size of a single frame at the current 16KB >>>>>>> maximum size. >>>>>>> >>>>>>> To address this requirement, the current draft (13) includes the >>>>>>> CONTINUATION frames, 0 or more of which may be sent after a HEADERS or >>>>>>> PUSH_PROMISE frame to contain the large headers. There has been significant >>>>>>> criticism of the CONTINUATION design, including: >>>>>>> >>>>>>> The total length of a HEADERS+CONTINUATION* sequence is not known >>>>>>> until the last frame in the sequence is processed. A receiver that wishes to >>>>>>> reject streams headers larger than a specific limit may have to process many >>>>>>> frames and hold the results in memory before it discovers the header is too >>>>>>> large. >>>>>>> >>>>>>> The size of header that an endpoint is prepared to receive is not >>>>>>> known in advance. The only way a sender can know if a header too large is by >>>>>>> attempting to send it and receiving an error in response. Error handling of >>>>>>> headers may be difficult for an endpoint to handle efficiently and can >>>>>>> result in the closure of the entire connection. >>>>>>> >>>>>>> The END_STREAM flag is not present on the CONTINUATION frame, thus it >>>>>>> is possible for a stream to send CONTINUATION frames after a HEADERS frame >>>>>>> that has the END_STREAM flag set. This is confusing and increases the >>>>>>> complexity of the state machine required to process streams. It is highly >>>>>>> desirable that a set END_STREAM flag truly indicates the last non control >>>>>>> frame of a stream. >>>>>>> >>>>>>> There is a significant discontinuity in the code path required to >>>>>>> process headers. Headers up to an indeterminate size (roughly 20-something >>>>>>> KB) can be handled in a single frame. Headers that exceed this size must be >>>>>>> handled in multiple frames of different types with different frame flags and >>>>>>> stream control logic. Because the vast majority of headers sent (>99.99%) >>>>>>> are below this indeterminate size, implementations will have a code path >>>>>>> that is seldom executed and probably insufficiently tested. This invites >>>>>>> poor and/or partial and/or incorrect implementation. >>>>>>> >>>>>>> Because of the HPACK compression algorithm, a sequence of >>>>>>> HEADERS+CONTINUATIONS frames may not be interleaved with any other frame. >>>>>>> This effectively makes the sequence a single large frame. Because of the >>>>>>> simplicity of description and implementation it is proposed that it would be >>>>>>> far simpler to meet the requirement of large headers by supporting large >>>>>>> frames. >>>>>>> >>>>>>> This proposal has been prepared as it is possible to meet the >>>>>>> requirements of CONTINUATIONS without the complications and criticisms >>>>>>> above. >>>>>>> >>>>>>> This proposal addresses the issue of sending/receiving large HTTP >>>>>>> headers without giving endpoints and intermediaries unlimited resource >>>>>>> commitments nor unknown limits >>>>>>> >>>>>>> Additional Frame Size Issues Addressed >>>>>>> >>>>>>> The current draft (13) has maximum frame size of 16KB, which is an >>>>>>> arbitrary value that has been selected on the basis of experience to provide >>>>>>> a reasonable compromise between the efficiency of transmitting data vs the >>>>>>> quality of service for multiplexed channels. >>>>>>> >>>>>>> Whilst this educated guess may be near optimal for today's networks >>>>>>> and traffic, it is entirely possible that some current and/or future >>>>>>> networks may require a different value to achieve an optimal balance. There >>>>>>> have already been proposals [1] put to the WG to reduce the frame size to >>>>>>> optimise multiplexing , as well as discussion that high capacity, low >>>>>>> latency networks can achieve satisfactory multiplexing quality of service >>>>>>> with large frame sizes. >>>>>>> >>>>>>> This proposal addresses the issue that a fixed frame size does not >>>>>>> allow tuning multiplexing performance based on current/future experience. >>>>>>> >>>>>>> It has also been noted that 16KB is near the middle of the peak of the >>>>>>> current HTTP Object size histogram [2], so that a small change in the frame >>>>>>> size may have a significant impact on the number of HTTP messages that can >>>>>>> be sent in a single frame, without significant impacts on QoS. The HTTP >>>>>>> Object size histogram has changed significantly over time and is expected to >>>>>>> continue to do so. >>>>>>> >>>>>>> This proposal addresses the issue of tuning the frame size based on >>>>>>> experience of actual payload sizes. >>>>>>> >>>>>>> There have also been issues raised that a 16KB frame size does not >>>>>>> allow efficient data transfer [3] even when the end points are aware that >>>>>>> only a single stream is likely to be required for the imminent future, or >>>>>>> that a particular stream is of high priority. >>>>>>> >>>>>>> This proposal addresses the issue of tuning the frame size for >>>>>>> transport efficiency for specific streams in specific situations. >>>>>>> >>>>>>> Large Frame Header Proposal >>>>>>> >>>>>>> This proposal is to alter the core http2 protocol to address the >>>>>>> issues identified above by supporting a variable length maximum frame size >>>>>>> controlled by peer limits. >>>>>>> >>>>>>> This proposal increases the length field in the frame header to 31 >>>>>>> bits, to match the maximum flow control window size. However, >>>>>>> implementations will not be able to use the full frame size without explicit >>>>>>> consent from peers using newly defined SETTINGS or an optional WINDOW_UPDATE >>>>>>> field. >>>>>>> >>>>>>> Frame Size Settings >>>>>>> >>>>>>> Two settings parameters are proposed: SETTINGS_HEADER_FRAME_SIZE for >>>>>>> the maximum header size and SETTINGS_FRAME_SIZE for all other frames. >>>>>>> >>>>>>> The SETTINGS_HEADER_FRAME_SIZE parameter supports the current >>>>>>> behaviour where large headers can be sent without changing the frame size >>>>>>> allowed for other frame types. ie A large header size limit can be set >>>>>>> without affecting the multiplexing efficiency of DATA frames. >>>>>>> >>>>>>> The SETTINGS_FRAME_SIZE applies to all other frames including DATA >>>>>>> frames and any other frame that may be defined by an extension. The use of >>>>>>> this parameter is intended to tune/optimise the connection for the general >>>>>>> case of multiple streams over the specific connection. >>>>>> >>>>>> >>>>>> Greg - >>>>>> >>>>>> Overall, great writeup. Thank you. >>>>>> >>>>>> Clarifying question: are you bundling these settings parameters >>>>>> because you think we need them? Or simply as a compromise? If you had to >>>>>> choose between a larger frame size and no settings change vs the current >>>>>> frame size, which would you find simpler? >>>>>> >>>>>> Thanks, >>>>>> Mike >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> Frame Size Updates >>>>>>> >>>>>>> To handle the issue of efficiently sending large data when an end >>>>>>> point is prepared to risk multiplexing efficiency, this proposal allows a >>>>>>> Max Frame Size to be applied to a specific stream as an optional field in a >>>>>>> WINDOW_UPDATE frame. >>>>>>> >>>>>>> By including a variable frame size in the flow control mechanism this >>>>>>> proposal allows the decision to increase the frame size to be deferred until >>>>>>> more knowledge about the specific situation are known and limited to the >>>>>>> stream that will benefit from the increased size. >>>>>>> >>>>>>> Consider the example of a server that has commenced sending a large >>>>>>> content to a client. The server may initially send 4 x 16KB frames to >>>>>>> consume the default stream flow control window, at which time it must wait >>>>>>> for the client to send a WINDOW_UPDATE frame before continuing. When >>>>>>> generating the WINDOW_UPDATE frame, the client may have knowledge of: >>>>>>> >>>>>>> The content-length header - so it knows that the amount of data >>>>>>> expected is large or is perhaps just slightly larger than a single frame. >>>>>>> The content-type header - so it knows if the content has high priority >>>>>>> in rendering the current page, or if the content is likely to include >>>>>>> references to other resources which may need to be multiplexed. >>>>>>> How many other streams are currently open and/or reserved - so it >>>>>>> knows if multiplexing is actually required. >>>>>>> How many other requests are pending - so it knows if new multiplexed >>>>>>> stream could soon be opened. >>>>>>> An approximate rough measure of the network latency and throughput - >>>>>>> This can be derived from the timing of the receipt of the first few data >>>>>>> frames and used to estimate the impact on QoS of any change to the maximum >>>>>>> frame size. >>>>>>> >>>>>>> The client can use this knowledge to make an informed decision as to >>>>>>> the benefit of changing the allowed max frame size against any risk to >>>>>>> multiplexing QoS. It can make several choices: >>>>>>> >>>>>>> No change. It can just not adjust the frame size, either because it is >>>>>>> too hard to consider or that there are too many other streams, or that the >>>>>>> content is video that needs to be received slowly. In any of these cases the >>>>>>> max frame size can be left unchanged and the protocol continues as it >>>>>>> currently does. >>>>>>> >>>>>>> Large frame. If the stream is the only expected stream or of >>>>>>> sufficiently high priortiy, then the window and frame size can be set to >>>>>>> allow as much of the remaining data as possible to be sent in a single >>>>>>> frame. >>>>>>> >>>>>>> Medium frame. The client can momentarily trade some QoS (for an >>>>>>> estimated duration) by increasing the frame size to something >16KB and < >>>>>>> content-length >>>>>>> >>>>>>> Sufficient frame. If the remaining content is only a small increment >>>>>>> over the current SETTING_FRAME_SIZE, the Max Frame Size can be increased to >>>>>>> receive the remaining content in a single frame without any significant QoS >>>>>>> impact. >>>>>>> >>>>>>> Minimal Compliance >>>>>>> >>>>>>> A minimally compliant implementation MUST handle the >>>>>>> SETTING_FRAME_SIZE and SETTINGS_HEADER_SIZE and ensure that no frame sent >>>>>>> exceeds the applicable limit. However no implementation is required to send >>>>>>> frames at or near these limits when set above the default 16KB. >>>>>>> >>>>>>> There is no requirement for an implementation to send or to handle the >>>>>>> Max Frame Size in a WINDOW_UPDATE and it is allowable for it to be ignored >>>>>>> if received. >>>>>>> >>>>>>> Anticipated Feedback >>>>>>> >>>>>>> It is too late in the process to change the framing layer and to do so >>>>>>> after so much discussion is an implicit fail of the WG >>>>>>> >>>>>>> To not consider issues and proposal brought to the WG would be a fail >>>>>>> of the process. This proposal is based on all the hard work to date done by >>>>>>> the WG and contributors to identify issues and test solutions. >>>>>>> >>>>>>> These issues can be handled in extensions. >>>>>>> >>>>>>> Optimising data transfers for large content could possibly be done in >>>>>>> an extension, however: >>>>>>> >>>>>>> It is not yet clear if extensions will be a viable way to enhance the >>>>>>> http2 protocol. There are significant hurdle to overcome to deploy >>>>>>> extensions. >>>>>>> Many of the issues are aimed at complexity and tuning of the core >>>>>>> protocol, and these cannot be addressed in an extension. >>>>>>> It is asymmetric to support large headers with one mechanism and large >>>>>>> data with another. >>>>>>> >>>>>>> >>>>>>> The proposed header costs 2 extra bytes per frame >>>>>>> >>>>>>> There is a small data cost to adopt this proposal, however this is >>>>>>> mitigated as: >>>>>>> >>>>>>> The proposal may be able to reduce the number of frames needed for >>>>>>> some content, thus saving 8 bytes. Whilst not likely to be a 25% frame >>>>>>> saving required to break even, it will still reduce cost to below 2 bytes. >>>>>>> There are options to have variable length headers or optional extended >>>>>>> headers that will preserve the semantics of this proposal and keep an 8 byte >>>>>>> header for small frames. If the 2 byte cost is considered prohibitive, then >>>>>>> these alterations can be considered. >>>>>>> >>>>>>> >>>>>>> The header is 10 bytes long and not 32bit word aligned. >>>>>>> >>>>>>> Frames sent after arbitrary data will not be word aligned anyway. If >>>>>>> alignment is important, then padding could changed to be part of the base >>>>>>> frame format, 2 header bytes used for a padding length (giving an aligned 12 >>>>>>> byte header) and all frames padded to a word boundary. >>>>>>> >>>>>>> 31 bits is also an arbitrary length >>>>>>> >>>>>>> It is true that a 31 bit large frame length is also an arbitrary limit >>>>>>> to the size of a frame. However, it is believed that 31 bits is sufficiently >>>>>>> large to efficiently handle almost all conceivable present and future use >>>>>>> cases. It would be possible to implement an unlimited size length field, but >>>>>>> this would also need changes to the flow control mechanism, which currently >>>>>>> also has a 31 bit maximum size. >>>>>>> >>>>>>> It does not support unlimited response headers >>>>>>> >>>>>>> A SETTINGS_HEADER_FRAME_SIZE of 2^31-1 is effectively unlimited for >>>>>>> all foreseeable response headers. >>>>>>> >>>>>>> >>>>>>> This was tried with SPDY and rejected >>>>>>> >>>>>>> SPDY did not have the settings to allow peers to set limits on the max >>>>>>> frame size. This proposal will not change to default behaviour of http2 with >>>>>>> regards to frame size. >>>>>>> >>>>>>> >>>>>>> Intermediaries will destroy multiplexing by setting frame size to >>>>>>> 2^31-1 >>>>>>> >>>>>>> Large frames require the participation of both sender and receiver. A >>>>>>> receiver may advise that it is willing to accept large frames, but a sender >>>>>>> is under no obligation to send them. Thus intermediaries nor any end point >>>>>>> can unilaterally change multiplexing QoS. >>>>>>> >>>>>>> >>>>>>> Cannot be hardware accelerated. >>>>>>> >>>>>>> Hardware acceleration is not part of the WG brief to support, nor is >>>>>>> it clear that this proposal is any less suitable than others for hardware >>>>>>> acceleration. >>>>>>> >>>>>>> Contributors >>>>>>> >>>>>>> This proposal was prepare by: >>>>>>> >>>>>>> Amos Jeffries squid3@treenet.co.nz >>>>>>> Greg Wilkins gregw@intalio.com >>>>>>> Jason Greene jgreene@redhat.com >>>>>>> Keith Morgan K.Morgan@iaea.org >>>>>>> Poul-Henning Kamp phk@phk.freebsd.dk >>>>>>> >>>>>>> [1] >>>>>>> http://lists.w3.org/Archives/Public/ietf-http-wg/2013AprJun/0926.html >>>>>>> [2] http://httparchive.org/interesting.php >>>>>>> [3] >>>>>>> http://lists.w3.org/Archives/Public/ietf-http-wg/2014AprJun/1664.html >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Greg Wilkins <gregw@intalio.com> >>>>>>> http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that >>>>>>> scales >>>>>>> http://www.webtide.com advice and support for jetty and cometd. >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Greg Wilkins <gregw@intalio.com> >>>>> http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that >>>>> scales >>>>> http://www.webtide.com advice and support for jetty and cometd. >>>> >>>> >>> >>
Received on Wednesday, 9 July 2014 02:17:30 UTC