Out-of-order delivery of chunked transfers

Hello.

I am seeking expert opinion on a candidate solution to a Real World Use 
Case with which I was recently presented.

TL;DR:

  * Would it be legitimate to extend HTTP/1.1 chunked transfer coding to
    support delivery of chunks out of order?
  * Is such an extension a useful thing to standardise?


  Use Case description

The Use Case involves transmitting Binary Large Objects (BLOBs) in a 
multicast FLUTE v1 <https://www.rfc-editor.org/rfc/rfc3926.html> session 
operating in carousel mode (i.e., FLUTE transmission objects in the 
session are sent repeatedly, possibly interleaved). The receiver is a 
gateway device offering an HTTP endpoint to downstream clients that want 
to acquire one or more transmission objects of interest to them.

Steps:

 1. A client requests an HTTP resource from the gateway, citing it's URL.
 2. The gateway joins the appropriate multicast group (based on some
    look-up mechanism not of concern here) and starts receiving FLUTE
    transmission objects, including the File Delivery Table
    <https://www.rfc-editor.org/rfc/rfc3926.html#section-3.2>
    transmission object.
 3. The gateway looks up the URL requested by the client in the File
    Delivery Table of the FLUTE session and this yields the Transmission
    Object Identifier that it needs to acquire.
 4. The gateway starts receiving FLUTE packets (well, actually ALC/LCT
    packets) for the identified transmission object and sends their
    payloads to the requesting client in the HTTP response body.

So far, so vanilla.

The twist is that the gateway has a low memory footprint, much smaller 
than the size of the BLOBs being transmitted in the FLUTE session. 
Hence, it cannot operate in a store-and-forward mode, and needs instead 
to deliver incoming data from the FLUTE session to its client in an 
efficient, pipelined fashion, minimising the need for buffering and 
anything beyond simple reassembly of received data blocks. Moreover, the 
client request to the gateway (step 1 above) arrives at a random time 
during the FLUTE session, in the worst case towards the end of a 
rotation of the carousel. In addition, the gateway may occasionally drop 
received FLUTE packets.


  Candidate solutions

My first instinct is always to try and reuse existing building blocks 
wherever possible, rather than inventing something new.


    HTTP/1.1

The *chunked transfer coding* feature that all HTTP/1.1 clients and 
servers are required to implement seems a good fit, except for one small 
detail: it assumes in-order delivery of chunks. Each chunk follows on 
from the last and is appended by the HTTP recipient to the previously 
received chunk (if any).

My idea is to include with every chunk a name–value pair (e.g. 
"offset=...") that additionally signals the byte position of this chunk 
in the overall resource representation. I reckon this could be achieved 
in the chunk delimiter using the *chunk-ext* production specified in 
section 7.1 of RFC 9112 
<https://www.rfc-editor.org/rfc/rfc9112#name-chunked-transfer-coding>. 
For example:

    1fb;offset="40d4"

With this additional signalling, the chunks could be sent on by the 
gateway to the client in whichever order they are received from the 
multicast FLUTE session, even if the gateway joins the multicast group 
partway through a carousel rotation. And the client could, for example, 
write received chunks to a sparse file using lseek() as they arrive in 
the HTTP response body. Happy days.

Once the entire transmission object has been received by the gateway and 
forwarded to the requesting client, the HTTP response body is terminated 
with the standard zero-length chunk. (More than one full rotation of the 
FLUTE carousel may be required to satisfy this termination condition in 
the presence of packet loss.)

In the absence of the proposed "offset=..." extension, a received chunk 
is assumed to belong immediately after the one previously received, as 
now, so the signalling is backwards-compatible with RFC 9112.


    HTTP/2

/(Not an immediate requirement, but I include an analysis for 
completeness.)/

This protocol version effectively uses chunked transfer for all message 
bodies, thanks to its binary framing. The HTTP/2 DATA frame includes 
length but not position. Being based on TCP, HTTP/2 client 
implementations are likely not expecting to receive DATA frames out of 
order. Since chunked transfer coding is explicitly prohibited by section 
8.1 of RFC 9113 
<https://www.rfc-editor.org/rfc/rfc9113.html#name-http-message-framing> 
it seems very difficult to achieve our of order delivery with this 
version of HTTP.


    HTTP/3

/(Not an immediate requirement, but I include an analysis for 
completeness.)/

Again, this protocol version natively uses chunked transfers. Similar to 
HTTP/2, the HTTP/3 DATA frame header signals the frame length. The 
position of this in the underlying QUIC stream is explicitly encoded in 
the /Offset/ field of the QUIC STREAM frame header, so there is 
potential to get cute with HTTP/3 DATA frame delivery order, provided 
the size of the HTTP/3 DATA frame headers (which can contain 
variable-length integers) is properly accounted for and no other frames 
are sent on that QUIC stream. I can't decide if that's elegant or evil. 
Maybe both at the same time.

Although QUIC streams are intended to offer an in-order bytestream 
abstraction to applications, being a UDP-based protocol, QUIC endpoints 
need to be robust to receiving datagrams out of order, to some extent. I 
note in particular the following in section 2.2 of RFC 9000 
<https://www.rfc-editor.org/rfc/rfc9000.html#name-sending-and-receiving-data>:

    "Endpoints MUST be able to deliver stream data to an application as
    an ordered byte stream. Delivering an ordered byte stream requires
    that an endpoint buffer any data that is received out of order, up
    to the advertised flow control limit.

    QUIC makes no specific allowances for delivery of stream data out of
    order. However, implementations MAY choose to offer the ability to
    deliver data out of order to a receiving application."

Hence, sending QUIC STREAM frames with unordered /Offset/ values seems 
to be a potentially viable solution to satisfy the Use Case that is sort 
of in keeping with the spirit of RFC 9000.

Whether HTTP/3 client implementations are robust to receiving DATA 
frames out of order is another matter, of course, but that would be a 
requirement for supporting this Use Case. (Unlike the 
resource-constrained gateway, the end client in this Use Case is less 
constrained.)