RE: Issue with "bytes" Range Unit and live streaming

Hi Craig, all,

Thanks for the clarification. Some further question inline

BR,
Thorsten

From: Craig Pratt [mailto:craig@ecaspia.com]
Sent: Monday, April 18, 2016 10:29 PM
To: Thorsten Lohmar; K.Morgan@iaea.org; fielding@gbiv.com
Cc: Göran Eriksson AP; bs7652@att.com; remy@lebeausoftware.org; ietf-http-wg@w3.org; rodger@plexapp.com; julian.reschke@gmx.de; C.Brunhuber@iaea.org; Darshak Thakore; STARK, BARBARA H
Subject: Re: Issue with "bytes" Range Unit and live streaming

[cc-ing the co-authors]

Hi Thorsten,

I'm happy to help provide whatever answers I can.

Reply in-line.

cp

On 4/18/16 8:10 AM, Thorsten Lohmar wrote:
Hi Craig, all,

My colleague Göran asked me some question around the problem and I would like to raise these questions directly to you. Of course, there are some alternative solutions available, where the client can work out the different things from a manifest. But you seem to look for a simple solution, which works with non-segmented media on a single HTTP session.

When I understood it correctly, an HTTP server is making a live stream available using HTTP. A normal live stream can be opened with a single HTTP request and the server can serve data "from the live point" either with or without HTTP chunked delivery. The server cannot give a Content-Length, since this is an ongoing live stream of unknown size.
[cp] all correct.


Your use-case seem to be about recording of content. Client should access content from the recorded part, but should be able to jump to the live-point. I assume that you are not looking into sliding window recordings (i.e. timeshift). I assume that the a single program is continuous recording and the HTTP object is growing until the end of the live session, correct?
[cp] I didn't spell it out in the draft, but I would like to consider adding clarifications for the time-shift cases. This should just be a matter of a Client requesting one thing and getting another. e.g. "Range: bytes-live=0-*" results in "Content-Range: bytes-live 123456-*". In either case, you're correct: the end of the representation is moving forward in time until the end of the live session.

[TL] The "Range: bytes-live=0-*" case is not clear to me. Your ID says "All bytes currently in the representation and those appended to the end of the representation after the request is processed". I get the impression, that the server is deleting all data before a certain timepoint (aka, behavior of a slighting window timeshift).  So, the client seems to request all data from the beginning of the timeshift buffer. Why does the server need to change the byte offset from 0 to 123456?
I can understand, that the server must signal "growing resource" in the response.

In any case, how does the client know "good" byte range offsets (i.e. service access points) to tune into the recording? Or is the assumption, that the client can synchronize to the media stream from any byte range?
[cp] For byte-level access, random access implementation is up to the client. For some containers this is easier than others. e.g. For MP4, the random access points can be established by reading the movie and fragment header(s). For something like MP2, it's trickier of course.
[TL] Well, in case of fMP4, the client needs to get the Movie Header for initialization. Then, proper access point are fragment boundaries. There are various ways to signal time to byte-offsets.
In case of TS, the client needs a PAT, PMT and PES starts for tune-in. It is a bit more tricky, but also here are solutions.
But the email talks about "none-segmented" media. The draft talks about "mimicking segmented media". fMP4 is actually the way to create ISO-BMFF segments. So, it is for segmented media, but without a separate manifest?


[cp] One major feature this draft allows is for retrieval of bytes just preceding the live point. So for example, a client can do a Range head request like "Range: bytes=0-", get a "Content-Range: bytes 0-1234567/*", then perform something like a "Range: bytes-live=1200000-*", and prime its framebuffer with 34567 bytes of data that precede the live point - allowing for the client to find an access point (e.g. mpeg2 start codes) and to allow live presentation to display much sooner than it would from the live point (without random access).
[TL] So, how does the client know, that the proper fragment boundary is at byte position 120000? Do you assume that the client first fetches a time-to-byte offset file, which tells the client that a access point (e.g. a fragment boundary) is at byte pos 120000? If yes, why does the client need the HEAD request, when it already has the byte position?


How should the client know, which byte ranges are already available on the server? When the client is playing back from the recorded part and would like to skip 5min forward, how does the client know, whether a normal range request is needed or whether the client should as for the live point? What type of HTTP Status code should be provided, when the range request is not yet available of the server?
[cp] We're not trying to come up with a universal solution for performing time-based seek on all media formats with this draft. So some of this is out of scope. But let me see if I can fill in some of the blanks.
[TL] Ok, not everything needs to be in-scope. But an essential assumption should be, whether the client has a time-to-byteoffset table or whether the client can determine precisely the fragment boundary positions.


[cp] Some applications of media streaming have time-based indexing facilities built-in. e.g. MP4 (ISO BMFF) containers allow time and data to be associated using the various internal, mandatory metadata "boxes". In other cases, applications may provide a separate resource that contains time-to-byte mappings (e.g. content index files). In either case, there's a facility for mapping time offsets to byte offsets - or sometimes the client incorporates heuristics to perform time skips (e.g. VLC will do this on some file formats).

[TL] Yes. fMP4 supports this and MPEG DASH is leveraging this. But the live-point is not described in the fragments. The client determines the livepoint from the manifest.

[cp] In all these cases, there's some mechanism that maps time offsets to byte offsets.
[TL] Yes


[cp] When it comes to the available byte range, a client can know what data range is available by utilizing a HEAD request with a "Range: bytes=0-". The "Content-Range" response can contain something like "Content-Range: bytes 0-1234567/*" which tells the client both the current randomly accessible content range (via the "0-1234567") and that the content is of indeterminate length (via the "*").
[TL] So, that is the existing Content-Range response, but with an '*' to indicate the unknown content-length, correct?


[cp] Putting this all together, a client would implement a 5-minute skip by:
    (1) Adding 5 minutes to your current play time,
    (2) determining the byte offset for that given time using the appropriate index/heuristic (e.g. "3456789"),
    (3) if the time is outside the index, jump to the live point and update the time to the last-index time or other means (e.g. using "Range: bytes-live=340000-*" to pre-buffer/pre-prime the frame/sample buffer),
    (4) if the time is inside the index, either perform a standard bytes Range request to retrieve an implementation-specific quantum of time or data (e.g. "Range: bytes=3456789-3556789") and render.
[TL] In (2), How does the client determine the byte offset? fMP4 requires precise byteoffset, In case of TS, the client can sync to the stream by first searching for 0x47 sync bytes. In (3), how does the client determine "outside of the index"? Seems that some sort of manifest is implicitly needed, which allows the client to understand the latest byte pos.



[cp] Again, some of this is out of scope, but I hope that clarifies a common use case.
[TL] Would be good to clarify, what information the client needs to get in order to do the operations. How the client gets the info can be left out-of-scope.


[cp] Regarding the status code, RFC7233 (section 4.4) indicates that code 416 (Range Not Satisfiable) must be returned when "the current extent of the selected resource or that the set of ranges requested has been rejected due to invalid ranges or an excessive request of small or overlapping ranges." This part of 4.4 applies to *all* Range requests - regardless of the Range Unit.
[TL] ok.


[cp] The bytes-live draft then goes on to say that "A bytes-live-range-specifier is considered unsatisfiable if the first-byte-pos is larger than the current length of the representation". This could probably be elaborated on a bit. But this is supposed to be the "hook" into the 4.4 language.


Can you please clarify the questions?
[cp] I hope I succeeded (at least partially). Apologies for the long response. I wanted to make sure I was answering your questions.
[TL] Gets a bit clearer, but I still don't understand the "mimic HLS or DASH". DASH / HLS focuses on CDN optimization by creating a sequence of individual files. The client can work out the live-point URL from the manifest. Each segment is a "good" access point (in DASH always box boundaries and in HLS always TS boundaries even with PAT / PMT). So, the key issue here is to clarify, how the client gets the byte offsets of the fragment boundaries for range requests.




BR,
Thorsten



From: Craig Pratt [mailto:craig@ecaspia.com]
Sent: Monday, April 18, 2016 11:04 AM
To: K.Morgan@iaea.org<mailto:K.Morgan@iaea.org>; fielding@gbiv.com<mailto:fielding@gbiv.com>
Cc: Göran Eriksson AP; bs7652@att.com<mailto:bs7652@att.com>; remy@lebeausoftware.org<mailto:remy@lebeausoftware.org>; ietf-http-wg@w3.org<mailto:ietf-http-wg@w3.org>; rodger@plexapp.com<mailto:rodger@plexapp.com>; julian.reschke@gmx.de<mailto:julian.reschke@gmx.de>; C.Brunhuber@iaea.org<mailto:C.Brunhuber@iaea.org>
Subject: Re: Issue with "bytes" Range Unit and live streaming

On 4/18/16 12:34 AM, K.Morgan@iaea.org<mailto:K.Morgan@iaea.org> wrote:

On Friday,15 April 2016 22:43, fielding@gbiv.com<mailto:fielding@gbiv.com> wrote:

Oh, never mind, now I see that you are referring to the second number being

fixed.



I think I would prefer that be solved by allowing last-byte-pos to be empty, just

like it is for the Range request.  I think such a fix is just as likely to be

interoperable as introducing a special range type (same failure cases).



....Roy



+1000



A very similar idea was proposed before [1] as an I-D [2] by Rodger Coombs. We've also brought this up informally with other members of the WG.



Alas, in our experience range requests don't seem to be a high priority :(  For example, the problem of combining gzip with range requests is still unsolved [3].



[1] https://lists.w3.org/Archives/Public/ietf-http-wg/2015AprJun/0122.html

[2] https://tools.ietf.org/html/draft-combs-http-indeterminate-range-01

[3] https://lists.w3.org/Archives/Public/ietf-http-wg/2014AprJun/1327.html
[cp] Yeah, it's unfortunate that no solutions have moved forward for this widely-desired feature. I can only assume that people just started defining proprietary solutions - which is unfortunate. I'll try to be "persistent"... ;^J

[cp] As was mentioned, the issue with just arbitrarily allowing an open-ended Content-Range response (omitting last-byte-pos) is that there's no good way for a client to indicate it can support reception of a Content-Range without a last-byte-pos. So I would fully expect many clients to fail in "unpredictable ways" (disconnecting, crashing, etc).

[cp] I see that the indeterminate length proposal you referenced in your first citation introduces a "Accept-Indefinite-Ranges" header to prevent this issue. But I think this brings with it some other questions. e.g. Would this apply to any/all Range Units which may be introduced in the future? How can a Client issue a request that starts at the "live point"? It feels like it has one hand tied behind its back.

[cp] If I could, I would prefer to go back in time and advocate for an alternate ABNF for the bytes Range Unit. Seeing as that's not an option, I think using this well- and long-defined Range Unit extension mechanism seems like a good path forward as it should not create interoperability issues between clients and servers.

[cp] And I would hope adding a Range Unit would have a low/lower bar for acceptance. e.g. If a Range Unit fills a useful role, is well-defined, and isn't redundant, it seems reasonable that it should be accepted as it shouldn't impact existing HTTP/1.1 semantics. In fact, the gzip case (referenced in your third citation) seems like a perfect application of the Range Unit (better than bytes-live). If there's interest, I'll write up an RFC to demonstrate...







This email message is intended only for the use of the named recipient. Information contained in this email message and its attachments may be privileged, confidential and protected from disclosure. If you are not the intended recipient, please do not read, copy, use or disclose this communication to others. Also please notify the sender by replying to this message and then delete it from your system.

--



craig pratt

Caspia Consulting

craig@ecaspia.com<mailto:craig@ecaspia.com>

503.746.8008




--


craig pratt

Caspia Consulting

craig@ecaspia.com<mailto:craig@ecaspia.com>

503.746.8008