Re: Review of section 7

On Tue, Apr 14, 2009 at 7:36 PM, Yves Lafon <ylafon@w3.org> wrote:
> On Mon, 13 Apr 2009, Silvia Pfeiffer wrote:
>
>>>> * we need to mention that the replies are not cachable for the full
>>>> resource, but cache only the fragmented resource, which can result in
>>>> n copies of some subset of the resource in caching web proxies. Since
>>>> it is typically expected that a fragment request will be used
>>>> repeatedly (e.g. a fragment URI being shared around), this is
>>>> potentially acceptable. But we have no experimental data to confirm or
>>>> deny that this may pose a challenge to caching web proxies.
>>>
>>> Why? it is the case only if the cache can't handle merging timed
>>> fragment,
>>> which is the case of general purposes caches (they will handle only
>>> bytes),
>>> but media procies might be able to do so. Once again, it is not sure that
>>> they will do that as requesting the whole resource and serve fragments
>>> from
>>> the cache might be more efficient for such caches anyway :)
>>
>> What I meant to say with this is: when every media fragment URI on a
>> parent resource returns a file that is not a byte-identical subsegment
>> of the parent resource, the cache has to eventually store a full copy
>> of the original file and n fragmented versions. The only media format
>> that would be capable in the single-GET request case to return
>> byte-identical subsegments is one that does not require any header
>> changes to return a valid resource. I think this is the case for MP3
>
> You mean retrieving the whole resource? or by header you say "data format
> header" (the latter I think)

No, not retrieving the whole resource. You can cut an MP3 file at any
point in time without destroying it's integrity - it will still play
in any MP3 player, because there are constantly re-sync headers
throughout the file format and these need not be adjusted. Thus, there
are byte-idnetical subsegments.


>> and MPEG-1 for the temporal dimension. I don't think this is the case
>> for many other formats. Even for MP3 an MPEG-1, I don't think they are
>> capable of storing that they are extracted from a parent resource at a
>> certain in and out time.
>
> Ok, so you want to rule out completely extracting a fragment from a
> resource.

No, not at all. I am just saying that using the 1-GET request method
implies that most - if not all - fragmented media resources will not
be cachable in Web proxies.


>> My point therefore was: where fragmented versions are not a
>> byte-identical subpart of the parent resource, merging fragmented
>> versions is basically impossible, even for intelligent media proxies,
>> simply because splicing in the compressed domain (and we may talk
>> re-compressed here) is a futile effort. Therefore they will end up
>
> Well, of course if it is re-compressed, it is impossible to merge things,
> unless it is a lossless compression, as in that case you can
> compress/recompress without changing the content. I can't see lossless video
> anytime soon ;)

OK. All I asked for is to clarify this in the text, since this wasn't
clearly stated.


>> with n copies, unless they trick the system and (as you say) download
>> the full resource, but do the fragmenting themselves, which defeats
>> the purpose of media fragments for the first time a fragment request
>> is being undertaken (think about a 5 hour long video where the client
>> requests the last 2 minutes).
>
> Why are you saying that it defeats fragmenting? My point was that for some
> proxies, the one that are likely to really have to deal with lots of traffic
> and content (mostly reverse proxies), the cost of transfer was pretty low
> (fast pipes), and even getting the ful 5h video to extract 2mn might be
> efficient. I am not talking about the proxy on someone's local network
> behind a 28.8kbps modem.

Sure, neither am I.

Why are you talking about reverse proxies? Why not just have that
functionality in the Web server itself? In fact, I was not referring
to a reverse Web proxy.

I was talking about caching Web proxies. So let's say the 5h long
video takes up 3GB. Across a 100MBit pipe that we can flood with the
video, it will take 4min to get that video into the Web proxy, where
it then does the extraction and transfers the last 2 min. I don't
think that is acceptable. That is what I meant by "defeating the
purpose of fragmenting" because the idea of fragment delivery is to
avoid the long delay required by downloading the full resource (even
if this delay is smaller since it's not across a low-bandwidth
consumer link, but a higher bandwidth Internet link).


>>>> * further we need to mention that there are encapsulation formats
>>>> (such as MP4) that have an index in their headers that can provide a
>>>> seconds->byte mapping for any offset. For such cases, an optimisation
>>>> is possible where a client retrieves the index part of the media file
>>>> first and then uses that to make byte range requests rather than time
>>>> range requests. These requests are cachable by any caching Web proxy
>>>> without replication for differing fragment requests. This is in fact a
>>>> third means of retrieving fragment URIs, which we might want to make
>>>> explicit as a section 7.3.
>>>
>>> This is needed for both options, btw.
>>
>> Which both options? I am confused. I thought about this as an
>> alternative to the single-GET and the two-GET request.
>
> I wanted to say that having a way to do the seconds->byte mapping is always
> needed, I read too fast that it was in the header :)

OK. :-)


> So you propose that every client should do
> GET http://example.com/fragf2f.mp4 HTTP/1.1
> Accept: video/*
> Range: bytes=0-65535 (or anything big enough to get the data headers)
>
> then based on what is retrieved, do another partial GET based on what the
> client mapped between the seconds and the bytes, right ?

It depends how far we want to take this. If we decide the best means
to deal with such special formats is to always do a request on the
beginning of the file to retrieve the index part before we do anything
else, that's possible. I was thinking more about the possibility for a
User Agent to do an internal improvement once it had received the
index in a previous retrieval request. Any consecutive media fragment
URI request can use this knowledge to avoid the need to do a seconds
range request, but to directly do a bytes range request.


> It should already work now without changing anything, without using a
> X-Accept-Range-Redirect header.

You're probably right. Ultimately it is an optimisation that the User
Agent can make and doesn't need to be included in this standard. We
can however point out that for some formats there is the possibility
to identify the byte range mapping from their header and that a User
Agent can optimise their approach for this.

Cheers,
Silvia.

Received on Tuesday, 14 April 2009 13:04:55 UTC