Re: HTML5 Last Call May 2011 & DASH/Adaptive Streaming

On Feb 18, 2011, at 12:22 AM, Jan Lindquist wrote:

Hi Mark,

I made some comments below on your responses.

Regards,
JanL

________________________________
From: Mark Watson [mailto:watsonm@netflix.com]
Sent: den 17 februari 2011 18:42
To: Jan Lindquist
Cc: Glenn Adams; Philippe Le Hegaret; Ali C. Begen (abegen); Silvia Pfeiffer; Jean-Claude Dufourd; Richard Maunder (rmaunder); public-web-and-tv@w3.org<mailto:public-web-and-tv@w3.org>
Subject: Re: HTML5 Last Call May 2011 & DASH/Adaptive Streaming


On Feb 17, 2011, at 4:59 AM, Jan Lindquist wrote:

Hi,

If I may piggyback this thread I did a comparison between the video element to what we have done in OIPF DAE specification. Here are some observations and differences. Copy of the DAE spec can be found at:
http://www.oipf.org/docs/Release2/OIPF-T1-R2-Specification-Volume-5-Declarative-Application-Environment-v2_0-2010-09-07.pdf

The purpose of sharing the information is to compare notes and try to find gaps that can be filled in HTML5. The focus is on HTTP adapative streaming.

Short Comparison:
1. The support of media and text tracks seem to be equivelent to what we have defined as AV Components (refer to section 7.16.5 in DAE spec). The media tracks has a different logic than what was done in DAE so several attributes in tracks element are not applicable. In DAE the components are treated as one source while video element allows it to be seperated as individual "streams". Might not be clear how a media element readyState supercedes that of the track element states.

2. The configuration of the terminal default setting is not available (refer to section 7.3.2.1 in DAE spec). Properties preferredLanguage and preferredSubtitleLanguage. This is more of a device management than video element issue.

3. Support of HTTP adapative streaming (DASH or equivelent) is supported in DAE, unfortunately it is not published and available to the generic public (available in 2 months). In order to share it a liaison has to be initiated towards OIPF with the request.
- At a high level we have means of reporting changes in quality for a represetation or a new period. The quality is the form of bandwidth according to the manifest 3GPP HAS (have not checked DASH equivelent).

Stream bitrate on it's own is not very useful unless you specify how it is calculated (average, peak, over what time window etc.). There may also be multiple streams with the same bitrate which differ in some other respect (codec, resolution, fps, etc.).
[JLI>] In the 3GPP manifest there is an attribute called bandwidth which is a representative value of what is expected to be the average bandwidth.
It's not an average bandwidth. There is a detailed definition of what it means and it involves the combination of the bandwidth and the minBufferTime. It's more of a peak, but it's not exactly that either.
So there is no calculation but it is a means to quantify the different streams. If an id is used then the application receiving the events needs to be aware of what the id represents. I would hope there is something in the manifest to help in this respect.
Yep, I think the application needs to know what the ids mean, but since the application and the content come from the same source (the service provider) that should not be a problem. (If they don't, I'm not sure what use the stream change events would be anyway?).

In DASH every representation has an @id attribute. It would make sense to report this at stream changes.

What to do about Periods (high level division in time of the content) requires some discussion as it is a DASH-specific concept. Perhaps some way to map to a generic timed-metadata reporting mechanism on the video tag ?
[JLI>] Are you referring to the text track that would contain the metadata? The problem in adapative stream is that the manifest may not contain the same information as the actual stream during playout. Period changes contains several details, my focus was on "bandwidth" changes. Other information that may change in a new period is available audio tracks and subtitles. It is not clear how the metadata for the manifest would then be presented against the actual values during playout.  In DAE we limited the change in period to only bandwidth while audio and subtitle tracks are reported during playout. Note we are looking at mpeg2-ts. It is not clear how the timed-metadata reporting will work in the video tag.
I was suggesting that if there was a general-purpose timed metadata reporting mechanism on the video tag (which would be useful for, err, timed metadata ;-), then this could be re-used to report Period changes as if they were a kind of timed metadata event. This would avoid embedding the DASH-specific Period concept into HTML.

As you mention, the available tracks can change at a Period boundary, which suggests another requirement for the Multi-track API that there is an event to indicate when the available tracks have changed.

- There is a means of retrieving the available representations (from min to max) and the current representation.

I'm not sure why this would be needed ?
[JLI>] This is the range of streams and the available "bandwidths". So you get a list of what is available and which one is being used.

By why? What would you use that list for ?

...Mark

...Mark


Regards,
JanL

________________________________
From: public-web-and-tv-request@w3.org<mailto:public-web-and-tv-request@w3.org> [mailto:public-web-and-tv-request@w3.org] On Behalf Of Mark Watson
Sent: den 16 februari 2011 07:11
To: Glenn Adams
Cc: Philippe Le Hegaret; Ali C. Begen (abegen); Silvia Pfeiffer; Jean-Claude Dufourd; Richard Maunder (rmaunder); public-web-and-tv@w3.org<mailto:public-web-and-tv@w3.org>
Subject: Re: HTML5 Last Call May 2011 & DASH/Adaptive Streaming

What I think would be useful, though, would be to specify that *if* DASH (say) is supported by an HTML5 video implementation *then* is should be done in a specific way.

Otherwise we have the prospect of multiple different implementations supporting different things.

For the examples you give below, it is at least clear what you support if you support a certain version of ECMAScript, say.

For DASH used in HTML5 a minimal set of things that should be nailed down are:
- that a URL for a DASH manifest can be provided in the @src or <source> element
- what error should be reported if the manifest is invalid, or if the media it points to cannot be found
- how to label DASH tracks so that the different kinds of track map to the Multi-Track API (or whatever is agreed for ISSUE-152) and a single way to do the mapping
- what event to use to report automatic bitrate changes

This is pretty basic stuff, but it would help if everyone that does it, does it the same way.

...Mark


On Feb 15, 2011, at 5:33 PM, Glenn Adams wrote:

For that matter, HTML5 does not require a UA to support either the HTML syntax of HTML5 or the XML syntax of HTML5. A compliant HTML5 UA could support neither (albeit with little utility).

HTML5 does *not* specify:

 *   which top-level document types must be supported
 *   which version of ECMAScript must be supported
 *   which version/modules of CSS must be supported
 *   which version/modules of DOM must be supported
 *   which URI schemes (protocols) must be supported
 *   which non-streaming media types must be supported
 *   which streaming media types must be supported

HTML5 is a technology framework specification that requires other specifications (profiles) to fill in these gaps. In the context of certain TV standardization activities, such work to define one or more profiles is already underway.

N.B. I am not criticizing HTML5 for not making such choices. In fact, I think the editor and group has taken the best approach in that regard.

G.

On Tue, Feb 15, 2011 at 5:59 PM, Philippe Le Hegaret <plh@w3.org<mailto:plh@w3.org>> wrote:
On Tue, 2011-02-15 at 18:40 -0500, Ali C. Begen (abegen) wrote:
> I think folks need to agree on the container format not the codec type. A good container format will be good for several codecs that exist today and will yet to come.

My understanding is that the IP issues surrounding the codec types are
also surrounding the container formats and the streaming technologies.
So, I'd be surprised if any agreement was reached within the HTML
Working Group on those topics. I can't imagine a different conclusion
that the H.264/Theora discussion at this point. In any case, as Glenn
alluded to, HTML has been technology neutral since the beginning. Unless
I'm mistaken, we don't require implementations to support a specific
image format.

Philippe