- From: Bernard Aboba <Bernard.Aboba@microsoft.com>
- Date: Tue, 3 Jul 2018 23:59:52 +0000
- To: Iņaki Baz Castillo <ibc@aliax.net>
- CC: WebRTC WG <public-webrtc@w3.org>, Robin Raymond <robin@opticaltone.com>
- Message-ID: <MW2PR00MB03295C5B603FAFBB8A2F6522EC420@MW2PR00MB0329.namprd00.prod.outlook.com>
Inaki said:
"BTW, IMHO RID should not be used to identify SVC layers and/or
relationship between them. RID means RtpStreamId, which is basically a way to identify
media streams by looking first at its RID header extension value (so the SSRC can
change at any time due to SSRC conflits and bla-bla-bla that never happen).
AFAIK SVC layers are carried by a single stream (single SSRC) so RTP
packets of any SVC layer must also have the same RID value (if used)."
[BA] I agree that RID should not be used to identify SVC layers in SRST encodings (VP8, VP9, AV1, most H.264/SVC implementations). Let's assume that MRST is out of scope for WebRTC 1.0 so we can avoid having to open that can of worms :)
I wouldn't reuse the `rid` or `encodingId` (in ORTC) setting for SVC
stuff. In the "Example of 2-layer spatial simulcast combined with 2-layer temporal
scalability" example in your first email, there are 4 rid values (L0, E0, L1, E1).
One may assume that there will be 4 RtpStreamId values in the wire, but it's not true. In fact,
I assume there would be just two (L0 and E0), but this is terribly
confusing IMHO.
[BA] In that example, I think there are only two rid values since there are only two simulcast streams. Within a given simulcast stream, each SVC layer should utilize the same SSRC and rid.
I wouldn't signal SVC layers as separate entries within
`RTCRtpParameters.encodings`. Instead, I'd propose that entries in `RTCRtpParameters.encodings` represent a
separate stream (so different SSRC and rid/encodingId, period).
And, I would add new fields within RTCRtpEncodingParameters that define the SVC
layers within *that* stream. Something like:
var encodings = [
{
rid: "L0",
temporalLayers: [
{
scaleFramerateDownBy: 2.0
},
{
scaleFramerateDownBy: 1.0
}
],
spatialLayers: [
{
scaleResolutionDownBy: 4.0
},
{
scaleResolutionDownBy: 2.0
},
{
scaleResolutionDownBy: 1.0
}
]
}
];
[BA] If the desire is to make things as simple as possible, could we do something like this?
var encodings = [
{
rid: "L0",
temporalLayers: 2,
spatialLayers: 3
}
The advantage of this approach is that the developer can't choose a nonsensical value of scaleFramerateDownBy.
Obviously, each entry within temporalLayers and spatialLayers depends
on the previous ones. If, instead, more complex use cases are required (such as more
complex layers relationships), an `id` and `dependencyIds` fields should be needed.
[BA] I'm not sure how often developers will need to set up more complex layer relationships. So far, most developers I have talked to have little or no interest in doing that (they are mostly using the defaults implemented in Chrome today).
________________________________
From: Iņaki Baz Castillo <ibc@aliax.net>
Sent: Tuesday, July 3, 2018 4:41:43 PM
To: Bernard Aboba
Cc: WebRTC WG; Robin Raymond
Subject: Re: Some ideas on SVC support in WebRTC 1.0 (Take 2)
Hi Bernard,
This approach is much better than the previous one (since it decouples both rid and encodingId). I'm just not sure whether it's too error prune (since each SVC layer is indicated as a new entry within RTCRtpParameters.encodings).
Can you please check my last response in the previous thread?
On Wed, 4 Jul 2018 at 01:35, Bernard Aboba <Bernard.Aboba@microsoft.com<mailto:Bernard.Aboba@microsoft.com>> wrote:
Here is a revised proposal for SVC support in WebRTC 1.0 (with some implied changes to ORTC):
Assumptions
1. No need for negotiation. If an RTCRtpReceiver can always decode any legal SVC bitstream sent to it without pre-configuration, then there is no need for negotiation to enable sending and receiving of an SVC bitstream. This assumption, if valid, enables addition of SVC support by addition of attributes to encoding parameters.
2. Codec support. Today several browsers have implemented VP8 (temporal) and VP9 (temporal, with spatial as experimental) codecs with SVC support. It is also expected that support for AV1 (temporal and spatial) will be added by multiple browsers in future. AFAIK, each of these codecs fits within the no-negotiation model (e.g. in AV1, support for SVC tools are required at all profile levels). As a side note, I believe that H.264/SVC could also be made to fit within the no-negotiation model if some additional restrictions are imposed upon implementations (such as requiring support for UC mode 1, described here: http://www.mef.net/resources/technical-specifications/download?id=106&fileid=file1<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.mef.net%2Fresources%2Ftechnical-specifications%2Fdownload%3Fid%3D106%26fileid%3Dfile1&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201113685&sdata=oHmJilXo01KOjx%2BWrZe0ggkoQZcQkYqGwXn0Rlma%2F3A%3D&reserved=0> ). So the no-negotiation model seems like it would allow support for current and potential future SVC-capable codecs.
3. Single SSRC/rid for all SVC layers. VP8, VP9, AV1 and most implementations of H.264/SVC send all SVC layers with the same SSRC. If we can assume that all SVC layers of a simulcast stream will utilize the same SSRC and rid, this provides some simplification because it means that the rid attribute is used *only* to differentiate simulcast streams. See below for the effect of this within examples.
General approach
Today, the encoding and decoding parameters appear as follows:
dictionary RTCRtpCodingParameters<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw3c.github.io%2Fwebrtc-pc%2F%23dom-rtcrtpcodingparameters&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201113685&sdata=phWYWzCu6ZYii3ZWk7nKDwGLKxpsJVZgDgFa83iR%2FvU%3D&reserved=0> {
DOMString<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fheycam.github.io%2Fwebidl%2F%23idl-DOMString&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201123695&sdata=xs%2Bkj3nppzFdUdVzItOS%2F8GE1mJ9iO36nacsHJCmy4I%3D&reserved=0> rid<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw3c.github.io%2Fwebrtc-pc%2F%23dom-rtcrtpcodingparameters-rid&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201123695&sdata=ULbaYn9Fj8hZ%2FOvBAUaunEiwtGccnkVPfhQVcmqttbE%3D&reserved=0>;
};
dictionary RTCRtpDecodingParameters<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw3c.github.io%2Fwebrtc-pc%2F%23dom-rtcrtpdecodingparameters&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201133700&sdata=XOHDIQFHyUFSQvzwz9x5gVkBqGTcqJ2QyIsZREJyvJM%3D&reserved=0> : RTCRtpCodingParameters<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw3c.github.io%2Fwebrtc-pc%2F%23dom-rtcrtpcodingparameters&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201133700&sdata=urj%2FSzSuIT3K1KOFjy4YedThPXd2dXIzUIXsoI5KRQk%3D&reserved=0> {
};
dictionary RTCRtpEncodingParameters<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw3c.github.io%2Fwebrtc-pc%2F%23dom-rtcrtpencodingparameters&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201143709&sdata=C8su%2FOpY2s3dsqIMBc3YKNMT2QymVQHxk5ngsrEu3oo%3D&reserved=0> : RTCRtpCodingParameters<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw3c.github.io%2Fwebrtc-pc%2F%23dom-rtcrtpcodingparameters&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201143709&sdata=0p7bDYZBaE%2F7d9yGBJ0qqC%2F0tnYylamdYq2u4vm%2BrOY%3D&reserved=0> {
octet<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fheycam.github.io%2Fwebidl%2F%23idl-octet&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201153714&sdata=F235h4dSrGv7xq06PKcESkns2YP4Exf%2FYCfy2xtpZQo%3D&reserved=0> codecPayloadType<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw3c.github.io%2Fwebrtc-pc%2F%23dom-rtcrtpencodingparameters-codecpayloadtype&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201153714&sdata=4f7gB2uOFRYuCUaulDaU3YNjVLg0VA9Lhh%2Bs9gPXhjg%3D&reserved=0>;
RTCDtxStatus<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw3c.github.io%2Fwebrtc-pc%2F%23dom-rtcdtxstatus&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201153714&sdata=Vg5JCJctOysJuSI53UDwS%2F5CG1KGC38%2FxR1Q%2FuCrFgI%3D&reserved=0> dtx<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw3c.github.io%2Fwebrtc-pc%2F%23dom-rtcrtpencodingparameters-dtx&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201163723&sdata=hn9A7Szg4ba%2Bfg2zizMWUce6tmx30WkP3%2B%2BB%2BAKJigU%3D&reserved=0>;
boolean<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fheycam.github.io%2Fwebidl%2F%23idl-boolean&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201163723&sdata=o2r%2B7e7JtCuu4Qqg%2BoeV%2FWtC9I4BAdtng5DIMeHyJ6Y%3D&reserved=0> active<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw3c.github.io%2Fwebrtc-pc%2F%23dom-rtcrtpencodingparameters-active&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201173728&sdata=ryo82FipXtUZRQBg7%2Brtcl%2B3sW0ENVY3qVcmmJjlRoc%3D&reserved=0> = true;
RTCPriorityType<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw3c.github.io%2Fwebrtc-pc%2F%23dom-rtcprioritytype&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201173728&sdata=bZ9nL5W%2BoUnRBKn9q95SLLa9Gow600F%2BJFOkZr%2BLp5w%3D&reserved=0> priority<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw3c.github.io%2Fwebrtc-pc%2F%23dom-rtcrtpencodingparameters-priority&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201183733&sdata=RdtCfDmRFN3LfZUCpiKYvPBGNiESq%2BjdzLPVlB5Iv0M%3D&reserved=0> = "low";
unsigned long<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fheycam.github.io%2Fwebidl%2F%23idl-unsigned-long&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201183733&sdata=Ka8KcfzEeUKQNvLu%2FRipUnuOg7vpPMOppYUfTegiDhs%3D&reserved=0> ptime<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw3c.github.io%2Fwebrtc-pc%2F%23dom-rtcrtpencodingparameters-ptime&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201193743&sdata=i9SKrDZhxoHtaE%2Byc%2FvGGj3yMyDxLGSe4j9Yrlami18%3D&reserved=0>;
unsigned long<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fheycam.github.io%2Fwebidl%2F%23idl-unsigned-long&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201193743&sdata=lNjKq8duLnOst%2FT%2Bpu3aEoRNUfxbWLAZ39OfqwzpvZU%3D&reserved=0> maxBitrate<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw3c.github.io%2Fwebrtc-pc%2F%23dom-rtcrtpencodingparameters-maxbitrate&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201203752&sdata=6H1luDP%2FJ5%2BcdrErFJ8f%2FtNTRckXG3jX%2FY3SuUmMSu4%3D&reserved=0>;
double<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fheycam.github.io%2Fwebidl%2F%23idl-double&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201203752&sdata=TvkIr4ujXYmz2hRJt8A4M6wKhT2JUJfXzeyLxPx3UJ4%3D&reserved=0> maxFramerate<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw3c.github.io%2Fwebrtc-pc%2F%23dom-rtcrtpencodingparameters-maxframerate&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201213757&sdata=aWJJMCrFxJVAFRvs2Bbl%2FdCembnei0p08WmKbXNVYT8%3D&reserved=0>;
double<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fheycam.github.io%2Fwebidl%2F%23idl-double&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201213757&sdata=UYKj46p%2BZkO9bTbhruGbg8RzRKJuGSH4GRQTT7dQj5s%3D&reserved=0> scaleResolutionDownBy<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw3c.github.io%2Fwebrtc-pc%2F%23dom-rtcrtpencodingparameters-scaleresolutiondownby&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201223766&sdata=ZeeVR3RDLUkfzoilsgCSvVLhV0Lc%2FG7bCi3cvkL7J58%3D&reserved=0>;
};
To provide SVC support, we proposed to add the following additional encoding parameters:
partial dictionary RTCRtpEncodingParameters<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdraft.ortc.org%2F%23dom-rtcrtpcodingparameters&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201223766&sdata=pTGQzrqePWpXlnaajnKeYimFMpE3BOuaWIHBRM%2BxCGQ%3D&reserved=0> {
d<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fheycam.github.io%2Fwebidl%2F%23idl-boolean&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201233771&sdata=NYyijVF%2Fhf8ri7lGiDArvyzx%2Fi1GqB9%2FOuTL%2FqMsQtw%3D&reserved=0>ouble scaleFramerateDownBy;
DOMString<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fheycam.github.io%2Fwebidl%2F%23idl-DOMString&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201233771&sdata=8kHmIZAqXdiMJc4mscp%2FXMKoKZoyxTo7y2VF7181x14%3D&reserved=0> encodingId<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdraft.ortc.org%2F%23dom-rtcrtpcodingparameters-encodingid&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201243785&sdata=yU75ibMPicbOwLXrH9CQoqxBX2do%2FtReYZFHfT%2B6w3E%3D&reserved=0>;
sequence<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fheycam.github.io%2Fwebidl%2F%23idl-sequence&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201243785&sdata=mdgvuZHNJZ3wYnLQ%2Fh73%2FbFSF5p60RWlajabPDs%2BtIM%3D&reserved=0><DOMString<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fheycam.github.io%2Fwebidl%2F%23idl-DOMString&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201253785&sdata=%2FyxCBwc0WiIQI3JdcmVFgtYxSHFeSFpznOEUMmMSZao%3D&reserved=0>> dependencyEncodingIds<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdraft.ortc.org%2F%23dom-rtcrtpcodingparameters-dependencyencodingids&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201253785&sdata=2molUrf4BEy11ycOLnPxr0v%2FlUC3ddIGHBEKUXQq%2FtU%3D&reserved=0>;
};
encodingId of type DOMString
An identifier for the encoding object. This identifier should be unique within the scope of the localized sequence of RTCRtpCodingParameters<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdraft.ortc.org%2F%23dom-rtcrtpcodingparameters&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201263795&sdata=RHQvAn3surj%2FGNKszG5RuionxCSHfltgW13%2BtIHknJ0%3D&reserved=0> for any given RTCRtpParameters<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdraft.ortc.org%2F%23dom-rtcrtpparameters&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201263795&sdata=XxJrX89Prmy2uJcbg0oQPILiOiliaL8nC8uJIISfKQg%3D&reserved=0> object. Values MUST be composed only of alphanumeric characters (a-z, A-Z, 0-9) up to a maximum of 16 characters.
scaleFramerateDownBy of type double
Inverse of the input framerate fraction to be encoded. Example: 1.0 = full framerate, 2.0 = one half of the full framerate. For scalable video coding, scaleFramerateDownBy refers to the inverse of the aggregate fraction of input framerate achieved by this layer when combined with all dependent layers.
dependencyEncodingIds of type sequence<DOMString>
The encodingId<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdraft.ortc.org%2F%23dom-rtcrtpcodingparameters-encodingid&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201273800&sdata=z%2FQobOKgy4aLmcQbRafazpdTMG7K3hiTD89URAkeSnM%3D&reserved=0>s on which this layer depends. Within this specification encodingId<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdraft.ortc.org%2F%23dom-rtcrtpcodingparameters-encodingid&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201273800&sdata=z%2FQobOKgy4aLmcQbRafazpdTMG7K3hiTD89URAkeSnM%3D&reserved=0>s are permitted only within the same RTCRtpCodingParameters<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdraft.ortc.org%2F%23dom-rtcrtpcodingparameters&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201283804&sdata=3EPGYlikPGR5AqmnP1nmAxwe2csg7s9W%2BTV48n%2F%2B41I%3D&reserved=0> sequence. In order to send scalable video coding (SVC<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdraft.ortc.org%2F%23dfn-svc&data=02%7C01%7CBernard.Aboba%40microsoft.com%7Cf225fe1a7e984ce48c7008d5e13e9043%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636662581201283804&sdata=oGUIkz%2FYeC7yVBdV2gqFMH9YsHDqncW6RyXajMEUvaI%3D&reserved=0>), both the encodingId anddependencyEncodingIds are required.
Here is an example of how this would work for a single temporal encoded stream (without simulcast):
// Example of 3-layer temporal scalability encoding with only a single stream
var encodings = [{
// Base framerate is one quarter of the input framerate
encodingId: "layer0",
scaleFramerateDownBy: 4.0}, {
// Temporal enhancement (half the input framerate when combined with the base layer)
encodingId: "layer1",
dependencyEncodingIds: ["layer0"],
scaleFramerateDownBy: 2.0}, {
// Another temporal enhancement layer (full input framerate when all layers combined)
encodingId: "layer2",
dependencyEncodingIds: ["layer0", "layer1"],
scaleFramerateDownBy: 1.0}];
The bitstream dependencies look like this:
[cid:da8d1820-850f-4d22-961d-deaf9bebf284]
Combining spatial simulcast and temporal scalability would look like this:
// Example of 2-layer spatial simulcast combined with 2-layer temporal scalability
var encodings = [{
// Low resolution base layer (half the input framerate, half the input resolution)
rid: "lowres",
encodingId: "L0",
resolutionScale: 2.0,
framerateScale: 2.0
}, {
// High resolution Base layer (half the input framerate, full input resolution)
rid: "highres",
encodingId: "H0",
resolutionScale: 1.0,
framerateScale: 2.0
}, {
// Temporal enhancement to the low resolution base layer (full input framerate, half resolution)
rid: "lowres",
encodingId: "L1",
dependencyEncodingIds: ["L0"],
resolutionScale: 2.0,
framerateScale: 1.0
}, {
// Temporal enhancement to the high resolution base layer (full input framerate and resolution)
rid: "highres",
encodingId: "H1",
dependencyEncodingIds: ["H0"],
resolutionScale: 1.0,
framerateScale: 1.0
}];
The layer diagram corresponding to this example is here:
[cid:b0306293-09e3-4c51-a882-f6fd4fb98f81]
At the F2F, we also talked about allowing developers uninterested in the details to easily generate appropriate configurations.
This could be done via helper functions, such as generateEncodings():
// function generateEncodings (spatialSimulcastLayers, temporalLayers)
var encodings = generateEncodings(1, 3); // generates the encoding shown in the first example
var encodings = generateEncodings(2, 2); // generates the encoding shown in the second example
var encodings = generateEncodings(); // generates a default encoding such as 3 spatial simulcast, 3 temporal layers
Limitations
What does this proposal *not* enable? RTCRtpFecParameters and RTCRtpRtxParameters were removed from WebRTC 1.0 a while ago, so this proposal does not enable support for differential protection (e.g. retransmission or forward error correction only for the base layer).
However, the proposal can support for spatial scalability, if there is interest in enabling this at some point. How this works is illustrated in the following example (which does not utilize rids):
// Example of 2-layer spatial scalability combined with 2-layer temporal scalability
var encodings = [{
// Base layer (half input framerate, half resolution)
encodingId: "T0S0",
resolutionScale: 2.0,
framerateScale: 2.0
}, {
// Temporal enhancement to the base layer (full input framerate, half resolution)
encodingId: "T1S0",
dependencyEncodingIds: ["0"],
resolutionScale: 2.0,
framerateScale: 1.0
}, {
// Spatial enhancement to the base layer (half input framerate, full resolution)
encodingId: "T0S1",
dependencyEncodingIds: ["T0S0"],
resolutionScale: 1.0,
framerateScale: 2.0
}, {
// Temporal enhancement to the spatial enhancement layer (full input framerate, full resolution)
encodingId: "T1S1",
dependencyEncodingIds: ["T0S1"],
resolutionScale: 1.0,
framerateScale: 1.0
}];
The diagram for looks like this:
[cid:4555029a-fbb4-4dc0-8cc0-86620fb9b26b]
--
Iņaki Baz Castillo
<ibc@aliax.net<mailto:ibc@aliax.net>>
Received on Wednesday, 4 July 2018 00:00:22 UTC