Re: [MSE] Timestamp offset mechanism

Ping. I haven't seen any responses to my last email on this thread so I'm
not sure how to proceed.

I'm retracting my support for the push/popTimestampOffset() proposal until
someone can address concerns I mentioned.

For now I'm proposing we just do the simplest thing that would work:
1. Add  an  "attribute double timestampOffset" to SourceBuffer.
2. The value of this attribute is added to media timestamps before the
frames are inserted into the source buffer. 0 is the default value for this
attribute since that results in unmodified timestamps.
3. If the app tries to set this attribute while in the middle of appending
a media segment, an INVALID_STATE_ERR exception is thrown and the attribute
is not updated. If the app doesn't know whether it is in the middle of a
media segment, it can always call abort() to reset the parser state.

Please let me know if you are ok with this simple attribute proposal or
whether you have arguments for the push/pop mechanism that I should
reconsider.

Thanks,
Aaron


On Wed, Jul 18, 2012 at 2:19 PM, Aaron Colwell <acolwell@google.com> wrote:

> So I'm starting to waver on my support for the pushTimestampOffset() &
> popTimestampOffset() mechanism and leaning towards the simple
> timestampOffset attribute again.
>
> Here is why:
> 1. The offset stack can be implemented in JavaScript. I'm not really
> seeing an advantage for putting this in the UA.
> 2. Mark's example looks like a common use case, but it only pushes things
> onto the stack. This is essentially equivalent to updating a
> timestampOffset attribute.
> 3. If the application isn't aware of the actual internal timestamps of a
> media segment, it could always append it to a "scratch" SourceBuffer on a
> different MediaSource object and discover the start timestamp by calling
> SourceBuffer.buffered.
>
> Am I missing something key here? Can someone provide other examples that
> show why it is advantageous for the UA to maintain the stack?
>
> Aaron
>
>
> On Thu, Jul 12, 2012 at 4:45 PM, Aaron Colwell <acolwell@google.com>wrote:
>
>> Ok. This seems reasonable to me.
>>
>> Question 2: I was just trying to determine if we wanted to provide a way
>> for the app to "rediscover" that an offset is currently active in case it
>> wasn't keeping track. Perhaps:
>>
>> // Number levels of timestampOffset (ie how many popTimestampOffset() I
>> need to clear them all)
>> readonly attribute int timestampOffsetDepth;
>>
>> or
>>
>>  // Indicates at least one timestamp offset is on the offset stack.
>> readonly attribute bool usingTimestampOffset;
>>
>> I'm not sure how critical this is, but I could definitely see people
>> getting confused if they forgot that they called pushTimestampOffset() and
>> then had no way to detect that an offset is still being applied.
>>
>>
>> Question 4: Yes I think that an offset should stay in place across
>> multiple segments. I was thinking about events that would fire when push &
>> pop actions actually become active. For example if you pushed an offset
>> while in the middle of appending a media segment, it wouldn't take effect
>> until the end of that segment. Once the segment ended a
>> 'timestampoffsetstart' event would fire since that is when the new offset
>> would start being applied. If a pop happens mid-segment then
>> a 'timestampoffsetend' would fire when the current segment has been
>> completely appended.
>>
>> I'm not sure whether these events are really that useful or not, but I
>> just wanted to ask the question to see what people thought.
>>
>> Aaron
>>
>> On Thu, Jul 12, 2012 at 2:33 PM, Mark Watson <watsonm@netflix.com> wrote:
>>
>>>
>>>  On Jul 12, 2012, at 1:59 PM, Mark Watson wrote:
>>>
>>>
>>>  On Jul 12, 2012, at 12:58 PM, Aaron Colwell wrote:
>>>
>>> Yes. The idea was to prevent the app from having to worry about the
>>> internal timestamp and only worry about the presentation times. I think
>>> this API needs some polishing though.
>>>
>>>  Here are some outstanding questions I have:
>>>
>>>  1. nextSegmentStartTime() & clearTimestampOffset() names are a little
>>> clunky and don't seem related. Alternate name ideas anyone?
>>>
>>>
>>>  pushTimestampOffset() and popTimestampOffset() ?
>>>
>>>
>>>  2. How does the app determine that an offset is currently being
>>> applied?
>>>
>>>
>>>  Does it need too ? Since the app is the only one which can establish
>>> one.
>>>
>>>  peekTimestampOffset() ?
>>>
>>>
>>>  3. Should we model this as offset stack? I think Adrian suggested this
>>> during the last Media Source telecom.
>>>
>>>
>>>  Yes.
>>>
>>>
>>>  4. Should events be defined to signal when an offset starts/stops
>>> being applied? Effectively this marks the beginning of the next media
>>> segment after either method is called.'
>>>
>>>
>>>  Hmm. I assumed the offset persists across multiple media segments
>>> until you cancel it.
>>>
>>>  You may, for example, have 200 segments of main media content that
>>> start at timestamps 0, 2, 3, … 398s. You feed in 100 of them without
>>> setting an offset. Then you have 15 ad segments with timestamps 0, … 28s
>>> and before feeding these in you call pushTimestampOffset( 200s ). Once
>>> finished with these you have to call popTimestampOffset() and
>>> pushTimestampOffset( 30 ) before feeding in the rest of the media segments
>>> from the main content.
>>>
>>>
>>>  Oops, I mean pushTimestampOffset( 230s ) for the last one here as we
>>> are specifying the actual source buffer time where we want the next segment
>>> to appear, not the offset between internal media time and source buffer
>>> time.
>>>
>>>
>>>
>>>  5. Does anyone have an alternate proposal for applying offsets?
>>>
>>>  Aaron
>>>
>>> On Wed, Jul 11, 2012 at 9:26 AM, Mark Watson <watsonm@netflix.com>wrote:
>>>
>>>>  Actually, I liked the nextSegmentStartTime (option 3). That is,
>>>> assuming I remember correctly that what it does: establish a mapping from
>>>> internal media time to source buffer time such that the next segment is
>>>> placed at the provided source buffer time. (strictly, the mapping is
>>>> established when the next segment arrives).
>>>>
>>>>  This obviates the need for the app to understand much about the
>>>> internal media time.
>>>>
>>>>  ...Mark
>>>>
>>>>  Sent from my iPhone
>>>>
>>>> On Jul 10, 2012, at 11:50 AM, "Aaron Colwell" <acolwell@google.com>
>>>> wrote:
>>>>
>>>>   Hi All,
>>>>
>>>>  I'm going to try to summarize what I've read on this thread to see if
>>>> I understand everyone's position
>>>>
>>>>  1. Eric, Steve, & Kevin all support the sourceTimestampOffset()
>>>> signature
>>>> 2. Mark doesn't have a strong preference either way, but is leaning
>>>> towards sourceTimestampMapping() because it is more explicit about the
>>>> application needing to know about the internal timestamps in the media
>>>> segments and how they should map to the presentation timeline.
>>>> 3. The alternative nextSegmentStartTime()/clearTimestampOffset()
>>>> suggestion mentioned at the bottom of
>>>> http://lists.w3.org/Archives/Public/public-html-media/2012Jun/0095.html got
>>>> no votes
>>>> 4. Duncan feels this mechanism isn't sufficient to address ad-insertion
>>>> use cases he cares about. I just started an Ad Insertion thread<http://lists.w3.org/Archives/Public/public-html-media/2012Jul/0033.html> to
>>>> continue the discussion. This may require adding a new feature.
>>>>
>>>>  Let me know if this is correct or if I've missed something.
>>>>
>>>>  Thanks,
>>>> Aaron
>>>>
>>>>
>>>> On Fri, Jun 15, 2012 at 3:40 PM, Aaron Colwell <acolwell@google.com>wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>>  Bug 17004 <https://www.w3.org/Bugs/Public/show_bug.cgi?id=17004> proposes
>>>>> a mechanism for adjusting timestamps inside media segments. This enables
>>>>> seamlessly splicing content together that doesn't originally exist on a
>>>>> common presentation timeline. Inserting advertisements into content is the
>>>>> most straight forward use case. There has been some useful discussion in
>>>>> the bug comments, but we haven't decided on which method signature to use
>>>>> for specifying the timestamp adjustments.
>>>>>
>>>>>  Here are the proposed signatures at this point:
>>>>>
>>>>>  void sourceTimestampMapping(in double presentationTimestamp, in
>>>>> double segmentTimestamp)
>>>>>  -- or --
>>>>> void sourceTimestampOffset(in double timestampOffset)
>>>>>
>>>>>  sourceTimestampMapping() just provides a way to say, "I want this
>>>>> timestamp in the segment to map to this presentation time."
>>>>> sourceTimestampOffset() is basically just saying, "Add this offset to
>>>>> any timestamp in the segment to get the desired presentation time."
>>>>>
>>>>>  Fundamentally they do the exact same thing, but I'm looking for
>>>>> input on which option people feel more comfortable with.
>>>>>
>>>>>  Which do you prefer?
>>>>> Are you ok with the semantics outlined in the bug description<https://www.w3.org/Bugs/Public/show_bug.cgi?id=17004#c0>
>>>>> ?
>>>>>
>>>>>  Aaron
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>
>