Re: [TTML2] Document relationship to media timings - re Issue-346, Issue-270, Issue-335

On Fri, Sep 26, 2014 at 9:35 AM, Nigel Megitt <nigel.megitt@bbc.co.uk>
wrote:

>  Thanks Glenn, I look forward to your further thoughts.
>
>  On the particular question 'can root temporal extent be defined by the
> media rather than the document'
>

I interpret this (in quotes) as NO, it is never defined by the media.
However, if there is a related media object, then the later (document
times) can be related to the former (media times).


> my expectation is that any presentation processor will treat any related
> media's timeline as being primary and, for example, commence and cease
> playback of both the media and the document at the begin and end of the
> media, thus potentially truncating the root temporal extent as I describe
> in the 4th row.
>

I agree that if there is related media, then I expect the active interval
of the document, i.e., the duration over which the document can be
displayed, to be bounded by the active interval of the media. However, I
don't view that as changing the document root extent. In other words I'm
treating the duration of the active interval of the document as being no
greater than but possible less than the duration of the root temporal
interval.

I don't view this bounding operating as actually changing the root temporal
extent. In other words, BEGIN(document) may be earlier than BEGIN(media)
and END(document) may be later than END(media), but presentation only
occurs between BEGIN(media) and END(media) even though the following
intervals may be non-empty: [ BEGIN(document), BEGIN(media) ), [
END(media), END(document) ).


>
>  If we were to take the alternate view then we would presumably require
> the presentation processor to commence playback of both media and document
> at EARLIER(BEGIN(media), BEGIN(document)) and cease playback by
> LATER(END(media), END(document)). That may be a very good idea, but not one
> that I expect to be implemented.
>

I don't expect presentation (of document) to occur during [
BEGIN(document), BEGIN(media) ) or [ END(media), END(document) ), but that
doesn't mean that TIME(document) during these two intervals is not defined
or is not (conceptually running).


>
>  I would also replace interval expressions such as "BEGIN(xxx) ->
> END(xxx)" with "[ BEGIN(xxx), END(xxx) )" for precision. That's a second
> order issue in my mind, right now, compared to getting the right things
> inside the brackets.
>
>  Kind regards,
>
>  Nigel
>
>
>   From: Glenn Adams <glenn@skynav.com>
> Date: Friday, 26 September 2014 17:21
> To: Nigel Megitt <nigel.megitt@bbc.co.uk>
> Cc: John Birch <John.Birch@screensystems.tv>, Timed Text Working Group <
> public-tt@w3.org>
> Subject: Re: [TTML2] Document relationship to media timings - re
> Issue-346, Issue-270, Issue-335
>
>   I need to spend more time analyzing this, but at first order, this
> diverges from my intended definition of root external extent, which is
> defined by the first (core) part of the definition:
>
>  The temporal extent (interval) defined by the temporal beginning and
> ending of a document instance
> <https://dvcs.w3.org/hg/ttml/raw-file/default/ttml2/spec/ttml2.html#terms-document-instance>
>  ...
>
>  In other words, root termporal extent is *always* an interval starting
> at BEGIN(document) and ending at END(document). How this relates to some
> other context is the second, ancillary part of the definition:
>
>  ...in relationship with some external application or presentation
> context.
>
>  So I have an issue with the table's fourth column relabeling root
> temporal extent in different terms, e.g., the fourth data row restates it
> as the interval BEGIN(media), END (media).
>
>  Also I should have been more explicit in the definition about closed
> versus open, as I intended it to be cast in terms of a left-closed,
> right-open interval.
>
>
> On Fri, Sep 26, 2014 at 8:54 AM, Nigel Megitt <nigel.megitt@bbc.co.uk>
> wrote:
>
>>  Perhaps we could clarify the definition of Root Temporal Extent with
>> respect to the timebase and the media to make sure we have common ground.
>> In TTML1 and TTML2 it is:
>>
>>  "[root temporal extent]
>>
>> The temporal extent (interval) defined by the temporal beginning and
>> ending of a document instance
>> <https://dvcs.w3.org/hg/ttml/raw-file/default/ttml2/spec/ttml2.html#terms-document-instance> in
>> relationship with some external application or presentation context."
>>
>>  It may be unclear to some whether the relationship is defined in terms
>> of the origin coordinates or the begin and end times, and indeed there may
>> be different modes depending on the related media. If we expand the table
>> of permutations we have:
>>
>>    *Related Media *
>>
>> *D**ocument timebase *
>>
>> *Document to media timing relationship *
>>
>> *Root temporal extent*
>>
>> None
>>
>> media
>>
>> undefined/fault state
>>
>> BEGIN(document) -> END(document) where either or both can be unconstrained
>>
>> None
>>
>> smpte
>>
>> undefined/fault state
>>
>> BEGIN(document) -> END(document) where either or both can be unconstrained
>>
>> None
>>
>> clock
>>
>> document times = clock times during media playback, i.e. dependent on
>> clock time at start of media playback.
>>
>> BEGIN(document) -> END(document) where either or both can be
>> unconstrained; any mappings between media times and document times must
>> defined externally, e.g. by knowing what clock time relates to BEGIN(media).
>>
>> Exists with SMPTE timecode
>>
>> media
>>
>> ORIGIN(document) = BEGIN(media)
>>
>> BEGIN(media) -> END(media)
>>
>> Exists with SMPTE timecode
>>
>> smpte
>>
>> document times = media times
>>
>> BEGIN(media) -> END(media)
>>
>> Exists with SMPTE timecode
>>
>> clock
>>
>> document times = clock times during media playback, i.e. dependent on
>> clock time at start of media playback.
>>
>> BEGIN(document) -> END(document) where either or both can be
>> unconstrained; any mappings between media times and document times must
>> defined externally, e.g. by knowing what clock time relates to BEGIN(media).
>>
>> Exists with no timecode*
>>
>> media
>>
>> ORIGIN(document) = ORIGIN(media) = BEGIN(media)
>>
>> BEGIN(media) -> END(media)
>>
>> Exists with no timecode*
>>
>> smpte
>>
>> ORIGIN(document) = ORIGIN(media) = BEGIN(media)
>>
>> BEGIN(media) -> END(media)
>>
>> Exists with no timecode*
>>
>> clock
>>
>> document times = clock times during media playback, i.e. dependent on
>> clock time at start of media playback.
>>
>> BEGIN(document) -> END(document) where either or both can be
>> unconstrained; any mappings between media times and document times must
>> defined externally, e.g. by knowing what clock time relates to BEGIN(media).
>>
>>
>>
>> * For media with no timecode we require that the play rate is somehow
>> defined, e.g. by counting frames and knowing the frame rate, and any
>> offsets are defined externally e.g. in an ISOBMFF wrapper.
>>
>>
>>
>> Points to watch out for:
>>
>> ·      When using timebase = clock with associated media, the overlap
>> between the document and the media is dependent on the clock times during
>> playback (or recording) of the media.
>>
>> ·      Using media or smpte timebase when there is no related media is
>> not prohibited, but by definition no timing relationship can exist. This
>> case may be useful for example in order to present text that changes over
>> time in a standalone page, where document playback is initiated by an
>> external event.
>>
>> ·      If media has SMPTE timecode and the document uses timebase media
>> then conversion is required (defined externally) to translate media times
>> to document times. For example the media may be video with a clock, slate,
>> colour bars or other material in advance of the main programme, with the
>> main programme given timecode e.g. 10:00:00 as per local practice, which
>> would correspond to media time 0s.
>>
>> ·      If media has no timecode and the document uses timebase media
>> then any offsets must be defined externally, e.g. edit lists that offset
>> the media slightly to accommodate the video decode latency in playback
>> devices. This is expected to be the common case for TTML documents and
>> media created for distribution, for example delivered within an ISOBMFF
>> wrapper.
>>
>> ·      If media has no timecode and the document uses timebase smpte
>> then no offset is defined. If a document begins at the value 10:00:00, then
>> regardless of any local practice defining the timecode relating to the
>> start of a programme, this implies that the first active text occurs 36000s
>> into the programme.
>>
>> ·      For documents with timebase smpte the markerMode is orthogonal to
>> this definition, however for documents with discontinuous timecode measured
>> against media with discontinuous SMPTE timecode, if the marker
>> corresponding to END(document) is repeated within the media then the first
>> such marker encountered will define the end point of the root temporal
>> extent of the document.
>>
>>
>>  I wonder if we are in full agreement about this expanded table or if
>> for some row there's an issue to be resolved?
>>
>>  Kind regards,
>>
>>  Nigel
>>
>>
>>   From: Glenn Adams <glenn@skynav.com>
>> Date: Thursday, 25 September 2014 18:12
>> To: John Birch <John.Birch@screensystems.tv>
>> Cc: Timed Text Working Group <public-tt@w3.org>
>> Subject: Re: ISSUE-346: Need ttp:mediaDuration parameter [TTML2]
>> Resent-From: <public-tt@w3.org>
>> Resent-Date: Thursday, 25 September 2014 18:13
>>
>>
>>
>> On Thu, Sep 25, 2014 at 8:36 AM, John Birch <John.Birch@screensystems.tv>
>> wrote:
>>
>>>  I cannot see how mediaduration can always be known… it certainly
>>> cannot be ‘a priori’ knowledge in the streaming live video scenario.
>>>
>>
>>  Please read the drafted text at [1].
>>
>>  [1]
>> https://dvcs.w3.org/hg/ttml/raw-file/tip/ttml2/spec/ttml2.html#parameter-attribute-mediaDuration
>>
>>
>>>   Consequently I do not see how TTML to ISD conversion can be
>>> normatively dependent on a parameter that is potentially unavailable.
>>>
>>
>>  It just means that in such a case, it a duration must be approximated
>> at some point in time or that everything must be explicitly timed (i.e., no
>> reliance on indefinite duration containers) or that conversion is not
>> possible.
>>
>>
>>>
>>>
>>> Am I misunderstanding and it is proposed that media duration is
>>> mandatory? (only when using media timebase?)
>>>
>>
>>  No. Read the draft text.
>>
>>
>>>
>>>
>>> Re: SMIL assumes they are relative to beginning of related media.
>>>
>>>
>>>
>>> Whilst I am certainly no expert on SMIL, I was under the impression that
>>> SMIL supports the concept of a sync-base that allows the origin point of
>>> timed events to be offset from a specific time point in the ‘host document’.
>>>
>>
>>  True, but we didn't support in TTML the full generality of SMIL timing
>> options. Let's just say that, at least for media time base, it is assumed
>> by TTML1 that document times are related to BEGIN(media), not
>> ORIGIN(media). This is no different from assuming that the origin of the
>> root container extent (in TTML1) is aligned with the origin of the related
>> media object (video).
>>
>>
>>>   TTML does not explicitly form a timing relationship with a related
>>> media within the TTML document, that relationship is established by
>>> normative prose in the specification – and thus could be elaborated upon.
>>>
>>
>>  Let's just say that in TTML1 this relationship is only discussed in the
>> Note in Appendix N.2, but it has been widely assumed and used in other
>> contexts, e.g., VTT, translation to HTML, etc.:
>>
>>  The above formalisms assumes that the *Root Temporal Extent* corresponds
>> with the beginning of a related media object. If this assumption doesn't
>> hold, then an additional offset that accounts for the difference may be
>> introduced when computing media time M.
>>
>>
>>>
>>>
>>> Re: However, apparently SMPTE time base authors have been using the
>>> origin of the time line of related media.
>>>
>>>
>>>
>>> I am not sure what you mean by ‘origin of the time line of related
>>> media’. The use of SMPTE timebase with ‘continuous’ is (rightly IMHO)
>>> deprecated.
>>>
>>
>>  Granted that for discontinuous mode my comment doesn't apply. Consider
>> my comment to apply only to smpte continuous mode *and* in the case that a
>> discontinuous mode document has been converted to smpte continuous or media
>> time base.
>>
>>
>>>  To effectively use SMPTE timebase you must be in discontinuous mode to
>>> engage the **reading of timecode values from the frames of the related
>>> media** - rather than use the play time of the related media – even if
>>> expressed as SMPTE12M. Timecode is and always has been a labelling scheme
>>> (that can – if certain constraints are met – also be used to calculate
>>> durations).
>>>
>>
>>  Yes, this is true for "SMPTE timecode" usage in the normal
>> discontinuous based interpretation, but not for SMIL clock-time expressions.
>>
>>
>>>  Since these constraints cannot be guaranteed in the TTML context (i.e.
>>> it has no knowledge of external media timecode contiguity) then
>>> ‘discontinuous’ must be used.
>>>
>>
>>  I don't want to turn this thread into a discussion of the utility of
>> smpte continuous mode, but I should say that I don't discount its utility
>> as you appear to do.
>>
>>
>>>  Further, if continuous was used with smpte timebase in a TTML
>>> document, then the time values in the TTML would have to be relative to the
>>> first frame of the media – even while being expressed in SMPTE12M… and
>>> consequently would typically be significantly offset from the timecode
>>> labels that might exist within the media itself (e.g. by 10 hours or 3
>>> hours etc.).
>>>
>>
>>  I am not making that assumption: that time values in a smpte continuous
>> mode TTML document are relative to the first frame of the media. In fact,
>> I'm assuming that there may be an arbitrary offset between the two, just as
>> I'm assuming there may be an arbitrary (positive or negative) offset in the
>> media time base scenario.
>>
>>
>>>  Yet further, those values would also have to take into account a
>>> potentially variable length slate that might be added, extended or
>>> truncated during the processing of media files, necessitating a change of
>>> the TTML document every time the related media object was processed (which
>>> is of course undesirable). This is of course all very broadcast media
>>> centric – but of course that is the domain of smpte timecode!
>>>
>>
>>  I'm not following this, since I don't know the meaning of "variable
>> length slate".
>>
>>
>>>
>>>
>>> Since SMPTE must be discontinuous,
>>>
>>
>>  Nope, we have smpte continuous mode in TTML1 and just today decided to
>> keep it in TTML2. [1]
>>
>>  [1] http://www.w3.org/AudioVideo/TT/tracker/issues/349
>>
>>
>>>  the ‘origin of a timeline’ is irrelevant? (as the Document Processing
>>> Context emits labelled synchronisation events).
>>>
>>
>>  It is definitely not irrelevant in media time base, in smpte continuous
>> mode, or in conversion of smpte discontinuous mode to media or smpte
>> continuous time base.
>>
>>
>>>
>>>
>>> Best regards,
>>>
>>> John
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *John Birch | Strategic Partnerships Manager | Screen *Main Line : +44
>>> 1473 831700 | Ext : 2208 | Direct Dial : +44 1473 834532
>>> Mobile : +44 7919 558380 | Fax : +44 1473 830078
>>> John.Birch@screensystems.tv | www.screensystems.tv |
>>> https://twitter.com/screensystems
>>>
>>>
>>>
>>> *Visit us at SMPTE Annual Technical Conference, Loews Hollywood Hotel,
>>> Stand 107, October 21-23 Languages & the Media, Hotel Radission Blu,
>>> Berlin, November 5-7 *
>>>
>>> *P** Before printing, think about the environment*
>>>
>>>  *From:* Glenn Adams [mailto:glenn@skynav.com]
>>> *Sent:* 25 September 2014 14:15
>>> *To:* John Birch
>>> *Cc:* Timed Text Working Group
>>> *Subject:* Re: ISSUE-346: Need ttp:mediaDuration parameter [TTML2]
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Sep 25, 2014 at 3:38 AM, John Birch <John.Birch@screensystems.tv>
>>> wrote:
>>>
>>> I am uncertain about the 'requirement' for a duration parameter?
>>> What is the current situation... does TTML1 work as if the media
>>> duration is 'indefinite' ?
>>> In which case what additional benefit does having a defined duration
>>> bring?
>>>
>>>
>>>
>>> The TTML to ISD conversion process will normatively use this parameter.
>>> It is also used by the proposed CP5 for conversion to HTML.
>>>
>>>
>>>
>>>
>>> Of far more importance (IMHO) is a media start value - that allows time
>>> values within a document to be 'offset' relative to the start of associated
>>> media.
>>>
>>>
>>>
>>> We have an interesting issue here: what are authors using for the origin
>>> of document times. SMIL assumes they are relative to beginning of related
>>> media. However, apparently SMPTE time base authors have been using the
>>> origin of the time line of related meda. But this is unrelated to
>>> ttp:mediaDuration.
>>>
>>>
>>>
>>>
>>> Best regards,
>>> John
>>>
>>> John Birch | Strategic Partnerships Manager | Screen
>>> Main Line : +44 1473 831700 | Ext : 2208 | Direct Dial : +44 1473 834532
>>> Mobile : +44 7919 558380 | Fax : +44 1473 830078
>>> John.Birch@screensystems.tv | www.screensystems.tv |
>>> https://twitter.com/screensystems
>>>
>>> Visit us at
>>> SMPTE Annual Technical Conference, Loews Hollywood Hotel, Stand 107,
>>> October 21-23
>>> Languages & the Media, Hotel Radission Blu, Berlin, November 5-7
>>>
>>> P Before printing, think about the environment-----Original Message-----
>>> From: Timed Text Working Group Issue Tracker [mailto:
>>> sysbot+tracker@w3.org]
>>> Sent: 21 September 2014 13:35
>>> To: public-tt@w3.org
>>> Subject: ISSUE-346: Need ttp:mediaDuration parameter [TTML2]
>>>
>>>
>>> ISSUE-346: Need ttp:mediaDuration parameter [TTML2]
>>>
>>> http://www.w3.org/AudioVideo/TT/tracker/issues/346
>>>
>>> Raised by: Glenn Adams
>>> On product: TTML2
>>>
>>> In order to perform ISD processing, it is necessary to know the duration
>>> of root external extent. When associate with a related media object, this
>>> is the simple duration of the related media object. If this is known at
>>> authoring time, then it is an important parameter that should be specified.
>>> I propose defining a ttp:mediaDuration parameter attribute that takes
>>> either an offset-time form of a time expression or the keyword
>>> "indefinite", where the latter is used (or implied) when no related media
>>> object exists or its simple duration is unknown or indefinite. If this
>>> parameter is not specified, then it would be treated as if indefinite were
>>> specified.
>>>
>>>
>>>
>>>   This message may contain confidential and/or privileged information.
>>> If you are not the intended recipient you must not use, copy, disclose or
>>> take any action based on this message or any information herein. If you
>>> have received this message in error, please advise the sender immediately
>>> by reply e-mail and delete this message. Thank you for your cooperation.
>>> Screen Subtitling Systems Ltd. Registered in England No. 2596832.
>>> Registered Office: The Old Rectory, Claydon Church Lane, Claydon, Ipswich,
>>> Suffolk, IP6 0EQ
>>>
>>>
>>>
>>>  This message may contain confidential and/or privileged information.
>>> If you are not the intended recipient you must not use, copy, disclose or
>>> take any action based on this message or any information herein. If you
>>> have received this message in error, please advise the sender immediately
>>> by reply e-mail and delete this message. Thank you for your cooperation.
>>> Screen Subtitling Systems Ltd. Registered in England No. 2596832.
>>> Registered Office: The Old Rectory, Claydon Church Lane, Claydon, Ipswich,
>>> Suffolk, IP6 0EQ
>>>     ­­
>>>
>>
>>
>