W3C home > Mailing lists > Public > public-tt@w3.org > September 2014

Re: [TTML2] Document relationship to media timings - re Issue-346, Issue-270, Issue-335

From: Nigel Megitt <nigel.megitt@bbc.co.uk>
Date: Fri, 26 Sep 2014 16:02:21 +0000
To: Nigel Megitt <nigel.megitt@bbc.co.uk>, Glenn Adams <glenn@skynav.com>
CC: John Birch <John.Birch@screensystems.tv>, Timed Text Working Group <public-tt@w3.org>
Message-ID: <D04B4944.122B3%nigel.megitt@bbc.co.uk>
To add a further data point, consider a sample-based wrapper such as ISOBMFF defining the interval for each sample relative to other media within the same wrapper. If each sample contains a single TTML document, then the sample interval must be the root temporal extent for that document.

For the purposes of a single TTML document occupying one sample in this context [ BEGIN(sample), END(sample) ) defines the root temporal extent, and by definition BEGIN(sample) >= BEGIN(media) and END(sample) <= END(media). Also ORIGIN(document) = ORIGIN(media) independent of the samples created.

Kind regards,

Nigel


From: Nigel Megitt <nigel.megitt@bbc.co.uk<mailto:nigel.megitt@bbc.co.uk>>
Date: Friday, 26 September 2014 17:35
To: Glenn Adams <glenn@skynav.com<mailto:glenn@skynav.com>>
Cc: John Birch <John.Birch@screensystems.tv<mailto:John.Birch@screensystems.tv>>, Timed Text Working Group <public-tt@w3.org<mailto:public-tt@w3.org>>
Subject: Re: [TTML2] Document relationship to media timings - re Issue-346, Issue-270, Issue-335
Resent-From: <public-tt@w3.org<mailto:public-tt@w3.org>>
Resent-Date: Friday, 26 September 2014 17:36

Thanks Glenn, I look forward to your further thoughts.

On the particular question 'can root temporal extent be defined by the media rather than the document' my expectation is that any presentation processor will treat any related media's timeline as being primary and, for example, commence and cease playback of both the media and the document at the begin and end of the media, thus potentially truncating the root temporal extent as I describe in the 4th row.

If we were to take the alternate view then we would presumably require the presentation processor to commence playback of both media and document at EARLIER(BEGIN(media), BEGIN(document)) and cease playback by LATER(END(media), END(document)). That may be a very good idea, but not one that I expect to be implemented.

I would also replace interval expressions such as "BEGIN(xxx) -> END(xxx)" with "[ BEGIN(xxx), END(xxx) )" for precision. That's a second order issue in my mind, right now, compared to getting the right things inside the brackets.

Kind regards,

Nigel


From: Glenn Adams <glenn@skynav.com<mailto:glenn@skynav.com>>
Date: Friday, 26 September 2014 17:21
To: Nigel Megitt <nigel.megitt@bbc.co.uk<mailto:nigel.megitt@bbc.co.uk>>
Cc: John Birch <John.Birch@screensystems.tv<mailto:John.Birch@screensystems.tv>>, Timed Text Working Group <public-tt@w3.org<mailto:public-tt@w3.org>>
Subject: Re: [TTML2] Document relationship to media timings - re Issue-346, Issue-270, Issue-335

I need to spend more time analyzing this, but at first order, this diverges from my intended definition of root external extent, which is defined by the first (core) part of the definition:

The temporal extent (interval) defined by the temporal beginning and ending of a document instance<https://dvcs.w3.org/hg/ttml/raw-file/default/ttml2/spec/ttml2.html#terms-document-instance> ...

In other words, root termporal extent is *always* an interval starting at BEGIN(document) and ending at END(document). How this relates to some other context is the second, ancillary part of the definition:

...in relationship with some external application or presentation context.

So I have an issue with the table's fourth column relabeling root temporal extent in different terms, e.g., the fourth data row restates it as the interval BEGIN(media), END (media).

Also I should have been more explicit in the definition about closed versus open, as I intended it to be cast in terms of a left-closed, right-open interval.


On Fri, Sep 26, 2014 at 8:54 AM, Nigel Megitt <nigel.megitt@bbc.co.uk<mailto:nigel.megitt@bbc.co.uk>> wrote:
Perhaps we could clarify the definition of Root Temporal Extent with respect to the timebase and the media to make sure we have common ground. In TTML1 and TTML2 it is:

"[root temporal extent]

The temporal extent (interval) defined by the temporal beginning and ending of a document instance<https://dvcs.w3.org/hg/ttml/raw-file/default/ttml2/spec/ttml2.html#terms-document-instance> in relationship with some external application or presentation context."

It may be unclear to some whether the relationship is defined in terms of the origin coordinates or the begin and end times, and indeed there may be different modes depending on the related media. If we expand the table of permutations we have:

Related Media

Document timebase

Document to media timing relationship

Root temporal extent

None

media

undefined/fault state

BEGIN(document) -> END(document) where either or both can be unconstrained

None

smpte

undefined/fault state

BEGIN(document) -> END(document) where either or both can be unconstrained

None

clock

document times = clock times during media playback, i.e. dependent on clock time at start of media playback.

BEGIN(document) -> END(document) where either or both can be unconstrained; any mappings between media times and document times must defined externally, e.g. by knowing what clock time relates to BEGIN(media).

Exists with SMPTE timecode

media

ORIGIN(document) = BEGIN(media)

BEGIN(media) -> END(media)

Exists with SMPTE timecode

smpte

document times = media times

BEGIN(media) -> END(media)

Exists with SMPTE timecode

clock

document times = clock times during media playback, i.e. dependent on clock time at start of media playback.

BEGIN(document) -> END(document) where either or both can be unconstrained; any mappings between media times and document times must defined externally, e.g. by knowing what clock time relates to BEGIN(media).

Exists with no timecode*

media

ORIGIN(document) = ORIGIN(media) = BEGIN(media)

BEGIN(media) -> END(media)

Exists with no timecode*

smpte

ORIGIN(document) = ORIGIN(media) = BEGIN(media)

BEGIN(media) -> END(media)

Exists with no timecode*

clock

document times = clock times during media playback, i.e. dependent on clock time at start of media playback.

BEGIN(document) -> END(document) where either or both can be unconstrained; any mappings between media times and document times must defined externally, e.g. by knowing what clock time relates to BEGIN(media).


* For media with no timecode we require that the play rate is somehow defined, e.g. by counting frames and knowing the frame rate, and any offsets are defined externally e.g. in an ISOBMFF wrapper.

Points to watch out for:

·      When using timebase = clock with associated media, the overlap between the document and the media is dependent on the clock times during playback (or recording) of the media.

·      Using media or smpte timebase when there is no related media is not prohibited, but by definition no timing relationship can exist. This case may be useful for example in order to present text that changes over time in a standalone page, where document playback is initiated by an external event.

·      If media has SMPTE timecode and the document uses timebase media then conversion is required (defined externally) to translate media times to document times. For example the media may be video with a clock, slate, colour bars or other material in advance of the main programme, with the main programme given timecode e.g. 10:00:00 as per local practice, which would correspond to media time 0s.

·      If media has no timecode and the document uses timebase media then any offsets must be defined externally, e.g. edit lists that offset the media slightly to accommodate the video decode latency in playback devices. This is expected to be the common case for TTML documents and media created for distribution, for example delivered within an ISOBMFF wrapper.

·      If media has no timecode and the document uses timebase smpte then no offset is defined. If a document begins at the value 10:00:00, then regardless of any local practice defining the timecode relating to the start of a programme, this implies that the first active text occurs 36000s into the programme.

·      For documents with timebase smpte the markerMode is orthogonal to this definition, however for documents with discontinuous timecode measured against media with discontinuous SMPTE timecode, if the marker corresponding to END(document) is repeated within the media then the first such marker encountered will define the end point of the root temporal extent of the document.

I wonder if we are in full agreement about this expanded table or if for some row there's an issue to be resolved?

Kind regards,

Nigel


From: Glenn Adams <glenn@skynav.com<mailto:glenn@skynav.com>>
Date: Thursday, 25 September 2014 18:12
To: John Birch <John.Birch@screensystems.tv<mailto:John.Birch@screensystems.tv>>
Cc: Timed Text Working Group <public-tt@w3.org<mailto:public-tt@w3.org>>
Subject: Re: ISSUE-346: Need ttp:mediaDuration parameter [TTML2]
Resent-From: <public-tt@w3.org<mailto:public-tt@w3.org>>
Resent-Date: Thursday, 25 September 2014 18:13



On Thu, Sep 25, 2014 at 8:36 AM, John Birch <John.Birch@screensystems.tv<mailto:John.Birch@screensystems.tv>> wrote:
I cannot see how mediaduration can always be known… it certainly cannot be ‘a priori’ knowledge in the streaming live video scenario.

Please read the drafted text at [1].

[1] https://dvcs.w3.org/hg/ttml/raw-file/tip/ttml2/spec/ttml2.html#parameter-attribute-mediaDuration


Consequently I do not see how TTML to ISD conversion can be normatively dependent on a parameter that is potentially unavailable.

It just means that in such a case, it a duration must be approximated at some point in time or that everything must be explicitly timed (i.e., no reliance on indefinite duration containers) or that conversion is not possible.


Am I misunderstanding and it is proposed that media duration is mandatory? (only when using media timebase?)

No. Read the draft text.


Re: SMIL assumes they are relative to beginning of related media.

Whilst I am certainly no expert on SMIL, I was under the impression that SMIL supports the concept of a sync-base that allows the origin point of timed events to be offset from a specific time point in the ‘host document’.

True, but we didn't support in TTML the full generality of SMIL timing options. Let's just say that, at least for media time base, it is assumed by TTML1 that document times are related to BEGIN(media), not ORIGIN(media). This is no different from assuming that the origin of the root container extent (in TTML1) is aligned with the origin of the related media object (video).

TTML does not explicitly form a timing relationship with a related media within the TTML document, that relationship is established by normative prose in the specification – and thus could be elaborated upon.

Let's just say that in TTML1 this relationship is only discussed in the Note in Appendix N.2, but it has been widely assumed and used in other contexts, e.g., VTT, translation to HTML, etc.:

The above formalisms assumes that the Root Temporal Extent corresponds with the beginning of a related media object. If this assumption doesn't hold, then an additional offset that accounts for the difference may be introduced when computing media time M.


Re: However, apparently SMPTE time base authors have been using the origin of the time line of related media.

I am not sure what you mean by ‘origin of the time line of related media’. The use of SMPTE timebase with ‘continuous’ is (rightly IMHO) deprecated.

Granted that for discontinuous mode my comment doesn't apply. Consider my comment to apply only to smpte continuous mode *and* in the case that a discontinuous mode document has been converted to smpte continuous or media time base.

To effectively use SMPTE timebase you must be in discontinuous mode to engage the *reading of timecode values from the frames of the related media* - rather than use the play time of the related media – even if expressed as SMPTE12M. Timecode is and always has been a labelling scheme (that can – if certain constraints are met – also be used to calculate durations).

Yes, this is true for "SMPTE timecode" usage in the normal discontinuous based interpretation, but not for SMIL clock-time expressions.

Since these constraints cannot be guaranteed in the TTML context (i.e. it has no knowledge of external media timecode contiguity) then ‘discontinuous’ must be used.

I don't want to turn this thread into a discussion of the utility of smpte continuous mode, but I should say that I don't discount its utility as you appear to do.

Further, if continuous was used with smpte timebase in a TTML document, then the time values in the TTML would have to be relative to the first frame of the media – even while being expressed in SMPTE12M… and consequently would typically be significantly offset from the timecode labels that might exist within the media itself (e.g. by 10 hours or 3 hours etc.).

I am not making that assumption: that time values in a smpte continuous mode TTML document are relative to the first frame of the media. In fact, I'm assuming that there may be an arbitrary offset between the two, just as I'm assuming there may be an arbitrary (positive or negative) offset in the media time base scenario.

Yet further, those values would also have to take into account a potentially variable length slate that might be added, extended or truncated during the processing of media files, necessitating a change of the TTML document every time the related media object was processed (which is of course undesirable). This is of course all very broadcast media centric – but of course that is the domain of smpte timecode!

I'm not following this, since I don't know the meaning of "variable length slate".


Since SMPTE must be discontinuous,

Nope, we have smpte continuous mode in TTML1 and just today decided to keep it in TTML2. [1]

[1] http://www.w3.org/AudioVideo/TT/tracker/issues/349


the ‘origin of a timeline’ is irrelevant? (as the Document Processing Context emits labelled synchronisation events).

It is definitely not irrelevant in media time base, in smpte continuous mode, or in conversion of smpte discontinuous mode to media or smpte continuous time base.


Best regards,
John



John Birch | Strategic Partnerships Manager | Screen
Main Line : +44 1473 831700<tel:%2B44%201473%20831700> | Ext : 2208 | Direct Dial : +44 1473 834532<tel:%2B44%201473%20834532>
Mobile : +44 7919 558380<tel:%2B44%207919%20558380> | Fax : +44 1473 830078<tel:%2B44%201473%20830078>
John.Birch@screensystems.tv<mailto:John.Birch@screensystems.tv> | www.screensystems.tv<http://www.screensystems.tv> | https://twitter.com/screensystems


Visit us at
SMPTE Annual Technical Conference, Loews Hollywood Hotel, Stand 107, October 21-23
Languages & the Media, Hotel Radission Blu, Berlin, November 5-7

P Before printing, think about the environment

From: Glenn Adams [mailto:glenn@skynav.com<mailto:glenn@skynav.com>]
Sent: 25 September 2014 14:15
To: John Birch
Cc: Timed Text Working Group
Subject: Re: ISSUE-346: Need ttp:mediaDuration parameter [TTML2]



On Thu, Sep 25, 2014 at 3:38 AM, John Birch <John.Birch@screensystems.tv<mailto:John.Birch@screensystems.tv>> wrote:
I am uncertain about the 'requirement' for a duration parameter?
What is the current situation... does TTML1 work as if the media duration is 'indefinite' ?
In which case what additional benefit does having a defined duration bring?

The TTML to ISD conversion process will normatively use this parameter. It is also used by the proposed CP5 for conversion to HTML.


Of far more importance (IMHO) is a media start value - that allows time values within a document to be 'offset' relative to the start of associated media.

We have an interesting issue here: what are authors using for the origin of document times. SMIL assumes they are relative to beginning of related media. However, apparently SMPTE time base authors have been using the origin of the time line of related meda. But this is unrelated to ttp:mediaDuration.


Best regards,
John

John Birch | Strategic Partnerships Manager | Screen
Main Line : +44 1473 831700<tel:%2B44%201473%20831700> | Ext : 2208 | Direct Dial : +44 1473 834532<tel:%2B44%201473%20834532>
Mobile : +44 7919 558380<tel:%2B44%207919%20558380> | Fax : +44 1473 830078<tel:%2B44%201473%20830078>
John.Birch@screensystems.tv<mailto:John.Birch@screensystems.tv> | www.screensystems.tv<http://www.screensystems.tv> | https://twitter.com/screensystems


Visit us at
SMPTE Annual Technical Conference, Loews Hollywood Hotel, Stand 107, October 21-23
Languages & the Media, Hotel Radission Blu, Berlin, November 5-7

P Before printing, think about the environment-----Original Message-----
From: Timed Text Working Group Issue Tracker [mailto:sysbot+tracker@w3.org<mailto:sysbot%2Btracker@w3.org>]
Sent: 21 September 2014 13:35
To: public-tt@w3.org<mailto:public-tt@w3.org>
Subject: ISSUE-346: Need ttp:mediaDuration parameter [TTML2]

ISSUE-346: Need ttp:mediaDuration parameter [TTML2]

http://www.w3.org/AudioVideo/TT/tracker/issues/346


Raised by: Glenn Adams
On product: TTML2

In order to perform ISD processing, it is necessary to know the duration of root external extent. When associate with a related media object, this is the simple duration of the related media object. If this is known at authoring time, then it is an important parameter that should be specified. I propose defining a ttp:mediaDuration parameter attribute that takes either an offset-time form of a time expression or the keyword "indefinite", where the latter is used (or implied) when no related media object exists or its simple duration is unknown or indefinite. If this parameter is not specified, then it would be treated as if indefinite were specified.



This message may contain confidential and/or privileged information. If you are not the intended recipient you must not use, copy, disclose or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. Screen Subtitling Systems Ltd. Registered in England No. 2596832. Registered Office: The Old Rectory, Claydon Church Lane, Claydon, Ipswich, Suffolk, IP6 0EQ


This message may contain confidential and/or privileged information. If you are not the intended recipient you must not use, copy, disclose or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation. Screen Subtitling Systems Ltd. Registered in England No. 2596832. Registered Office: The Old Rectory, Claydon Church Lane, Claydon, Ipswich, Suffolk, IP6 0EQ
  ­­


Received on Friday, 26 September 2014 16:02:55 UTC

This archive was generated by hypermail 2.3.1 : Thursday, 5 October 2017 18:24:18 UTC