- From: Pierre-Anthony Lemieux <pal@sandflow.com>
- Date: Wed, 9 Jul 2014 15:51:20 -0700
- To: Nigel Megitt <nigel.megitt@bbc.co.uk>
- Cc: Timed Text Working Group <public-tt@w3.org>
Hi Nigel, Mapping media time expression to video frames is to only means of allowing authors to unambiguously specify on which frame of the related video object a subtitle will appear. If a system chooses to decimate the video for distribution, i.e. after the content has been authored and QCed, the system can simply specify that the video be played back at the original frame rate, at least for the purposes of subtitle/caption rendering. In other words, the related video object at playback will have the same frame rate as during authoring. Best, -- Pierre On Fri, Jun 20, 2014 at 11:34 AM, Nigel Megitt <nigel.megitt@bbc.co.uk> wrote: > Hi Pierre, > > Comments in line: > > On 29/05/2014 07:20, "Pierre-Anthony Lemieux" <pal@sandflow.com> wrote: > >>Hi Nigel, >> >>As I understand it, the motivation to map media time expression to >>video frames is to allow authors to unambiguously determine on which >>frame of the related video object a subtitle will appear. I would >>think this is particularly important on scene boundaries. > > I formerly was of that opinion too; however I now believe, from user > feedback, that this is not a strong enough concern to create a design > constraint for. Certainly it is less important than maintaining a maximum > reading speed and equivalent flow of content to the audio to which > subtitles and captions. Maximum reading speed is a matter of regulation in > the UK. > >>This unambiguous determination is not possible if the time expression >>mapping depends on the display frame rate, which is not known at time >>of authoring and largely arbitrary. >> >>I would expect the timed text content to remain usable in the current >>approach as long as the duration of intermediate synchronic documents >>duration remains larger than the frame duration. Is there a use case >>where this would systematically not be the case? > > I do not believe that such a rule would achieve the requirements. Even > with ISD durations that are longer than frame duration quantisation > effects could in principle reduce the time available for reading each > word, especially for low frame rates. I would consider 6 frames per second > to be potential and realistic targets for distribution of video that goes > alongside captions and subtitles. > >>In general, without more information on the use case below (e.g. what >>is the minimum frame rates/why not use reduced resolution instead, how >>frequently will this frame rate reduction happen, is IMSC1 even >>applicable/desirable...), I am reluctant to suggest a change to an >>approach that appears to be working for users. > > Reduced resolution is a comparable but separate topic. > > Kind regards, > > Nigel > > >> >>Best, >> >>-- Pierre >> >>On Tue, May 27, 2014 at 4:07 AM, Nigel Megitt <nigel.megitt@bbc.co.uk> >>wrote: >>> Hi Pierre, >>> >>>>>Yes, they are able to reduce frame rate as an available >>>>>option for adapting to network conditions. >>>> >>>>Is the BBC doing this? If not, who? Just trying to fully understand >>>>the use case. >>> >>> You're hitting my limits on how much I can freely discuss on this forum. >>> Reduced frame rate distribution options are likely to be used in the way >>> I've described by organisations you've heard of within the lifetime of >>> IMSC1. Knowing who won't enhance the use case further. >>> >>> Kind regards, >>> >>> Nigel >>> >>> >>> >>> On 23/05/2014 17:46, "Pierre-Anthony Lemieux" <pal@sandflow.com> wrote: >>> >>>>Hi Nigel, >>>> >>>>> Yes, they are able to reduce frame rate as an available >>>>> option for adapting to network conditions. >>>> >>>>Is the BBC doing this? If not, who? Just trying to fully understand >>>>the use case. >>>> >>>>Thanks, >>>> >>>>-- Pierre >>>> >>>>On Fri, May 23, 2014 at 9:24 AM, Nigel Megitt <nigel.megitt@bbc.co.uk> >>>>wrote: >>>>> On 23/05/2014 17:20, "Pierre-Anthony Lemieux" <pal@sandflow.com> >>>>>wrote: >>>>> >>>>>>> Yes - it's likely that adaptive streaming distribution mechanisms >>>>>>> will do this. >>>>>> >>>>>>Do they do this today? >>>>> >>>>> Yes, they are able to reduce frame rate as an available option for >>>>> adapting to network conditions. >>>>> >>>>> Kind regards, >>>>> >>>>> Nigel >>>>> >>>>>> >>>>>>Best, >>>>>> >>>>>>-- Pierre >>>>>> >>>>>>On Fri, May 23, 2014 at 9:19 AM, Nigel Megitt <nigel.megitt@bbc.co.uk> >>>>>>wrote: >>>>>>> Hi Pierre, >>>>>>> >>>>>>>>Hi Nigel, >>>>>>>> >>>>>>>>> 2. Video encoded for distribution at 7.5fps. >>>>>>>> >>>>>>>>Can you point to actual use case/deployment for this? >>>>>>> >>>>>>> Yes - it's likely that adaptive streaming distribution mechanisms >>>>>>>will >>>>>>>do >>>>>>> this. Profiles for this purpose that go down to e.g. 25/4=6.25 fps. >>>>>>>are >>>>>>> likely to be used for mobile devices for example. I assume that >>>>>>>something >>>>>>> similar will result from a starting point of 30fps. >>>>>>> >>>>>>> Kind regards, >>>>>>> >>>>>>> Nigel >>>>>>> >>>>>>>>On Fri, May 23, 2014 at 9:11 AM, Nigel Megitt >>>>>>>><nigel.megitt@bbc.co.uk> >>>>>>>>wrote: >>>>>>>>> Hi Pierre, >>>>>>>>> >>>>>>>>> It would certainly help to explain the model more clearly, along >>>>>>>>>the >>>>>>>>>lines >>>>>>>>> that you've outlined. The specific proposal wouldn't help though, >>>>>>>>>since >>>>>>>>> the precise timing information would have been lost at the point >>>>>>>>>of >>>>>>>>> temporal quantisation and could not be regenerated later. >>>>>>>>> >>>>>>>>> For example this suggests a chain such as: >>>>>>>>> >>>>>>>>> 1. IMSC Document authored against video at 30fps. >>>>>>>>> 2. Video encoded for distribution at 7.5fps. >>>>>>>>> 3. Receiving system must align the resolved TTML time expressions >>>>>>>>>with >>>>>>>>>the >>>>>>>>> 7.5fps 'quanta' prior to display, as per the rule in the current >>>>>>>>>spec. >>>>>>>>> >>>>>>>>> Even if the display device refreshes at 60fps it would be >>>>>>>>>forbidden >>>>>>>>>from >>>>>>>>> using the original timings because the spec references the encoded >>>>>>>>>video. >>>>>>>>> >>>>>>>>> What I'm trying to get to is a solution that is permitted >>>>>>>>>(actually >>>>>>>>> encouraged) to align with display frames as late as possible while >>>>>>>>>losing >>>>>>>>> minimal information. In some real world systems that's unavoidably >>>>>>>>>earlier >>>>>>>>> than the display, but we shouldn't use the lowest common >>>>>>>>>denominator >>>>>>>>>to >>>>>>>>> set the rule for all implementations. >>>>>>>>> >>>>>>>>> Kind regards, >>>>>>>>> >>>>>>>>> Nigel >>>>>>>>> >>>>>>>>> >>>>>>>>> On 23/05/2014 16:33, "Pierre-Anthony Lemieux" <pal@sandflow.com> >>>>>>>>>wrote: >>>>>>>>> >>>>>>>>>>Hi Nigel, >>>>>>>>>> >>>>>>>>>>IMSC 1.0 §4.4 [1] refers to synchronization with the related video >>>>>>>>>>object against which the timed text content is delivered, not >>>>>>>>>>synchronization to the displayed frame rate by the >>>>>>>>>>terminal/UA/device/display/TV. In other words, if a >>>>>>>>>>terminal/UA/device/display/TV chooses to alter the video frame >>>>>>>>>>rate >>>>>>>>>>of >>>>>>>>>>the related video object it receives (for whatever reason), then I >>>>>>>>>>expect it will accordingly alter the timed text display (perhaps >>>>>>>>>>along >>>>>>>>>>the lines of what is suggested below), with the knowledge that the >>>>>>>>>>timed text was authored according to the constraints of Section >>>>>>>>>>4.4. >>>>>>>>>> >>>>>>>>>>Would a note to that effect help? >>>>>>>>>> >>>>>>>>>>Thanks, >>>>>>>>>> >>>>>>>>>>-- Pierre >>>>>>>>>> >>>>>>>>>>On Thu, May 22, 2014 at 3:19 AM, Timed Text Working Group Issue >>>>>>>>>>Tracker <sysbot+tracker@w3.org> wrote: >>>>>>>>>>> ISSUE-317 (IMSC should not require frame alignment): IMSC should >>>>>>>>>>>not >>>>>>>>>>>require frame alignment [TTML IMSC 1.0] >>>>>>>>>>> >>>>>>>>>>> http://www.w3.org/AudioVideo/TT/tracker/issues/317 >>>>>>>>>>> >>>>>>>>>>> Raised by: Nigel Megitt >>>>>>>>>>> On product: TTML IMSC 1.0 >>>>>>>>>>> >>>>>>>>>>> IMSC 1.0 §4.4 [1] currently requires temporal quantisation of >>>>>>>>>>>media >>>>>>>>>>>times to frame display times. This rule comes into play when >>>>>>>>>>>times >>>>>>>>>>>are >>>>>>>>>>>not expressed in frames, and therefore the same document may >>>>>>>>>>>apply >>>>>>>>>>>to a >>>>>>>>>>>range of related media objects covering different frame rates. In >>>>>>>>>>>the >>>>>>>>>>>case when frames are used the document can only be displayed >>>>>>>>>>>alongside >>>>>>>>>>>media of the same frame rate so there's no need for the frame >>>>>>>>>>>alignment >>>>>>>>>>>expression. >>>>>>>>>>> >>>>>>>>>>> This approach prevents implementations from changing caption >>>>>>>>>>>display >>>>>>>>>>>at >>>>>>>>>>>screen refresh rate quantisation and enforces quantisation based >>>>>>>>>>>on >>>>>>>>>>>the >>>>>>>>>>>encoded video frame rate. This means that if a low frame rate >>>>>>>>>>>video >>>>>>>>>>>is >>>>>>>>>>>provided, e.g. quarter rate which could be around 6 frames per >>>>>>>>>>>second, >>>>>>>>>>>the effective word reading rate may be increased to the point >>>>>>>>>>>where >>>>>>>>>>>text >>>>>>>>>>>becomes hard to read. >>>>>>>>>>> >>>>>>>>>>> Consider a streaming environment in which there is enough >>>>>>>>>>>network >>>>>>>>>>>capacity to provide audio and captions but the video experience >>>>>>>>>>>is >>>>>>>>>>>badly >>>>>>>>>>>impacted: in this case it must be permitted that the >>>>>>>>>>>implementation >>>>>>>>>>>continue to present captions alongside the audio regardless of >>>>>>>>>>>the >>>>>>>>>>>frames of video that are displayed. >>>>>>>>>>> >>>>>>>>>>> I propose a solution to this problem that implementations SHALL >>>>>>>>>>>display >>>>>>>>>>>captions as temporally close to the media time specified as the >>>>>>>>>>>display >>>>>>>>>>>device permits, independent of video frame rate. >>>>>>>>>>> >>>>>>>>>>> Note that where frames are used in media time expressions this >>>>>>>>>>>reduces >>>>>>>>>>>to exactly the current behaviour. >>>>>>>>>>> >>>>>>>>>>> [1] >>>>>>>>>>>https://dvcs.w3.org/hg/ttml/raw-file/ea1a92310a27/ttml-ww-profile >>>>>>>>>>>s/ >>>>>>>>>>>tt >>>>>>>>>>>ml >>>>>>>>>>>-w >>>>>>>>>>>w-profiles.html#synchronization >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> >>>>> >>>>> >>>>> >>>>> ----------------------------- >>>>> http://www.bbc.co.uk >>>>> This e-mail (and any attachments) is confidential and >>>>> may contain personal views which are not the views of the BBC unless >>>>>specifically stated. >>>>> If you have received it in >>>>> error, please delete it from your system. >>>>> Do not use, copy or disclose the >>>>> information in any way nor act in reliance on it and notify the sender >>>>> immediately. >>>>> Please note that the BBC monitors e-mails >>>>> sent or received. >>>>> Further communication will signify your consent to >>>>> this. >>>>> ----------------------------- >>> >>> >>> >>> ----------------------------- >>> http://www.bbc.co.uk >>> This e-mail (and any attachments) is confidential and >>> may contain personal views which are not the views of the BBC unless >>>specifically stated. >>> If you have received it in >>> error, please delete it from your system. >>> Do not use, copy or disclose the >>> information in any way nor act in reliance on it and notify the sender >>> immediately. >>> Please note that the BBC monitors e-mails >>> sent or received. >>> Further communication will signify your consent to >>> this. >>> ----------------------------- >
Received on Wednesday, 9 July 2014 22:52:09 UTC