RE: ISSUE-98 (Clear timing.): Eliminate separate clear timing from dynamicFlow [DFXP 1.0]

[SH] First off, thanks for doing the additional archaeology here. I find the public archive not the easiest to deal with when trying to trace down these details. I was not trying to suggest that there was no discussion on this, just that it didn't seem to be recorded in the public archive.

[GA] Notwithstanding John's further comments in [7], my original formulation of Appendix B was an attempt to formalize the Temporal Flow Model described in [1]. In my attemt to formalize the Temporal Flow Event Timing of [1], I interpreted "after the fill interval (i.e., at t = t + fill interval)" to mean the time at which the fill timer expires, and I interpreted "after the inter-fill interval (i.e., at t = t + inter-fill interval)" to mean the time at which the clear timer expires. In this context, I mapped the value computed by the intra() flow interval function to the notion of fill interval and the value computed by the intra() flow interval function to the notion of inter-fill interval. Notice that the text of [1] refers to these two events as fill event(s) and clear event(s), respectively. So I think the terminology I used in Appendix B is consistent with [1].

[SH] I think the problem here has largely been that we have had no actual working model to ground this discussion, so it has been difficult to guarantee that any of this actually works, or that there are not corner case logic errors. To that end I have this weekend created a simulator for Section B.5 and B.4 (at least for the character word and line modes). While this doesn't count as an implementation of dynamicFlow as such (it inputs the values directly and doesn't use timed text at all), it did go a long way to illuminating the issues for me.

What I've found is that while what is specified today seems to work after a fashion, the results are in fact somewhat less than useful.

The first thing I have discovered is that there are indeed issues in the timing behavior described by the current WD.  One is  that that the behavior described in [1] below:

A clear event is allowed to occur when there is content selected into a region that is not currently displayed in that region AND the region is full. A clear event is also allowed to occur when content is de-selected from a region in which it is currently displayed.
Is not met. In [1] "after the inter-fill interval (i.e., at t = t + inter-fill interval)" is within a condition.  If we look at the full algorithm in [1].

 *   At the instant content is first selected into a region, a fill event occurs (the initial fill event is always at t = 0).
 *   The following process is then repeated until no further content is available for display - or until the region is no longer active.
 *   If not explicitly defined, the fill interval and inter-fill intervals are computed (see below).
 *   IF the region is not full (i.e. a unit of addition will fit in the region), the next fill event occurs after the fill interval (i.e. at t = t + fill interval).
 *   ELSE (the region is full - i.e. it has insufficient space to accept the current unit of addition), a clear event occurs after the inter-fill interval (i.e. at t = t + inter-fill interval). A re-flow event may be generated as a result of this clear event. If the region is not full after this clear event (and any subsequent re-flow event), a single fill event occurs. If the inter-fill interval is greater or equal to the fill interval, (i.e. the fill interval has been satisfied) the fill event occurs immediately. If the fill-interval is greater than (or equal to) the inter-fill interval, the fill event occurs at the time point (measured from the last fill event) that satisfies the fill interval. This time point will be (fill-interval - inter-fill interval) after the clear event.
 *   REPEAT.
While not exactly crystal clear, I think the intention seems to have been that the fill interval alone applies up to the point the region is full. When the region is full, the inter-fill interval starts to apply to insert a clear (and possible reflow) event before, or concurrently with, the next timed fill event (in which case it may suspend the fill interval).

In the current formulation the clear operation is too 'greedy', and starts work immediately. Thus it is extremely difficult to balance the clear and fill rates to maintain any kind adequately readable region.

The next problem is that the two timers are not independent, and we have the clear operation effectively performing a fill and resetting the fill timer, therefore the rate of flow through the region  doesn't stay  constant throughout. What I think was intended seems to have been a queue process which is trying to feed content in at one end, and take it out of the other, maintaining as much as possible a full pipeline and constant speed.

To capture the behavior implied in [1], the current draft could be modified as follows:
B.5.2 Clear Timer

The clear timer is used to pace the removal of formatted content from the presentation region undergoing dynamic flow processing.

1.      If the presentation region contains insufficient space to accept a unit of formatted content that corresponds to a fill unit, or the fill buffer is empty and the presentation region is non-empty then perform the processing described in B.4.3 Clear Operation<http://www.w3.org/TR/ttaf1-dfxp/#flowOperationClear>;

2.      If the space created by performing the clear operation indicated by the above step would be sufficient to accept a unit of formatted content that corresponds to a reflow unit and if sufficient formatted content remains after this point in the presentation region, then perform the processing described in B.4.2 Reflow Operation<http://www.w3.org/TR/ttaf1-dfxp/#flowOperationReflow>, where this operation commences at the same time as the clear operation;

3.      If the fill buffer is non-empty, then reset the clear timer to the current value of the computed clear interval.
In the highlighted section (part 1 above if the formatting is lost), the condition has been modified to apply the clear operation only when the fill is blocked, which addresses the startup mode, or if the fill buffer is exhausted which addresses the teardown mode. The steady state operates with the presentation region continually being filled and cleared. We don't want the clear to affect the fill operation at all, and just let the fill timer work on the fill.

NB If there is any content in the presentation buffer, it will always constitute a character unit or (trivial) word, line or block unit; so I have removed that requirement; it may be required if we can define inline effectively.

The other modification is to change the fill timer so it no longer refers to the clear timer.
B.5.1 Fill Timer

The fill timer is used to pace the addition of formatted content to the presentation region undergoing dynamic flow processing.

Each time the fill timer expires, the following ordered steps are performed:

1.      If the fill buffer is non-empty and contains sufficient content to compose a fill unit and if sufficient space is present in the presentation region to accept a unit of formatted content that corresponds to a fill unit, then perform the processing described in B.4.1 Fill Operation<http://www.w3.org/TR/ttaf1-dfxp/#flowOperationFill>, where, if the fill timer was expired or simulated as a result of performing a prior reflow operation, this operation commences at the same time as the reflow operation;

2.      If the fill buffer is non-empty, then reset the fill timer to the current value of the computed fill interval;
Now we have cleanly separated the two timers. If we adopt this fix (which I think we need to otherwise the model is pretty unusable), it probably makes sense to define that the clear interval is always the same as the fill interval, and that both are controlled by the inter parameter.

I also implemented the above model and it seems to work reasonably well.

I still think it would be better to combine the fill and clear operations into a singled timed operation which would simplify the exposition considerably, Also there seems to be nothing in the current draft which actually corresponds to the original meaning of the inter-fill timing; which was a pause between determining a clear is required and actually performing the clear. If we are going to have a second intra time, then we could introduce this pause.
However doing all that may be too big a change. I haven't investigated how to adapt the specification to do this, but now I have actual code corresponding to annex B.5, I probably could do so, but given my next issue, I think this may be a waste of time.

Having possibly dealt with the timing issue and played around with the various modes, I'm now concerned we are going to a lot of trouble to specify a complex processing model, despite the fact that only a few combinations of in and out unit actually make practical sense.

The main problem is that XSL:FO line breaking rules don't admit a break at an arbitrary point; (and I believe we at one point decided not to require complex hyphenation processing although I haven't traced that decision down yet). Whereas I believe John was basing his design on a teletext like model which places individual characters, and breaks words at arbitrary points.

Thus in timed text when applying wrapping, line breaks basically occur at whitespace boundaries, as in HTML. This means that for example, in the character out mode, the first line will flow out at a character rate while subsequent lines flow out at word rates, and since words in English are of differing lengths in multi line blocks the reflowing process in each line is also occurring at different rates. Thus we don't get the nice 'snake' mode described by John, but an extremely visually disconcerting mess. Similar remarks apply to word out mode.

With the XSL:FO line-break model, character and word out dynamicFlow modes really only make sense when the region is constrained to a single line. Also when the content contains paragraph blocks, then the second and subsequent paragraphs remain visually static, while the first undergoes flowing - again this looks very odd and is not very readable.

So, If we wanted to retain character and word out modes for dynamicFlow over multiline regions for a real 'snake', I think we would need to define a 'wrapCharacter' mode for the wrapOption style which ignores the language processing rules and only breaks at the region edge. This would be a new feature we would need to describe however, and one which is not part of XSL:FO.  So I think it better if we simply drop the word and out character modes, possibly defining a shortcut "crawl" mode like we did for rollup which creates a single line tickertape model.

Line out mode does actually seem to behave well with all three in modes, as it interoperates much better with the XSL:FO line wrapping behavior, however even here (line out/character in) doesn't seem like a very useful option. (Line out/word in) looks like the classic live stenographer mode of captioning, and so may be useful to keep; although whether this could actually be deployed in a live production I'm not sure.

Line out/Line in is the rollup shorthand we have defined.

Thus of the 9 possible combinations:

Character out  - {character,word } in  : OK in single line crawl applications. Does not work in multiline
Character out  - line in  :  Strange behavior in that the first line scrolls out while the rest remain static until there is space for a new word on the first line when everything reflows.

Word out  - {character,word } in  : OK in single line crawl applications. Does not work in multiline
Word out  - line in  :  Even more visual disruption as words flow out when everything reflows.

Line out - character in : visually OK, not very useful.
Line out - word in : visually OK, steno mode.
Line out - line in : visually OK, rollup.

The inline unit and block unit are not very well defined, so I haven't explored those but I expect similar problems. A region-full mode would behave very much like line mode, that is it would probably only work as an out mode, and the only practical behavior being for region in region out, which could be captured as a pop-on shortcut.

So, I'm increasingly of the opinion we ought to dump all of the syntactic options for dynamicFlow except for 4 modes: pop-on , rollup, crawl and steno. Possibly allowing each to take an optional rate. Whether we then describe each of these directly, or in terms of Annex B processing I'm not sure.

Sean Hayes
Media Accessibility Strategist
Accessibility Business Unit
Microsoft

Office:  +44 118 909 5867,
Mobile: +44 7875 091385

From: public-tt-request@w3.org [mailto:public-tt-request@w3.org] On Behalf Of Glenn A. Adams
Sent: 16 May 2009 4:18 AM
To: Public TTWG List
Subject: Re: ISSUE-98 (Clear timing.): Eliminate separate clear timing from dynamicFlow [DFXP 1.0]


inline

On 5/15/09 5:49 PM, "Timed Text Working Group Issue Tracker" <sysbot+tracker@w3.org> wrote:

>
> ISSUE-98 (Clear timing.): Eliminate separate clear timing from dynamicFlow
> [DFXP 1.0]
>
> http://www.w3.org/AudioVideo/TT/tracker/issues/98
>
> Raised by: Sean Hayes
> On product: DFXP 1.0
>
> I can see no justification for having separate clear timing in any of the
> examples from John Birch's original documents (the only hard evidence I can
> find in the history trail of a design discussion). The notion of a separate
> clear timer seems to have been added between the March 14 draft and March 21
> draft without discussion on the public list. What is more, the presence of a
> separate clear interval seems at somewhat at odds with the behavioral
> requirement.

[GA] I guess you are referring to the 2005 WDs. In any case, the notion of clear timing was present in the draft Timed Text 1.0 Fill Module [1], from as early as May 2004, where it is included in the concept of temporal inter-fill interval, which is the origin of our current syntax expressed by the inter() flow interval function.

[1] http://lists.w3.org/Archives/Member/member-tt/2004Sep/att-0036/fillscroll_JB.html

In the March 14, 2005 WD, this was defined at [2], and expressed in terms of clearing (and reflowing) content, as follows (emphasis added):

[2] http://www.w3.org/TR/2005/WD-ttaf1-dfxp-20050314/#style-attribute-dynamicFlow
If one <duration> value is specified, then it denotes the time interval to wait between dynamic fill insertions. If two <duration> values are specified, then the second denotes the time interval to wait prior to clearing content from and flowing content within the region after the region has become full as a result of a dynamic fill insertion. The former <duration> is referred to as the fill interval, while the latter is referred to as the inter-fill interval.

The enhanced definition in the March 21, 2005 WD was the result of elaborating the syntax and the accompanying prose, so there was no new concept being added there. Indeed, see [3] where an improved syntax was being discussed on March 15, 2005, where it is referred to as interFillInterval.

[3] http://lists.w3.org/Archives/Member/member-tt/2005Mar/0030.html

Then there was the lengthy discussion of dynamic flow semantics, including clear timing, a few weeks later on Days 2 and 3 (April 12-13, 2005) of the face-to-face meeting in Cupertino [4], at which you were present (and actively discussing given your recorded comments at the time).

[4] http://lists.w3.org/Archives/Member/member-tt/2005Apr/0011.html

There seems to be a good dialogue recorded on the issues of independent timers by John and I at [7] in early 2006, in his review of what was (then) to be published in Appendix B.

[7] http://lists.w3.org/Archives/Member/member-tt/2006Feb/0015.html

I note that in the above citation that John's model for this "inter-fill interval" is not exactly the same as, but is more comprehensive than the strict notion of a clear timer interval as we have presently defined it:
"The inter interval is intended to define a period when the display must be stationary, this would typically be after a fill event. So for example a new line is added to the display, the display remains static for a period, then clear, reflow and fill events may again take place. For a smoothly scrolling region, the inter interval would be defined as 0."

He goes on to say:
"Is it possible that for a completely comprehensive model we need three timers, or perhaps a better way of defining how flow events may be related?

I personally think two time values are sufficient, one to set the rate of new flow units... the second to enforce a period of stasis on the display to allow easy reading. I do not consider it useful to be able to set a delay interval between a clear operation and a susequent reflow... but others may have a different view.

Perhaps the timing would be best defined as

* a fill interval (period between fill operations) - thus setting flow rate through presentation container... A master interval that by definition constrains other intervals.
* a post clear delay (period after clear but before any reflow)
* a post reflow delay (period before any subsequent fill)
* a post fill delay (period after a fill operation - before any subsequent clear)

Of course by defining the fill intreval and two of the above delays, the third delay is effectively defined. The only other wrinkle is that if events are synchronous, e.g. A synchronous clear and reflow, but a non synchronous fill. In these cases, I would suggest that the valid delay would be the value associated with the last logical operation in a synchronous set of operations. So in the just preceeding example (synchronous clear and reflow) the reflow delay would be valid, but the clear delay would be ignored....

For smooth scrolling only one interval is required (the fill rate), plus the indication that all flow events are coupled and synchronous. The post fill delay is zero indicating that all of the fill interval is to be used by the transitions of the flow operations.

For paused scrolling two intervals are required, (the fill rate), a stasis time for an interval between the fill and a clear (the post fill delay), plus the indication that all flow events are coupled and synchronous."
Notwithstanding John's further comments in [7], my original formulation of Appendix B was an attempt to formalize the Temporal Flow Model described in [1]. In my attemt to formalize the Temporal Flow Event Timing of [1], I interpreted "after the fill interval (i.e., at t = t + fill interval)" to mean the time at which the fill timer expires, and I interpreted "after the inter-fill interval (i.e., at t = t + inter-fill interval)" to mean the time at which the clear timer expires. In this context, I mapped the value computed by the intra() flow interval function to the notion of fill interval and the value computed by the intra() flow interval function to the notion of inter-fill interval. Notice that the text of [1] refers to these two events as fill event(s) and clear event(s), respectively. So I think the terminology I used in Appendix B is consistent with [1].

>
> If I have a two line caption region , and I wish to flow the text through it,
> with text entering at a word rate and leaving at a line rate. Then I don't
> want a clear to occur until after two lines have been added, but then I want
> to clear the lines after each additional line is completed; which implies
> clear timing can't start until the region starts to overflow, so the first
> clear interval needs to be extended, and after that we want the clear interval
> to be synchronized with the fill interval which is based on irregular length
> words, so a single value seems inappropriate.
>
> In the example flow below: The first clear/reflow occurs at t14, the next at
> t20, then at t27 and at t33  (clear intervals of 14, 6, 7 6), then we
> presumably want to clear the remaining two lines to be cleared in some manner.
>
> T01 Alice
>
> t02 Alice was
>
> ....
>
> t06 Alice was beginning to get very
>
> t07 Alice was beginning to get very
>        tired
>
> ....
>
> t13 Alice was beginning to get very
>        tired of sitting by her sister on
>
> t14 tired of sitting by her sister on
>        the
> ...
> T19 tired of sitting by her sister on
>         the bank and of having nothing
>
> T20 the bank and of having nothing
>         to
> ....
>
> T26 the bank and of having nothing
>         to do: once or twice she had
>
> T27 to do: once or twice she had
>         peeped
>
> T33 peeped into the book her sister
>         was
>
> T34 peeped into the book her sister
>         was reading
>
>
>
>
>