Re: Unifying the rendering approach

Hi Philip,

good to see the details emerging now!


On Sun, Feb 23, 2014 at 6:48 PM, Philip Jägenstedt <philipj@opera.com> wrote:
> On Sun, Feb 23, 2014 at 12:44 PM, Silvia Pfeiffer
> <silviapfeiffer1@gmail.com> wrote:
>> Hi Philip,
>>
>> On Sun, Feb 23, 2014 at 4:14 PM, Philip Jägenstedt <philipj@opera.com> wrote:
>>> On Sun, Feb 23, 2014 at 11:05 AM, Silvia Pfeiffer
>>> <silviapfeiffer1@gmail.com> wrote:
>>>> Victor, Philip, all,
>>>>
>>>> Does this make sense to you? If so, I can start preparing a branch
>>>> with patches for this.
>>>
>>> I've read all the mails in this thread but don't see a concrete
>>> suggestion for unification. What concrete changes are we talking
>>> about, and what are they unifying?
>>
>> I was referring in particular to the examples that I had sent with the
>> suggestions at the end.
>
> Thanks, here it is again so I can reply:
>
>> * the spec currently says that cues in region are added one line at a
>> time below existing cue lines -> this will need to be conditioned on
>> the cue being in a scrolling region only
>> * the spec currently prohibits "line" settings on cues in regions -
>> this should be allowed for cues that are not in scrolling regions
>>
>> Once that is done, the rendering of non-scrolling regions and
>> anonymous regions is identical with only scrolling regions having some
>> special rules. That should make it easier to consolidate the rendering
>> section. Have I overlooked any corner cases?
>
> While I don't know precisely which bits of the spec will be changed,
> this sounds good. However, the unification I'm most interested in is
> actually that of scrolling regions with the non-regions case, which is
> what the rest of my email is about:

Right. Well, right now we have non-region cues, non-scrolling region
cues and scrolling region cues. My approach was to make non-region and
non-scrolling region cues the same in the first step by making them
all non-scrolling region cues. Then in a second step I would make more
explicit what actually are the differences when making a region
scrolling. Hopefully we're all thinking similarly, just with different
words. To find that out, I decided to go with examples, since they
make things more explicit (see previous email).


>>> As for an implicit anonymous region, I think that is the wrong way to
>>> phrase this in the spec.
>>
>> I'm surprised about this because I thought at FOMS we had all agreed
>> that this would be the right approach.
>
> I certainly agreed that we should try to make the two cases more
> similar, but I hadn't actually read the rendering algorithm for
> regions at FOMS, so the issue of the big fork in the rendering
> algorithm wasn't really clear to me.

Right.


> From a DOM/CSS point of view, the output of the rendering algorithm is
> a bunch of boxes that are children of the video element.  With regions
> they're instead contained in a regions box that is a child of the
> video element. That's why I think an anonymous or implicit region is
> an odd way to phrase this: there's no need to insert an extra box that
> is the same size as the video for cues that aren't in a region. It's
> not an anonymous region, it's no region at all.

If you look at the rendering of the cues in Chrome (probably the new
Opera, too, though I haven't checked), you will notice that there is
already such a box (the video::-webkit-media-text-track-container). It
exists implicitly and it covers the whole viewport. It just seems to
make sense to call this an anonymous 'region' so we can simply use a
single rendering approach based on regions.


> (I suspect that we're simply using words slightly differently, since
> we haven't gone into the details of how to unify the rendering
> before.)
>
>>> Certainly currently it would be completely
>>> wrong since a region changes the rendering algorithm completely.
>>
>> Not really. Most of the rendering between the region and non-region
>> paths is identical. This is why creating an anonymous region around
>> the normal rendering algorithm and making the rest of the non-region
>> rendering approaches available to regions should work.
>
> There's a fork in the rendering algorithm at "If cue's text track cue
> region is null, run the following substeps". Only the non-regions case
> does "Apply WebVTT cue settings". This fork is what I want to
> eliminate to the greatest extent possible. Ideally, the only between
> regions and no regions in the rendering algorithm should be what the
> parent box is, I think.

The "Apply WebVTT cue settings" part of non-region cues needs a full
rewrite to accommodate the new way in which we do vertical and
horizontal positioning. I should probably do this first, so as to
reduce the differences between non-region and region cues.

I do agree that the intent is to make that section usable by region
and non-region cues and all my proposals were working towards that. I
think that my focus was on getting rid of the difference between
non-region and non-scrolling region cues, while you're more concerned
with the scrolling region cues. So let's talk about the scrolling
region cues now.


>>> I'd
>>> like us to have an internal concept of constraining the size available
>>> for layout that is honored for snap-to-line cues and that a region
>>> simply establishes such a constraint. We don't need an anonymous
>>> region because lack of a region is simply a lack of such a
>>> constraint...
>>
>> Are you suggesting to completely drop the region rendering path and
>> try to reconstruct it within the normal layout algorithm by
>> constraining the available size for layout for snap-to-line cues? That
>> would mean that this is the only difference between regions and
>> non-regions? However, what do you do with the other "constraints":
>> width, region anchor, viewport anchor, and the scrolling? You can't
>> replicate scrolling unless you have an explicit box for the cues and
>> for the regions. Just restricting the size for the background box
>> doesn't resolve this IMO.
>
> Yes, the details are fuzzy, but that's exactly the path I think we
> should take. As I've outlined in another mail, I think that scrolling
> should be a property of individual cues, not something that they get
> from being in a region.

I think you're referring to:
http://lists.w3.org/Archives/Public/public-texttracks/2013Nov/0012.html
and
http://lists.w3.org/Archives/Public/public-texttracks/2014Jan/0025.html

I've previously given feedback in
http://lists.w3.org/Archives/Public/public-texttracks/2014Jan/0038.html
, but I am not sure I understood your approach correctly. I don't
believe we should attribute scrolling to individual cues, but instead
keep scrolling behaviour a property of a region as is the intent of
what is currently specified. Let me see if I can explain.

You wrote:
"First position the cue in its preferred location. If it
overlaps any other cue, it move it down until it does not overlap.
Then it would be moved up, pushing along with it as many cues as are
necessary to not cause (new) overlap. This "push" may be animated or
not, subject to author stylesheet and user preference."

Let me ask some clarifying questions based on an example of this.

Rendered situation at which cue2 comes along:
---------
|
|
|
| cue1
---------

cue2 it is moved "down" until it doesn't overlap any more, resulting
in, I assume, a position off screen below cue1 (outside the viewport)
as the first "free" position?

Next it moves up into position -1, moving cue1 up, I assume resulting in:
---------
|
|
| cue1
| cue2
---------

Question: which of the cues has a "scroll" setting? cue1? or cue2? or
both? What happens if one of them doesn't have a "scroll" setting,
e.g. cue1 doesn't have a scroll setting, but cue2 has? Will cue2 get
stuck off screen or will it be displayed?

Just to explain how the scrolling has been specified right now:

The region gets the following CSS setting (amongst else):
* the 'display' property must be set to 'inline-flex'
* the 'flex-flow' property must be set to 'column'
* the 'justify-content' property must be set to 'flex-end'
If region's text track region scroll setting is 'up' and region
already has one child, set region's 'transition-property' to 'top' and
'transition-duration' to '0.433s'.

Then the 'top' property of the region is adjusted based on the number
of lines rendered into it the region. I.e. the first one sets 'top' to
the bottom of the region box plus the height of the number of lines
rendered into the region. Once the region is "full" (e.g. when 3 lines
are rendered into a region of height 3), then the "top" property
doesn't change any more and cues simply scroll out of the region and
disappear.

I've implemented that here:
http://html5videoguide.net/test/WebVTT-with-regions/player.html
Just hit the "play" button.

It has the advantage that you don't have to check for any overlaps for
scrolling cues at all. You just deal with the region as an entity.

I must admit, I just now checked the spec and although I thought that
I had specified that last bit of how to adapt the 'top' property, I
can't find it any more (doh). I will prepare a patch to fix that,
since right now the spec is broken for this. (This is actually
https://www.w3.org/Bugs/Public/show_bug.cgi?id=22114 - fix i)).



>> Are you referring to the case of giving a cue a position offset in a region?
>> I know that this is an explicit feature of CEA708. It allows rendering
>> of the following (i.e. indented lines in a region):
>>
>> ---------
>> | This line is left aligned in the region.
>> |      This line has an indent.
>> | This line is again left aligned.
>> ----------
>>
>> Each line is a separate cue in this example.
>> Ken (+CC) can clarify if this is a feature of CEA708 (PAC) is in
>> actual use - I just made sure we can replicate it in regions.
>
> I can't find anything in the regions rendering path that does anything
> with any of the cue-level position settings, that's all in the "Apply
> WebVTT cue settings" which isn't run. Am I missing it, or is it TDB?

It's all in the "Otherwise, run the following substeps:" part of
rendering step 12.
Point 6 deals with the position setting.
Point 7 deals with the align setting.
The only cue settings that are currently ignored in a region are
"vertical" (we were going to specify scrolling vertical regions later
- now is probably a good time), "line" (as suggested above, this
should be added for non-scrolling regions) and "size" (actually, this
should now get added as well, which is now possible because we've
changed the way "size" is interpreted in non-region cues, too).

So, I think for non-scrolling regions, making all the WebVTT cue
settings work the same as for non-region cues should be possible. For
scrolling regions, we only need to disallow explicit line positioning
and do the extra CSS work as explained above.


> In any event, I don't think it would actually involve any extra
> complexity to have non-snap-to-lines cues scrolling in a region, but
> it certainly seems like an odd case.

Agreed. That's why we disallow the "line" cue setting on a cue that is
part of a region. However, in future we can move that restriction to
only apply to cues within a *scrolling* region.


> The indent case you give can be
> done with position/size/align AFACIT, it wouldn't require resorting to
> a percentage line.

Right, I did not follow what you meant. Ignore that last bit about
indent, since it has nothing to do with line positioning.

Cheers,
Silvia.

Received on Monday, 24 February 2014 04:34:11 UTC