Re: [ttml2] Does tts:backgroundImage affect size of content area?

On Mon, Apr 18, 2016 at 7:31 PM, Michael Dolan <mdolan@newtbt.com> wrote:

> Sorry, Glenn. I was not appreciating the new semantics in TTML2.  My head
> was still around TTML1 semantics, not TTML2 and the new semantics around
> div and bpd. Now that I have read it J …
>

OK, as long as you keep in mind that (1) TTML1 doesn't have background
images, so we never dealt with this issue in that context; (2) ST2052 and
apparently IMSC1 (though this latter is only assumed) deviate from
XSL-FO/CSS semantics (of background images) in calling for the background
image size to affect the size of the content rectangle of the area(s)
generated by the container content element (div), and (3) for TTML2, we
need to decide how to deal with this deviation from XSL-FO/CSS semantics, a
decision which will affect how ST2052 and IMSC1 is mapped to TTML2.

I happen to be dealing with this mapping problem just now in TTPE because
it does not format IMSC1 directly, but rather, uses TTX to transform IMSC1
into a TTML2 ISD sequence, then formats this latter using TTML2 semantics.


>
>
> Yes, it is very safe to assume that the DECE validator does not check the
> details from your example.
>
>
>
> This new behavior is outside the original high level requirements for
> image subtitles.  The original requirements were about enabling pixel-level
> authoring control of glyphs specifically including the management of the
> extent of the enclosing area of rendered glyphs (especially horizontal
> extent).  Although originally conceived to mainly address non-Latin fonts,
> I am told that there is growing interest in the use of image subtitles for
> even Latin languages in cinema applications. Maybe the requirements have
> more recently expanded? Maybe it would be good to make blocks/lines of
> images scroll like text? (I’m just guessing here).
>
>
>
> I would recommend reaching out to potential users and ask them how they’d
> like it to behave.
>
>
>
>                 Mike
>
>
>
> *From:* Glenn Adams [mailto:glenn@skynav.com]
> *Sent:* Monday, April 18, 2016 4:53 PM
>
> *To:* Michael Dolan <mdolan@newtbt.com>
> *Cc:* TTWG <public-tt@w3.org>
> *Subject:* Re: [ttml2] Does tts:backgroundImage affect size of content
> area?
>
>
>
>
>
>
>
> On Mon, Apr 18, 2016 at 4:51 PM, Michael Dolan <mdolan@newtbt.com> wrote:
>
> >> BTW, what do authors do to satisfy the DECE CFF validator? i.e., how
> to they ensure the content size matches?
>
>
>
> The “package” in this case is an ISOBMFF file coded according to ISO
> 14496-30 and DECE CFF. The TTML document and images are in a sample using
> smpte:backgroundImage=”URN:…” to reference them within the sample.
>
>
>
> DECE/IMSC1 made a simplifying constraint on SMPTE-TT where a <region> has
> exactly one <div> that has exactly one image file reference.
> smpte:backgroundImageHorizontal and smpte:backgroundImageVertical are
> forbidden. So, the TTML region extent is compared to the extent of the
> image file that is referenced within the one <div>.  There are not multiple
> <div>’s per region, and thus not multiple image files per region.
>
>
>
> ok, but that test does not test whether the content element (div) has a
> content rectangle that matches the size of its background image;
>
>
>
> that test, only tests if the background image of some content element
> selected into the region will fit or match the geometry of the region;
>
>
>
> as I have pointed out, the content rectangle of a div without any content
> is empty (by XSL-FO/CSS rules). So the background image will not be
> displayed unless, and only unless you do one of the following:
>
>    1. add content to the div, e.g., significant whitespace, that provides
>    adequate extent in the bpd dimension to encompass the image's bpd;
>    2. specify bpd (or height in horizontal WM) on the content element
>    (div) that matches or exceeds the background image's bpd;
>    3. deviate from XSL-FO/CSS rules, e.g., by requiring the background
>    image size to affect the content rectangle size of the content element
>    (div) on which the background image is specified;
>
> Option 1 won't work in IMSC, since you can't mix text with images. Option
> 2 won't work with either ST2052 or IMSC, since there is no bpd (height)
> property to specify. Option 3 seems the only option and is supported by the
> language I cite from ST2052 about making the minimum content rectangle of
> div match the background image size.
>
>
>
> Option 1 might look like:
>
>
>
> <region xml:id='r1' tts:extents='100px 100px'/>
>
> ...
>
> <div region='r1' smpte:backgroundImage='100x100.png'>
>
>   <p tts:fontSize='100px' tts:lineHeight='100px'>&#x00A0;</p>
>
> </div>
>
>
>
> Here, we add content to the div, a single NBSP character, with height
> 100px, to ensure the bpd of the content rectangle is (at least) 100px. The
> ipd of the content rectangle of the div gets the ipd from the region
> (100px), so we have matched the size of the content rectangle of the div to
> the background image size.
>
>
>
> I'm sure that the DECE CFF validator doesn't test this.
>
>
>
> Option 2 might look like:
>
>
>
> <div region='r1' smpte:backgroundImage='100x100.png' tts:bpd='100px'/>
>
>
>
> of course there is no tts:bpd in ST2052 or IMSC1.
>
>
>
> In TTML2, one could have either:
>
>
>
> Option 1'
>
>
>
> <div region='r1'>
>
>   <image src='100x100.png'/>
>
> </div>
>
>
>
> This works by having a block level *content image* as the content of div,
> which does use the intrinsic size of the image to generate an block area
> child of the div's block area.
>
>
>
> or
>
>
>
> Option 2'
>
>
>
> <div region='r1' tts:backgroundImage='100x100.png' tts:bpd='100px'/>
>
>
>
> This works since we can explicitly style the div's bpd (height here) in
> TTML2.
>
>
>
>
>
>
>
> Regards,
>
>                 Mike
>
>
>
> *From:* Glenn Adams [mailto:glenn@skynav.com]
> *Sent:* Monday, April 18, 2016 2:05 PM
>
>
> *To:* Michael Dolan <mdolan@newtbt.com>
> *Cc:* TTWG <public-tt@w3.org>
> *Subject:* Re: [ttml2] Does tts:backgroundImage affect size of content
> area?
>
>
>
>
>
>
>
> On Mon, Apr 18, 2016 at 2:51 PM, Michael Dolan <mdolan@newtbt.com> wrote:
>
> I see.  I did not note the zero height div.  But there is no text writing
> in the IMSC1 image profile (text is forbidden).
>
>
>
> Exactly. So in IMSC1, even if you wanted to match the content size to the
> background image size, you don't have any way to do that explicitly, other
> than by relying on the semantics of ST2052-1:2013 with respect to making
> the minimum width and height be at least as large as the background image
> size (which is at variance with CSS).
>
>
>
>
>
> Required extent matching was the intent in ST 2052-1 regardless of how
> softly it was stated. And, FYI the DECE CFF validator would reject images
> that do not match.
>
>
>
> If it is required to match, then I wonder why the language
>
>
>
> *The min-height and min-width of the area associated with the image shall
> be set to the intrinsic height and width of the image source. *
>
>
>
> This effectively makes the content size matching requirement rather
> pointless.
>
>
>
> BTW, what do authors do to satisfy the DECE CFF validator? i.e., how to
> they ensure the content size matches?
>
>
>
>
>
> I was only trying to point out that a 3rd option is to forbid the
> mismatch.
>
>
>
> I don't think we need to do this in TTML2, since there are well defined
> XSL-FO/CSS semantics for how to handle this case. In any case, I don't see
> it being forbidden by ST2052-1:2013 or IMSC1, regardless of whether the
> DECE validator would fail that content. I don't even see how the DECE
> validator can check this constraint for that matter.
>
>
>
>   If that still doesn’t work, that’s fine.  Of the other options you have
> proposed, I don’t have a preference.
>
>
>
> Regards,
>
>
>
>                 Mike
>
>
>
> *From:* Glenn Adams [mailto:glenn@skynav.com]
> *Sent:* Monday, April 18, 2016 1:23 PM
>
>
> *To:* Michael Dolan <mdolan@newtbt.com>
> *Cc:* TTWG <public-tt@w3.org>
> *Subject:* Re: [ttml2] Does tts:backgroundImage affect size of content
> area?
>
>
>
>
>
> On Mon, Apr 18, 2016 at 1:53 PM, Michael Dolan <mdolan@newtbt.com> wrote:
>
> Perhaps I am missing something subtle.  I thought you were trying to
> prescribe decoder behavior when the target content element size does not
> match the referenced image size.  This condition is an authoring error in
> SMPTE-TT, so the behavior is undefined (although arguably the decoder could
> reject the package).
>
>
>
> The subtlety is that a content element, e.g., div, has a size of zero in
> the block progression dimension (height in horizontal writing modes). So if
> that div has a background image, it will not be displayed (in XSL-FO and
> CSS).
>
>
>
> However, ST2052-1:2013 does something different, when it states (in
> §5.5.6):
>
>
>
> The min-height and min-width of the area associated with the image shall
> be set to the intrinsic height and width of the image source.
>
>
>
> The example I included in the issue [1] shows that CSS does not use this
> behavior.
>
>
>
> [1] https://github.com/w3c/ttml2/issues/157
>
>
>
> So, as we introduce tts:backgroundImage to TTML2, it is natural that we
> consider whether to adopt XSL-FO/CSS behavior (my preference) or
> SMPTE-TT/IMSC1 behavior.
>
>
>
> Also, the above cited section in ST2052-1 also goes on to say:
>
>
>
> authors should ensure that the div will be sized to match the given
> pre-rendering
>
>
>
> But it doesn't mandate this or make it invalid to fail to do this.
> Further, it begs the question of how one would ensure the div will be
> adequately sized. Since ST2052-1 (nor TTML1) defines a way to explicitly
> size a content element (like div), one wonders how one can satisfy this
> condition [that the div will be sized to match].
>
>
>
>
>
> If TTML2 wishes to allow this situation and thus define decoder behavior
> for it, OK.
>
>
>
> I don't see where ST2052-1 or IMSC1 prohibits "this situation" as you have
> described it. Just the opposite, I don't see how one can ensure the size of
> the content matches or exceeds the background image size. For ST2052-1, you
> could try to include invisible text content (by using tts:visibility of
> 'hidden'), but it will be difficult to get that text size to match the
> background image size. In IMSC1 you can't even do this, since you can't
> include text at all.
>
>
>
> Then I guess I don’t have a position.  Alternatively TTML2 could also
> declare that condition to be an error and therefore not address it in the
> first place, which is what I was trying to suggest.
>
>
>
>                 Mike
>
>
>
> *From:* Glenn Adams [mailto:glenn@skynav.com]
> *Sent:* Monday, April 18, 2016 12:21 PM
>
>
> *To:* Michael Dolan <mdolan@newtbt.com>
> *Cc:* TTWG <public-tt@w3.org>
> *Subject:* Re: [ttml2] Does tts:backgroundImage affect size of content
> area?
>
>
>
> I'm afraid I'm still not following you. My comment is not related to
> validity, but the semantics of background image impact on sizing the
> content element that refers to it.
>
>
>
> SMPTE-TT says that the minimum size of the area generated by the content
> element is expanded to fit the size of the background image. This is not
> the behavior in CSS or XSL-FO, so it presents an issue for how to deal with
> background image in TTML2 and how to translate SMPTE-TT or IMSC1 to TTML2.
>
>
>
> My current thinking is that tts:backgroundImage in TTML2 will not have the
> behavior of SMPTE-TT or IMSC1, i.e., it will follow the XSL-FO and CSS
> behavior. So if background image is used in TTML2, the author will need to
> ensure the size of the generated area is sufficiently large, e.g., by
> including sufficient content or by using tts:ipd or tts:bpd.
>
>
>
> For the purpose of translating SMPTE-TT or IMSC1 to TTML2, a couple
> options exist:
>
>    - translate to <image> element rather than tts:backgroundCheck
>    - or use tts:backgroundCheck but also specify tts:{ipd,bpd}
>
> I don't see any issue here with respect to validity.
>
>
>
> On Mon, Apr 18, 2016 at 11:59 AM, Michael Dolan <mdolan@newtbt.com> wrote:
>
> The IMSC1 instance document alone can be valid IMSC1.  But in the context
> of it with the collection of (e.g. PNG) image file properties, the
> “package” is not valid.  Just because we don’t have a formalization of the
> “package” doesn’t mean that a decoder should adopt some compensation
> behavior when faced with such a “package”.
>
>
>
>                 Mike
>
>
>
> *From:* Glenn Adams [mailto:glenn@skynav.com]
> *Sent:* Monday, April 18, 2016 9:26 AM
> *To:* Michael Dolan <mdolan@newtbt.com>
> *Cc:* TTWG <public-tt@w3.org>
> *Subject:* Re: [ttml2] Does tts:backgroundImage affect size of content
> area?
>
>
>
> Could you please explain? What is invalid?
>
>
>
> On Mon, Apr 18, 2016 at 8:46 AM, Michael Dolan <mdolan@newtbt.com> wrote:
>
> This condition is an invalid document and thus decoder behavior is
> undefined.
>
>         Mike
>
>
> -----Original Message-----
> From: Glenn Adams via GitHub [mailto:sysbot+gh@w3.org]
> Sent: Saturday, April 16, 2016 1:47 PM
> To: public-tt@w3.org
> Subject: [ttml2] Does tts:backgroundImage affect size of content area?
>
> skynavga has just created a new issue for
> https://github.com/w3c/ttml2:
>
> == Does tts:backgroundImage affect size of content area? == According to
> SMTPE ST2052-1:2013 §5.5.6:
>
> > The min-height and min-width of the area associated with the image
> shall be set to the intrinsic height and width of the image source.
>
> and
>
> > The presented image shall not be scaled and the XSL background-color
>  trait shall be visible for any background areas of the ``<div>``
> outside the image, therefore, authors should ensure that the div will
> be sized to match the given pre-rendering.
>
> The effects of these two requirements are that if an image is larger
> than the nominal minimum content size of the generated area, then that
>  minimal content size is to be increased to encompass the intrinsic
> image size as required. While if the nominal minimum content size is
> already larger than the intrinsic image size, the background color of
> the generated area (if not transparent) will be visible outsize the
> image, i.e., the image is not scaled in either dimension to fill the
> nominal minimum content size in such a case.
>
> IMSC1, which makes use of the ``smpte:backgroundImage`` extension,
> does not specify any details about sizing, other than to say that the
> intrinsic image size must correspond to the size of the region into
> which its associated generated area(s) are flowed. As such, one may
> presume that IMSC1 intends the above language from ST2052 to apply
> (Pierre?).
>
> The problem here is that this behavior is at variance with both XSL-FO
>  and CSS, where the minimum content size of generated areas associated
>  with a background image **is not** increased to the intrinsic image
> size. In other words, in XSL-FO or CSS, if there is no content in the
> element, then no area is generated, or if one is generated, it has
> zero for one or both content area dimensions. On the other hand, if
> there is content in the element, but its nominal formatted size is
> smaller than the intrinsic size of the background image, then some of
> the image will be clipped. I am attaching a test file that
> demonstrates the CSS behavior below.
>
> Now, the problem here in TTML2 is how to handle this discrepancy
> w.r.t. XSL-FO and CSS introduced by SMPTE-TT and apparently adopted by
>  IMSC1. One way to handle it would be to require that ``tts:ipd`` and
> ``tts:bpd`` be used with TTML2 if one wants such behavior with a
> background image (in a like manner to how one is forced to specify
> width and height properties in XSL-FO or CSS). Or we could add a
> ``tts:{ipdMinimum,bpdMinimum}`` pair in TTML2 to specify instead. Or
> we could generalize the values of ``tts:{ipd,bpd}`` to allow
> specifying a (minimum, optimum, maximum) tuple. Or we could allow the
> minimum values of these properties to be specified with a keyword, say
>  ``backgroundImageSize``.
>
> In any case, we have an issue here, so I have added an editorial note
> to the section on
> (tts:backgroundImage)[
> https://rawgit.com/w3c/ttml2/master/spec/ttml2.html#style-attribute-backgroundImage
> ]
>  in the TTML2 ED.
>
> Following is the CSS example file:
>
> [test.html.zip](https://github.com/w3c/ttml2/files/222397/test.html.zip)
>
>
>
> Please view or discuss this issue at
> https://github.com/w3c/ttml2/issues/157 using your GitHub account
>
>
>
>
>
>
>
>
>
>
>