Re: Processor Profile, Content Profile and codecs

On Mon, Oct 19, 2015 at 4:04 AM, Andreas Tai <tai@irt.de> wrote:

> Since years the discussion about the right signaling of TTML derived
> standards inside and outside of TTML documents is popping up every now and
> then. Some of the exchanged arguments are repeated, some of them are new
> because the context of the discussion changes. Like may be others I am
> hesitant to step into the discussion again because there is always the
> danger that we have a circular discussion with no end point.
>
> The TTML profile registry is/was a perspective to meet on common ground
> although it left some ambiguities. Now there are two important new
> circumstances why it is worthwhile to re-think the scenario and fix an
> agreed strategy that can easily be explained to "non" TTML'ers who
> want/need to process TTML documents:
>
> 1) The registry was made for TTML2 but it has become clear that a
> mechanism is needed before TTML2 becomes a recommendation. We certainly
> need a mechanism that can be applied to TTML 1 documents.
>
> 2) We have a detailed review of the approach from the implementation side
> by Cyril (which also includes a standardisation perspective with respect to
> MPEG).
>
> The questions Cyril raised are super important (thanks so much for taking
> the time for looking into the topic). They are so important because the
> review comes from an "external" perspective which is pointing to statements
> that are difficult to understand for implementers.
>
> To further support the current adoption of TTML and it "derivates" it is
> necessary to clarify the best approach to signal conformance/processor
> requirements. Therefore I propose to set this topic on the agenda for the
> next f2f meeting at TPAC on 29./30.10. 2015.
>
> Two arguments from Cyril need from my view special attention:
>
> a) The reasoning behind separation of content and processor profiles is
> not clear and is not made in similar (non TTML) encoder/decoder scenarios.
>

Not only is it completely clear, but it is simply explained (F/Es =
features/extensions):

   - a processor profile defines a set of constraints that apply to what
   F/Es a processor must support in order to process a document, but does not
   constrain what/how F/Es are used in a document;
   - a content profile defines a set of constraints that apply to what/how
   F/Es are used in a document, but does not constrain what features are
   supported by a processor;
   - a processor profile can be inferred content profile; more
   particularly, two types of processor profiles can be inferred: a strict
   processor profile and a loose processor profile, where the difference
   between the two is that if a F/Es is optional in the content profile then
   it is required in the strictly inferred processor profile, but remains
   optional in the loosely inferred processor profile;
   - a content profile cannot be inferred from a processor profile, because
   optional and non-specified F/Es in the processor profile may be used in a
   document, about which the processor profile provides no formal guidance of
   use;

In systems that do not formally distinguish between these two distinct
concepts, it is generally assumed that a *loose* processor profile is
inferred from a formally defined content profile, and that the following
processing semantics hold:

   - if content violates a content constraint, then processing behavior is
   undefined; e.g., the processor may abort processing or may behave in an
   undefined manner (presumably in a manner that protects the integrity of the
   processor);
   - if content uses a feature unknown to processor, then the processor may
   ignore the feature if possible;

One problem with this latter approach (always inferring a loose processor
profile) is how optional F/Es are treated: the author has no way to specify
that a processor must support some optional F/E the author believes to be
essential to support in order to achieve the desired results.

In the realm of markup languages, it is often the case that all F/Es are
optional in content, and, as a consequence the loose inferred processor
profile is the empty set. That is the case, e.g., for HTML, SVG, and SMIL.
In HTML, there is no mechanism that would allow an author to require a
processor to support some optional F/E; however, in SVG and SMIL, there is
such a mechanism, which was originally defined in SMIL, and was adopted (in
some form) in both SVG and TTML.

That SMIL mechanism was originally defined as the "system-required"
attribute in SMIL 1.0 [1] in 1998. It was further elaborated and expanded
in SMIL 2.0 [2] in 2001. In SMIL 1.0, no specific F/E identifiers were
defined; however, in SMIL 2.0, a set of 51 identifiers [3] are defined that
designate F/E at the medium grained level of SMIL modules, including a few
designators for collections of modules and a few non-module specific
feature sets. These module designators are further delineated as specific
fine grained syntactic and semantic features in [4].

[1] http://www.w3.org/TR/1998/REC-smil-19980615/#test
[2]
http://www.w3.org/TR/2001/REC-smil20-20010807/smil20.html#smil-content-adef-systemRequired
[3]
http://www.w3.org/TR/2001/REC-smil20-20010807/smil20.html#smil-modules-smilModulesNSSMIL20ModuleIdentifiers
[4]
http://www.w3.org/TR/2001/REC-smil20-20010807/smil20.html#smil20-profile-q3

Shortly after the publishing of SMIL 2.0, SVG 1.0 was published wherein was
defined the *requiredFeatures* [5] and *requiredExtensions* [6] attributes,
and additionally defined a programmatic use with the
DOMImplementation.hasFeature() method [7].

[5] http://www.w3.org/TR/SVG10/struct.html#RequiredFeaturesAttribute
[6] http://www.w3.org/TR/SVG10/struct.html#RequiredExtensionsAttribute
[7] http://www.w3.org/TR/SVG10/svgdom.html#FeatureStrings

When we began work on TTML in early 2003, we reviewed SMIL usage and
eventually determined that in order to manage the complexity of feature
profiling, we needed something like SMIL/SVG's required{Feature,Extension}
mechanism to specify processor support requirements. We also had the
practical problem of defining which features we would test for the purpose
of satisfying the W3C testability requirements for REC track specifications.

Together, these considerations led to the current profile system defined in
TTML1 and further elaborated in TTML2.


>
> b) The interdependency of profiles with features as their lower level
> constituents may be to "expensive" to be implemented.
>

This is sheer speculation on the part of folks that have not actually
implemented it. I have implemented it, and that implementation was included
in one of the reference implementations used when moving TTML1 to REC, and
that implementation was simple and bordering on trivial.


>
> In the thread EBU-TT is mentioned several times and the questions have
> been raised why EBU-TT Part 1 (EBU Tech 3350) and EBU-TT-D (EBU Tech 3380)
> do not use the profile attribute and the profile mechanisms.
>
> The TTML profile mechanisms have been investigated and discussed quite
> intensively in the EBU group prior to the first publication in 2012 and
> after that. In the end the profile mechanism has not been adopted. Similar
> arguments to a) and b) have been raised. Both EBU-TT Part 1 and EBU-TT-D do
> not make a distinction between processor and content profile.
>

And in that sense, I believe EBU-TT is ambiguous. This same ambiguity has
crept into IMSC as well, e.g., see [8].

[8] http://www.w3.org/AudioVideo/TT/tracker/issues/450


> They define constraints for authors to use only a subset of the TTML
> vocabulary. But through that they define also of course the requirements to
> process this "subset" according to the specs.
>

I don't believe that is true. Tech 3350 and 3380 only discuss processing
requirements in a scattered manner, and never formalize a processing model.
Interestingly 3380 states:

   - "An EBU-TT-D processor is not required to support TTML 1.0 style
   attributes that are not used in the EBU-TT-D specification."

but nowhere states the converse:

   - An EBU-TT-D processor is required to support all features that can be
   used in an EBU-TT-D document.

Note that if it did state the latter, then that would constitute a strictly
inferred processor profile (in accordance with TTML2 terminology). BTW, I
don't consider Annex A of Tech 3380 as contributing to this discussion,
since it is merely informative.


> Encoder and decoder have the same reference.
>

You may think so, but as far as I can discern this is never fully
specified, and even then, a decoder may interpret/decode a feature without
actually supporting it (semantically), so the mere decoding process is too
low a bar.


>
> The profile registry may be a good ground to bring the different
> strategies together and make it usable for different encoder/decoder
> exchange scenarios. It may be as little to add that some of the “TTML
> derivates” define processor requirements through constraints of the TTML
> "content" structure.
>

Again, this last sentence is doubtful in my mind if you are referring to
EBU-TT.

The profile registry should not attempt to redefine or constrain the
terminology or language regarding profiles as defined in TTML{1,2}. In my
mind, it serves as nothing more than a way to define public abbreviations
of a specific set of URIs used to identify TTML processor profiles.


>
> Best regards,
>
> Andreas
>
>
>
>
> Am 08.10.2015 um 10:53 schrieb Nigel Megitt:
>
>
> From: Glenn Adams <glenn@skynav.com> Date: Wednesday, 7 October 2015 16:48
>
>
> On Wed, Oct 7, 2015 at 2:54 AM, Cyril Concolato <
> cyril.concolato@telecom-paristech.fr> wrote:
>
>> Le 06/10/2015 15:19, Nigel Megitt a écrit :
>>
>>> On 06/10/2015 12:39, "Cyril Concolato"
>>> <cyril.concolato@telecom-paristech.fr> wrote:
>>>
>>> Hi all,
>>>>
>>>> Consider the following scenario:
>>>> - someone has authored a TTML document
>>>> - someone else needs to generate the "codecs" attribute for this
>>>> document (either because the document is served by an HTTP server which
>>>> wants to compare the associated processor profile with what the HTTP
>>>> client accepts, or because it needs to add a codecs attribute to a DASH
>>>> MPD, or because the document is packaged in an MP4 and the MP4 is to be
>>>> fed to a MediaSource Buffer or described in a DASH MPD).
>>>>
>>>> The question is simple: how to find out which processor profile a given
>>>> document conforms to?
>>>>
>>> That's easy. The answer is always none - a document conforms to a content
>>> profile and a processor conforms to a processor profile. You might want
>>> to
>>> ask how to find out which processor profile is required to process a
>>> document that may optionally state a content profile though.
>>>
>> Having two dimensions of profiles really doesn't help. Audio streams,
>> video streams don't have that distinction.
>> As a player implementation, your really want to be able to tell rapidly:
>> "Can I play this file?". As a packager (like MP4Box) or as a server, you
>> want to be able to provide that information to a player. This may need some
>> processing of the file, but that processing should be limited (finding an
>> attribute value on the root element is ok).
>>
>
> The only utility in declaring a content profile is for
> validation/verficiation purposes. It allows the author to declare which
> content constraints were used when constructing the document.
>
> For the purpose of a decoding content, the processor profile is more
> relevant. This is why the current profile MIME parameter, and presumably, a
> new codecs parameter should focus only on processor profile.
>
>
>>
>> Are there restrictions in derived specs to always either
>>> use the [ttp:profile or ttp:processorProfiles] attributes?
>>> No there aren't such restrictions.
>>>
>> That is really annoying because it makes the job of server/packager
>> difficult, which means that they will probably put a catch-all profile (if
>> any), which won't be useful.
>>
>>> I understand that the notion of profile in TTML1 is equivalent to the
>>>> notion of 'processor profile' in TTML1;.
>>>>
>>> s/'processor profile' in TTML1/'processor profile' in TTML2
>>>
>> Yes, sorry for the typo.
>>
>>> The differences with TTML2 seems to be that:
>>>> - there is no default or inferred profile.
>>>>
>>> There is: from TTML1SE §5.2: "If neither ttp:profile attribute nor
>>> ttp:profile element is present in a Document Instance, and if the
>>> Document
>>> Interchange Context does not make an implicit or explicit reference to a
>>> pre-defined profile or does not specify a Profile Definition Document or
>>> another equivalent set of feature designations, then the DFXP
>>> Transformation profile applies."
>>>
>> Thank you for the pointer. I will have to check what the part about
>> "Document Interchange Context" means but this looks quite complex to me.
>> Would it be correct to find the profile identifier as follows: a) if a
>> ttp:profile attribute exists, find in the registry the short name
>> corresponding to the attribute value, b) if not, use 'tt1t' ? I fear this
>> wouldn't be correct.
>
>
> Document Interchange Context abstracts the delivery context, allowing, for
> example, the envelope or container to specify a profile without doing so in
> the document itself. Though I would always prefer the document to do so.
>
> As for finding what processor profile applies to a document, it is rather
> more complicated than simply looking for a ttp:profile attribute, even in
> TTML1. This is because it is possible to declare a profile within the
> document instead of by reference, i.e., by using the ttp:profile element.
>
>
>>
>> - the ttp:profile element does not have a designator attribute.
>>>>
>>> What would a designator attribute be for? Features have designators, not
>>> profiles. There is a use attribute though.
>>>
>> According to the registry, profiles have designators. According to 5.2 in
>> TTML1, profiles have designator. It seems very hard to determine if a
>> document can be played by a player by inspecting all features to see if
>> they match existing profiles.
>
>
> Not really. This has been implemented multiple times without difficulty.
> The process works as follows:
>
>    - determine the effective processor profile EPP that applies to
>    document D
>    - for each feature F in EPP
>       - if F is required, but not supported, then abort processing unless
>       abort is overridden
>    - for each extension E in EPP
>       - if E is required, but not supported, then abort processing unless
>       abort is overridden
>
> There is nothing difficult about this process.
>
>
>>
>> So for cases where the ttp:profile attribute is not used, how can the
>>>> profile identifier be determined? Are there recommendations in derived
>>>> TTML specifications (EBU, SMPTE, ...) to always set the ttp:profile
>>>> attribute?
>>>>
>>> In TTML1SE § 5.2, a Note includes: "it is permitted that the Document
>>> Interchange Context determines the applicable profile through private
>>> agreement, out-of-band protocol, or common use (between sender and
>>> receiver) of a profile defined by an external specification."
>>>
>>> In other words, the spec is deliberately silent on how the profile
>>> identifier can be determined if it is not within the document itself.
>>>
>> That's bad, as it makes it practically impossible for a player to know in
>> advance if it can play the content.
>
>
> The algorithm "determine the effect processor profile" always returns some
> profile, so that is the profile the processor should nominally use.
>
>
>>
>>
>>> I'm not aware of a specific recommendation to set the ttp:profile
>>> attribute. In EBU-TT it is actually prohibited - an alternate
>>> ebuttm:conformsToStandard element is permitted, cardinality 0..*, to
>>> indicate all of the standards to which the EBU-TT document claims
>>> conformance. For EBU-TT Part 1 v1.1 there is a recommendation to include
>>> the value "urn:ebu:tt:exchange:2015-09". The EBU did not consider that
>>> the
>>> ttp:profile attribute carries the same semantic.
>>>
>> The registry indicates that "urn:ebu:tt:exchange:2015-09" is a profile
>> designator. I would expect to find it in the ttp:profile attribute. Please
>> clarify the registry!
>> The fact that the use of the "ebuttm:conformsToStandard" attribute is
>> only a recommendation not a requirement is not good. There needs to be a
>> simple way to identify a EBU-TT document!
>
>
> I agree with this sentiment.
>
>
> TTML2 deprecates ttp:profile attribute in favour of ttp:processorProfiles,
> ttp:contentProfiles etc. The ttp:contentProfiles attribute closely matches
> the semantic of ebuttm:conformsToStandard, in my opinion. Nevertheless I
> don't think there's any proposal to make the presence of those attributes
> mandatory, and it would be unhelpful for some parts of the
> author->distribute chain to require them, so I would object to it. However
> it may be reasonable for an encoder/packager to require their presence on
> documents provided as input – that's out of scope of our specifications
> though.
>
>
>
>>
>> Additionally, the "profile" parameter defined in TTML1's MIME type
>>>> registration says:
>>>> "The document profile of a TTMLDocument Instance may be specified using
>>>> an optional|profile|parameter, which, if specified, the value of which
>>>> must adhere to the syntax and semantics of|ttp:profile|parameter defined
>>>> by Section*6.2.8 ttp:profile*
>>>> <http://www.w3.org/TR/ttml1/#parameter-attribute-profile>of the
>>>> published specification."
>>>>
>>>> IIUC, this means that for TTML1 the 'profile' MIME parameter is
>>>> equivalent (but with long URI values) to the 'codecs' we are discussing.
>>>>
>>> The use of short codes instead of long URI values is significant. The
>>> addition of extra syntax would also be significant. But if you omit the
>>> operators and consider the short code and the URI to be synonyms, then
>>> yes, I think there's a form of equivalence. What do you conclude or gain
>>> from that?
>>>
>> A better understanding of TTML1!
>>
>> My ultimate goal is to be able to produce, with GPAC, MP4 files and DASH
>> content that use TTML documents and that can be used meaningfully. This
>> currently looks desperately impossible. I'm wondering if I shouldn't stop
>> working on TTML in MP4 ...
>
>
> I agree that the lack of proper document internal declaration of processor
> profile is frustrating, and I feel the same. However, to conclude it is
> impossible to meaningfully use TTML seems an exaggeration. Let's look at
> the possible scenarios:
>
> Scenario #1 - Processor Supports Profile System
>
>    - EPP is always determined, but may not match authorial intentions in
>    case no document internal/external evidence declares profile;
>    - if EPP is defaulted, and default requires a feature/extension not
>    supported, then processing may be aborted if not overridden; this would
>    suggest the default be lenient (fully or relatively unconstrained);
>    otherwise, unsupported features will be ignored;
>
> Scenario #2 - Processor Does Not Support Profile System
>
>    - processing never aborts processing, but simply ignores unsupported
>    features
>
> There is nothing impossible about performing this processing. Obviously,
> it makes sense to *always* declare a profile within a document, but
> apparently some communities have not chosen to do this for whatever reason.
>
>
> I disagree: it does not make sense to always declare a profile within a
> document. This depends which part of the chain you are in and what other
> knowledge is available. For example a workflow may permit a document to be
> authored and then subsequently in a separate process to be transformed to
> compute and insert the profile details.
>
> In data rate constrained environments especially when there are a large
> number of documents, e.g. in a live streaming, it is likely to be
> preferable to define the profile in use across the end points and avoid the
> large amount of redundant data definition.
>
> It must be possible at least to permit this as an operational practice
> even if it is less generally useful. For the intersection of that practice
> and Cyril's use case I would expect there to be some kind of independent
> input like a command line flag, config file etc to tell the software what
> profile to use in the absence of other information – I'm veering away from
> the scope of the working group though here.
>
> Perhaps they believe that a processor will always choose either the fully
> unconstrained 'null' profile or automatically choose the profile they are
> using due to context of use (e.g., processors in region R assume a specific
> profile commonly used in region R).
>
>
> Yes, that (context of use) is a likely practical solution.
>
> Nigel
>
>
>
>>
>>
>> Cyril
>>
>>
>> --
>> Cyril Concolato
>> Multimedia Group / Telecom ParisTech
>> http://concolato.wp.mines-telecom.fr/
>> @cconcolato
>>
>>
>>
>
>
> --
> ------------------------------------------------
> Andreas Tai
> Production Systems Television IRT - Institut fuer Rundfunktechnik GmbH
> R&D Institute of ARD, ZDF, DRadio, ORF and SRG/SSR
> Floriansmuehlstrasse 60, D-80939 Munich, Germany
>
> Phone: +49 89 32399-389 | Fax: +49 89 32399-200
> http: www.irt.de | Email: tai@irt.de
> ------------------------------------------------
>
> registration court&  managing director:
> Munich Commercial, RegNo. B 5191
> Dr. Klaus Illgner-Fehns
> ------------------------------------------------
>
>