- From: Nigel Megitt <nigel.megitt@bbc.co.uk>
- Date: Thu, 10 Nov 2022 17:30:35 +0000
- To: TTWG <public-tt@w3.org>
- Message-ID: <16B5005D-4294-482F-A9D4-1E75AA5BEEBE@bbc.co.uk>
Thanks all for attending today’s TTWG teleconference. Minutes can be found in HTML format at https://www.w3.org/2022/11/10-tt-minutes.html
In text format:
[1]W3C
[1] https://www.w3.org/
Timed Text Working Group Teleconference
10 November 2022
[2]Previous meeting. [3]Agenda. [4]IRC log.
[2] https://www.w3.org/2022/10/27-tt-minutes.html
[3] https://github.com/w3c/ttwg/issues/232
[4] https://www.w3.org/2022/11/10-tt-irc
Attendees
Present
Atsushi, Cyril, Gary, Nigel, Pierre
Regrets
-
Chair
Gary, Nigel
Scribe
nigel
Contents
1. [5]This meeting
2. [6]Rechartering Formal Objection Council update
3. [7]IMSC-HRM
1. [8]Plan to reference IMSC-HRM from future revisions of
IMSC?
2. [9]CR1 exit criteria w3c/imsc-hrm#56
4. [10]DAPT
5. [11]Meeting close
Meeting minutes
This meeting
group: [discussion of zoom access, URLs, the ability to use W3C
zoom]
Atsushi: I will send the Chairs an email about W3C zoom
Nigel: Thank you
Nigel: Today we have DAPT, a couple of points on IMSC-HRM, and
if there's a status update on the
… charter FO Council, we can take that too.
… Is there any other business?
Pierre: On IMSC-HRM I would like to cover CR Exit criteria - I
created a Pull Request.
Nigel: OK, thank you
group: [no other business]
Rechartering Formal Objection Council update
Nigel: Anything to report?
Atsushi: I just know that the Council is set up and they're
working to coordinate on the time of the meeting
… so nothing further has happened.
Nigel: Thank you. Any questions?
IMSC-HRM
Plan to reference IMSC-HRM from future revisions of IMSC?
Nigel: This comes from a comment in w3c/imsc-hrm#51 where
Pierre said he thought we
<Github> [12]https://github.com/w3c/imsc-hrm/issues/51 : If
referencing from IMSC, references will be circular
[12] https://github.com/w3c/imsc-hrm/issues/51
Nigel: should not reference IMSC-HRM from IMSC.
… First question: are you saying we should remove the normative
requirement to meet IMSC-HRM from IMSC?
Pierre: Yes, that's correct. It would remove that from
conformance in future editions of IMSC.
Nigel: The argument is that it's not used?
Pierre: Two arguments. A) to avoid the circular reference.
… B) It won't have a significant difference because
organisations that want to enforce
… the HRM have called it out separately.
… I don't think it will cause an impact in practice.
… We can always have a note in IMSC that points to IMSC-HRM,
e.g.
<atsushi> +1 for informative note to inform, consumer can
choose to apply both or only imsc
Pierre: saying "this spec does not impose any constraints on
presentation complexity, here's where to find it".
Nigel: This raises the question of if IMSC-HRM needs to be a
Rec track document?
Pierre: To the extent that other orgs would use it for
conformance I think they would appreciate it being
… a Rec track document. To a large extent it depends on those
external organisations.
… I know some that would be reluctant to reference a WG Note
for instance.
Nigel: And the positive reason for removing from IMSC - is it
only to avoid the circular reference?
Cyril: Why is it a problem to have circular references? Other
specs do that a lot - almost every standard in ISO!
Nigel: We've always tried to be clear about the chain of
dependencies from spec to spec, and avoided
… circular references.
… That was my next question too, is this the only way to avoid
circular references?
Atsushi: I'm curious too - I believe this is something that
humans can check, for circularity,
… e.g. if A defines in terms of B and B is defined in terms of
A. That can even be in one specification.
… It cannot easily be detected automatically.
Nigel: That's right, there are ways to deal with this. Also,
can IMSC-HRM stand on its own, or can
… it be used only for IMSC.
Pierre: It's a constraint on IMSC, the dependency is pretty
obvious that IMSC-HRM depends on IMSC.
… For better or worse, for many implementations, renderers have
implemented IMSC but not the HRM validation.
Pierre: I think circular references are bad simply because
they're circular.
… I don't want this to be an obstacle. If there's a desire to
have a circular reference I'm not going to
… no-vote the document.
… Imagine: circular refs are bad because if IMSC gets updated
without updating IMSC-HRM, then you don't
… go back to where you started if you follow the reference.
Sometimes that's harmless, sometimes harmful.
… Being serious about conformance. which some orgs are, then we
would have a separate conformance
… document.
Nigel: That would be the IMSC spec though?
Pierre: Not always. I'm simply advocating having IMSC-HRM
reference IMSC, and have an undated
… reference from IMSC to IMSC-HRM.
… Maybe later we'll have a spec on readability complexity for
Western European languages, as a separate spec.
Nigel: I'd like two things to happen:
… 1. We arrange it so that the decision about how to reference
IMSC-HRM from IMSC is made separately,
… which means:
… 2. We word IMSC-HRM so that it can be referenced normatively
from IMSC, if that's what we some time want.
… An easy way to deal with this circular reference is to
duplicate the definition.
Pierre: That is an editorial nightmare - I'd object to that.
Nigel: Is it though? Is it worse?
Pierre: Yes, it causes work.
Nigel: My argument is that a change to one doc doesn't then
require a change to the other, because
… the definition has document-level scope only.
Pierre: I'd prefer circular references than defining the same
term twice in two different specifications.
… I've never seen that end well.
Nigel: I'm thinking that carefully arranged circular references
might be the best option here.
… My original issue was that there should be no circular
references _that cannot be resolved_, so if we
… can check that, it should be fine.
Pierre: I think the references are undated now, so I think it
will work.
Nigel: Let's say we make some tweak to a defined term in a
future version of IMSC, can we make
… it so that the reference to that term in IMSC-HRM is to the
one for the version of IMSC that applies?
Pierre: We should try to avoid doing that and if we do, we
should do it very carefully.
… One could argue that we should reference a particular version
of IMSC.
… Nowadays in practice people look at the latest version.
… The responsibility would be on us, if we make a substantive
change to a definition or something
… depended upon in the spec, then we better warn people, it
could be disruptive.
… Like changing glyph buffer Gn to glyph back buffer is really
editorial, but it we were to change
… Presented Region that would have a negative impact and we
might want to define a new term instead.
Github: [13]https://github.com/w3c/imsc-hrm/issues/51
[13] https://github.com/w3c/imsc-hrm/issues/51
Pierre: Not to be philosophical, let's not make a new version
of IMSC that's incompatible with an old one!
… I've seen that in other specs.
SUMMARY: Keep the references circular for the time being, and
check that they can all be resolved successfully.
Pierre: We can defer this until we revise IMSC to reference
IMSC-HRM.
Nigel: No, I want us to make sure we don't put something in
IMSC-HRM that gives us a problem later.
Pierre: I'm not aware of anything that gives us that problem
right now.
Nigel: That may well be the case.
SUMMARY: Nigel to verify that the current text will not cause
any issues.
github-bot, end topic
CR1 exit criteria w3c/imsc-hrm#56
<Github> [14]https://github.com/w3c/imsc-hrm/pull/56 : CR1 exit
criteria
[14] https://github.com/w3c/imsc-hrm/pull/56
github: [15]https://github.com/w3c/imsc-hrm/pull/56
[15] https://github.com/w3c/imsc-hrm/pull/56
Nigel: This is a draft PR, maybe that's why I did not notice it
ahead of the call.
Pierre: Walking through this.
Nigel: [shares [16]https://github.com/w3c/imsc-hrm/pull/56/
files?short_path=37df7cf#diff-37df7cf5e1fb2c2cfdd1d6eed9ffb1dda
a7070ffc160a1bf1aa717ecf60db1cc on screen]
[16] https://github.com/w3c/imsc-hrm/pull/56/files?short_path=37df7cf#diff-37df7cf5e1fb2c2cfdd1d6eed9ffb1ddaa7070ffc160a1bf1aa717ecf60db1cc
Pierre: I thought adding a MD doc was a good way to kick around
ideas.
… This lays out how we will demonstrate Implementation
Experience.
… The idea is that each IMSC document is accompanied by an
assertion that says if it passes or fails.
… The idea is that each document is tested against each
implementation and we determine that it
… meets the assertion.
… Features at risk: it occurred to me that the current
implementation of imsc-hrm does not support
… image profile, because nobody has asked for it. Unless we can
find actual image profile documents,
… from people who use it at scale, we should just remove that
feature from IMSC-HRM
… Then there's a bunch of synthetic tests to test the
boundaries and features of the HRM.
… If we are happy with this structure, then I'll add more
synthetic tests.
… That's my proposal for exit criteria.
… Atsushi asked if we want exit criteria to be different for
each CR version.
… In the past I have found that CR exit tests are not
necessarily useful for the community, long term.
… Right now the synthetic tests are just in a directory.
Cyril: Two questions.
… First, the collection of sample IMSC documents.
… I'm wondering, e.g. for Netflix, how we can contribute them,
for IP reasons.
Pierre: Thanks for the offer! I would expect Netflix to run the
implementation internally and just report
… on the results, I would be satisfied with that.
Cyril: Okay, that makes sense.
… Second, do you have a list of features that you want to test?
Pierre: Not yet. There are different ways the HRM can be
exceeded,
… maximum glyph buffer size, maximum rendering time.
… You'd want a test for each of those.
… There's also IPD, a maximum render time, to test.
… We should also test failure on the first ISD of a sequence,
because it's somewhat of a special case.
… We should test the ideograms as having a different rendering
rate from non-ideograms.
… We should have test documentations that fail if the glyph
buffer is not being used.
… I am planning to clean up the tests for the implementation
and make them available.
Cyril: I'm wondering if there's a list of those features?
Pierre: Yes, we could add a feature test column.
Cyril: In other standard bodies we're working on having
explicit references from the spec to the tests.
… Tools like bikeshed or respec help, they can generate assert
ids, which you add with special markup
… in the spec text. Then you can use those in the naming of the
tests.
… Respec has a way to show you the tests, if you
cross-reference them properly.
… It may be worth investigating.
Pierre: Absolutely, I'll look at that, thanks.
Nigel: Elephant in the room - this approach is going to be
dependent on the result of the FO Council.
… I think we cannot finalise this approach until we have the FO
Council outcome.
Atsushi: Usually they have a single call to arrive at their
conclusions, so we may hear something at the
… end of this month, at least. Of course it make take time to
generate the formal document to be published.
… In any case I believe we can get a decision this month if the
pattern follows previous FO Councils.
Pierre: By definition the deliberation does not stop our work.
Anyway end of month works for
… timing for requesting transition to CR.
Nigel: For the collections of documents, you want an assertion
for each document whether it passes or fails?
Pierre: For the synthetic ones, its our responsibility to
generate those.
… For third parties, e.g. Netflix, BBC, it's up to them to say
whether they think a document should pass
… or fail, and report whether the implementation agreed with
their expectation.
… If it agrees, there's nothing more to do.
… If it does not agree, then we have to determine what to do.
… It could be the content is invalid, an error in the spec, an
error in the implementation.
Atsushi: One concern with building a test suite for HRM. It
needs to have at least
… two generated documents from different implementations.
… Should there be some example of presentation?
… And IMSC-HRM implementation will go over that and generate a
pass/fail result.
… If I am correct I think we need to have some descriptive note
for our test case?
Pierre: My proposal is that we ask folks like BBC, like Netflix
and others to test their commercial content.
… That would be their criteria. Content that they use
commercially or plan to use commercially.
… We cannot specify any particular style, it's different for
every provider.
… That's what's most useful for the industry - it will tell us
if the HRM is useful, and right.
Atsushi: Usually, just seeing it built against each API call or
value definition is ideal. I'm not sure how
… to convince readers of the implementation report that it
covers all defined functionality in the HRM.
… Like which test is related to each definition, etc. It's
difficult to regenerate the cases.
Pierre: I agree it's different than other specs, e.g. API based
specs, for sure.
… We cannot control what people create commercially which
ultimately is what people care about.
… It does not fit the template of API specs.
Nigel: If we were to be able to show that the HRM is successful
at selecting between content that plays
… successfully on some player and content that does not, that
would be really useful.
Pierre: We can fairly easily architect a synthetic test that
presents alright but is likely to cause most players
… to have a problem, and is rejected by the HRM.
… Then we could invite folks with players to try it. We could
do that too.
Nigel: That's the big unknown to me, how well the HRM tracks
real world playability - that's really hard to grasp.
Pierre: It would be helpful actually to generate synthetic
documents.
… Any objection to the overall approach, before I start working
on it?
Cyril: No, this is fine, but maybe we should wait for the FO
Council?
… You might waste effort.
Pierre: I think we can address their conclusions later.
SUMMARY: No immediate objections, Pierre to proceed. TTWG to
continue reviewing.
github-bot, end topic
Pierre: Thanks for the feedback.
DAPT
Cyril: Suggest scheduling an editor's session
Nigel: +1 let's arrange offline
Cyril: OK
Meeting close
Nigel: We're over-time - thanks everyone. Next call in 2 weeks.
[rrsagent, make minutes]
Minutes manually created (not a transcript), formatted by
[17]scribe.perl version 196 (Thu Oct 27 17:06:44 2022 UTC).
[17] https://w3c.github.io/scribe2/scribedoc.html
Received on Thursday, 10 November 2022 17:30:57 UTC