- From: Nigel Megitt <nigel.megitt@bbc.co.uk>
- Date: Sat, 28 Sep 2024 00:56:36 +0000
- To: "public-tt@w3.org" <public-tt@w3.org>
- Message-ID: <2DE7963D-F21E-43DC-9603-95810D57E11C@bbc.co.uk>
Thanks all for attending today’s TTWG face to face meeting at TPAC. Minutes can be found in HTML format at https://www.w3.org/2024/09/27-tt-minutes.html
We made 3 resolutions:
* Resolution 1<https://www.w3.org/2024/09/27-tt-minutes.html#r01>: Proceed with a minor revision of IMSC as detailed in the minutes
* Resolution 2<https://www.w3.org/2024/09/27-tt-minutes.html#r02>: Liaise with ARIB about feature improvements in IMSC, specifically the backlog issues
* Resolution 3:<https://www.w3.org/2024/09/27-tt-minutes.html#r03> Merge the open Pull Requests and then go through the admin needed to get a WG decision to request transition to CR
The third resolution concerns DAPT specifically.
Those minutes in plain text:
[1]W3C
[1] https://www.w3.org/
Timed Text Working Group Teleconference
27 September 2024
[2]Previous meeting. [3]Agenda. [4]IRC log.
[2] https://www.w3.org/2024/09/12-tt-minutes.html
[3] https://github.com/w3c/ttwg/issues/291
[4] https://www.w3.org/2024/09/27-tt-irc
Attendees
Present
Atsushi, Bernd, Chris_Needham, Cyril, Dana,
Eric_Carlson, Gary, James_Craig, Jean-Yves_Avenard,
Jer_Noble, Nigel, Pierre, sideshowbarker,
Surya_Sunkavelli
Regrets
-
Chair
Gary, Nigel
Scribe
nigel, cyril, atsushi, gkatsev, cpn
Contents
1. [5]Introductions
2. [6]Agenda bash
3. [7]Netflix demo of TTAL <--> DAPT
4. [8]Streamable Text Tracks
5. [9]IMSC
6. [10]Next part of the day's meetings
7. [11]Afternoon session
1. [12]IMSC (continued)
2. [13]WebVTT Interop and issues
3. [14]New ATTRIBUTES block w3c/webvtt#523
4. [15]DAPT
5. [16]Break
8. [17]TTML2
1. [18]TTML2 CR snapshot outdated banner
2. [19]TTML Media Type Definition and Profile
3. [20]Joint meeting follow-ups
9. [21]Conclusion
10. [22]Summary of resolutions
Meeting minutes
Introductions
Nigel: Nigel Megitt, BBC, co-chair TTWG
atsushi: Atsushi Shimono, W3C Japan, team contact TTWG
jcraig: James Craig, Apple, accessibility groups like ARIA,
crossing over to groups like this when there's an overlap of
interest
cyril: Cyril Concolato, Netflix
gkatsev: Gary, Invited Expert, NBC Universal/Peacock, co-chair
TTWG and WebVTT editor
pal: Pierre-Anthony Lemieux, supported by MovieLabs, editor
IMSC
surya: Surya Sunkavelli, working on ...
Surya: Hey everyone, my name is Surya Sunkavelli. I work on the
Media Foundations team within Encoding at Netflix so my work
usually features studio facing use cases and media inspection
related projects. With that being said, lately I’ve been
working with Cyril for about a couple months now in tracking
the DAPT spec as it develops and gets
updated and mapping it back to TTAL, which is a Netflix
internal script file exchange format used to interchange
between third party dubbing tools and Netflix internal tools
like Scripting tool.
Agenda bash
nigel: Iterates through topics
[23]Meeting topics
[23] https://www.w3.org/wiki/TimedText/tpac2024#Topics
<jernoble> +present Jer Noble
group discussion of agenda topics and timings
[24]Results of agenda bash
[24] https://github.com/w3c/ttwg/issues/291#issuecomment-2379654028
Netflix demo of TTAL <--> DAPT
surya: TTAL is an internal Netflix specification for script
file exchange
… I wanted to demo the status of the converter between TTAL and
DAPT
… and the key DAPT features that are being mapped properly
cyril: Netflix blog post explaining TTAL
jcraig: This is not a programming script, but a script for a
piece of media?
cyril: Yes, think of it like a transcript
[25]https://netflixtechblog.com/
introducing-netflix-timed-text-authoring-lineage-6fb57b72ad41
[25] https://netflixtechblog.com/introducing-netflix-timed-text-authoring-lineage-6fb57b72ad41
cyril: this explains the history behind TTAL
… It is already used in production, we have partners all around
the world
… that deliver scripts, dubbing scripts mostly but also audio
description scripts
… using this format, but Netflix wants to migrate to DAPT when
it is finalised.
surya: [shares screen]
… List of example DAPT files generated by the converter.
… [shows example file]
… xml:lang showing language of file
… daptm:langSrc to show origin language
… daptm:represents - I know this is going to change, but looks
like this now
cyril: Mentioned previously - Netflix commissioned production
of dubbing scripts for some
… of its open content, where we share content that can be used
to test features like HDR etc.
… One of the titles we have is Meridian and we have asked for
the generation of dubbing scripts
… in ~8 languages. We intend to release that as open source too
so
… people will be able to see the TTAL and DAPT scripts for this
title.
surya: Yes, I can see around 8 languages.
… Here's an example of one of the Meridian ones.
… This one is an audio description script.
… It's mapped to visualNonText and visualText
… With frame rates, we capture that using ttp:frameRate and
ttp:frameRateMultiplier.
Nigel: are you using frame based time expressions?
surya: In TTAL we use frame based time expressions, so we do
the same in DAPT
cyril: This is the easiest conversion. We may decide to have a
different mode where
… we go to seconds and forget the frame rate.
surya: Features prefixed with nttm: is for Netflix internal
metadata from TTAL.
… We're just storing them here but we're still talking about
how best to organise internal specific metadata.
cyril: There are so many ways to capture metadata, attributes,
elements, metadata elements,
… either one or multiple.
… Would like to have guidance in DAPT about how people should
do metadata.
surya: Events and Characters
cyril: In this example here, there is a `ttm:desc` child of a
`ttm:agent`
… Not sure if that's allowed?
nigel: ttm:agent does not accept ttm:desc children
surya: That's something to keep in mind for agents, where to
store the metadata
nigel: One option is to change it in TTML2
surya: Specific elements, div captures ids, begin and end
times, style ids
… so the mapping from the events to the style ids.
… ttm:agent reference, which we map back to ttm:agent elements
… One issue we haven't finalised yet, is that TTAL and DAPT
think differently about how to
… map script events or timing events. We haven't ironed out the
mapping yet, some issues with TTML.
… We would like segments to map to span elements with time
offsets and styles for dubbing.
… Have to finalise that still.
… Earlier mentioned daptm:represents, so with the introduction
of that we will change
… how we use daptm:eventType
Nigel: Reminding myself, we got rid of event type didn't we
cyril: Yes, it's now represents
cyril: To summarise, no problem mapping the core types of TTAL
into DAPT
… The question is about mapping additional metadata that is not
in the core DAPT vocabulary
… How best to use the registries or user defined values for
some of the attributes
… That's what we have to work on now
Nigel: Will you share those more widely to see if there's
interest in adding those to the registries?
Cyril: Yes, some of them
Nigel: That's great, a working implementation of a DAPT
generating tool
<eric_carlson> +present eric_carlson
surya: Yes, and there's a tool to go back the other way that
we'll complete
cyril: This tool will be open sourced
nigel: Thank you for that demo
Streamable Text Tracks
Nigel: In Media WG yesterday I raised a follow-up to a
conversation in Media and Entertainment
… Interest Group from Monday,
… which was about adding to the Media Source Extensions text
tracks; it's currently only audio and video.
… There was a positive discussion, with consideration of some
of the thorny technical issues
… that would need to be resolved.
Cyril: Would like to go through the issues and understand which
groups will deal with them.
jer: Two major areas are streaming of raw formats, WebVTT in
hls.js, and feature level compat
… between MSE and HLS, and how to append WebVTT files
… The MSE API allows you to append in a blob of binary data
with a specific MIME type
… The concept would be you could init a source buffer with a
MIME type of webvtt and append chunks
… of WebVTT files.
… The Media WG would need to come up with a description of
WebVTT in our media file format
… registry that describes initialisation and media segments for
WebVTT and any other formats we would
… want to support including TTML.
cyril: Are the registries specific to a media type or a MIME
type?
… The registry deals with ISOBMFF, not audio in ISOBMFF, video
in ISOBMFF etc.
eric_carlson: I think that's right
jer: Yes we're jumping ahead to the second use case,
… which is delivering captions inside a media container, not
just as raw payload
… Not just embedded VTT or TTML, but also 608 and 708 captions,
which are more commonly found
… on the web at the moment.
… Either exposing the raw samples to the client application, so
an MSE app in a browser would get
… a callback maybe or an event fired when new samples or
DataCues (not standardised yet),
… some packet that describes a thing that happens in the
timeline,
… or alternatively, my preference, when MSE encounters timed
text, turn it directly into captions
… and turn it into Text Track Cues that are displayable
natively by the user agent.
eric_carlson: I agree that's better, but there's always a
situation where the UA encounters a format
… it doesn't know how to parse, so we need an escape hatch
where the page can override the behaviour,
… or can handle a format that the UA just doesn't know how to
parse.
cyril: I fully support that, with a Netflix hat
… We care about how we display our subtitles and the
consistency of our subtitles across UAs and devices.
… I agree sometimes the page wants to let the UA do it,
… and we have to work with regulation, privacy etc.
jer: We're bleeding into the interop issue for later about not
being able to rely on the UA's behaviour
cyril: You want the page to be able to render the TTML
eric_carlson: If the page just wants to override the UA
behaviour, a page could do what
… many do now, taking the parsed output and do the rendering
themselves.
… We have to handle the case where the UA doesn't know how to
parse.
Nigel: Is there already an equivalent for audio and video
formats that the UA doesn't know how to decode?
eric_carlson: No
Cyril: The UA processing and performance requirements are a
different order of magnitude
… Maybe why there wouldn't be that approach
… Maybe it could be WebCodecs in the future
eric_carlson: Thinking about it, the problem I was thinking of
doesn't make sense, because
… if the UA doesn't know how to parse the content then it just
doesn't know, and its the web app
… that's going to push the data into the parser... No, I take
that back. If it's muxed content...
cyril: If it's TTML in MP4, the application will just push it,
then the TTML gets extracted and
… passed back to the application at the appropriate time
eric_carlson: Yes, that's what we need to figure out
… There isn't an issue where a raw format would be pushed in
and we need to deal with
… exposing that to the page somehow.
jer: A lot of the issues have to do with the MSE specification
itself.
… One common way to do ad insertion is to reuse the same source
buffer and append media from
… a different source, maybe with a call to changeType() to
signal the change of codecs.
… But the MSE API has a requirement that there are always the
same number and type of tracks,
… with unchanging identifiers, even if the payload (codec) type
changes.
… So if you have a main presentation with two 608 tracks, if
you wanted to switch midstream to an
… ad then that would have to have exactly the same number of
text tracks too, according to MSE.
… I don't know if the reason for that is still relevant to text
tracks
… There may be a need to relax that requirement only for text
tracks so that you can continue
… to play without breaking.
Nigel: I can see UI issues about changes to the user's
preferred selection
Jer: Exactly, and how do you present that change to the text
track
eric_carlson: We have that in HLS already, it can switch
rendition to one that has a different set of tracks
… Nobody has ever complained about it, it could technically be
a problem
jer: Good to know how many people choose a specific language
preference
gkatsev: That's also for source buffers - for audio and video
you're not supposed to have gaps,
… and that's more likely for text tracks, where there may not
be any for a long time.
… How does this interact with the Safari experiment with the
HTML block for rendering?
… Would this supersede that, or is it independent?
eric_carlson: In my mind they're separate things.
… I think this is just an extension of what we already do with
inband caption tracks
… where we turn them into WebVTT cues as long as it's a format
that we understand.
… We just use the existing text track mechanism.
eric_carlson: The other problem we need to solve is how to
signal 608 and 708
… Technically the init segment has to describe the stream.
cyril: That is probably an MPEG question
… A solution is getting published in a couple of months.
… There will be an amendment to ISOBMFF part 12 that enables
signalling T35 messages in the init,
… and you can put as many bytes as you need.
… I got confused though because 608 is SEI messages not T35
messages.
jer: HTML has a side spec called Sourcing Inband Tracks
… It looks like, to signal 608 or 708 you need to have a trac
fragment in your moov header.
<jernoble> [26]https://dev.w3.org/html5/
html-sourcing-inband-tracks/#mpeg4
[26] https://dev.w3.org/html5/html-sourcing-inband-tracks/#mpeg4
cyril: Can we have a forum for this conversation to continue?
… Is it better in TTWG or MediaWG or somewhere else?
eric_carlson: I think it would be better to start it here and
then take it to the MediaWG.
… Just my quick opinion.
Nigel: My 15s of thought says the opposite.
eric_carlson: I think it helps to make progress in a smaller
group and then take that to a bigger group.
Nigel: I would say the spec is squarely in MediaWG so I'd
propose setting up a task force in MediaWG to
… look at this, and then take the results back up to the wider
group.
… Then if there are interested people from here they can join
that TF.
Cyril: Need to get the terminology right, useful to do it in a
task force
Nigel: someone needs an action to take this task force proposal
to the chair of the MediaWG
jernoble: I will take that action
IMSC
Nigel: May need to let this topic flow over into a later
session if we don't have time now
Pierre: Background: a couple of weeks ago an issue was reported
by a user of IMSC, a service
… provider to a large content platform, that their French
subtitling and captioning team had run
… into an issue where they couldn't get superscript and
subscript characters.
… They turn out to be more important in French than in English
because France has a
… governmental organisation that sets standards for the
language.
… Those standards say that the abbreviation of first and
second, like 1er, must be in superscript.
… Everyone in France understands it if they are not
superscript, but French standards are to put
… the ordinal numeral as superscript.
… So this is a shortcoming of IMSC, evidently.
… The good news is there's already a TTML2 feature that allows
super or sub to be specified.
… Because we have this significant issue and other backlog
issues it seems like a good time to
… do a minor revision to IMSC
cyril: Is there a list?
pal: Yes, I'll go through it in a few seconds.
… The intent is a minor revision, and fix this substantial
issue.
… I've gone through, and sent an email to the reflector
yesterday, and taken a stab
… at labelling the backlog issues that we should try to tackle
if we are going to embark on this.
… I labelled this as 1.3 because we're at 1.2 now.
… There's one big category of backlog issues which are related
to the results of a liaison with ARIB,
… a Japanese organisation that have defined a profile of TTML
for the Japanese broadcast market.
… Where we left off is we alerted ARIB about 1.1 or 1.2 and got
suggestions back that require more
… detailed discussion to make sure we would not do something
that would not work for them.
… We never got that collaboration going.
… Something we should probably do would be to send a
communication to ARIB saying we are about
… to start on IMSC1.3, now is the time to have this discussion.
… Otherwise those issues will stay in the backlog.
Nigel: Atsushi, do you have any contacts there?
atsushi: There is NHK, but my main contact in this area has
moved company, so I need to.
… establish new contacts. I'm not sure if we should write a
liaison or make a direct contact.
Nigel: We can do both
atsushi: Both would be fine. There stance is not to be too
responsive generally.
pal: That's fine, I just don't think we should do anything
without active collaboration with ARIB.
atsushi: I understand, yes
… Let me contact during lunch, if I can, today, some colleagues
and give me the chance to feed back
… during the afternoon session.
pal: If we go ahead we should craft a formal liaison.
atsushi: Yes, of course.
pal: In terms of backlog...
[27]IMSC 1.3 issues
[27] https://github.com/w3c/imsc/issues?q=is:issue+is:open+label:imsc1.3
pal: I suggest running through them, and taking notes if there
are any.
… [go throughs the list]
… Super/sub support
… Factor out HRM
… Namespace name docs are out of date - not a spec issue, a
repo issue
… Editorial bug about the wrong feature being referenced with
time offset
… Improve the introduction
… Improve para wording for WCAG SC 1.1.1
… was controversial and text is convoluted, time to clean up.
Pierre: IMSCvnext is prepared for future version, and the same
as backlog
… also should be fine to be removed
… one live feedback on character sets of Japanese
… some suggestion from APA on description on semantics
… we should at least ping folks
… make sure we will pick in next slot
… following three items, 493, 492, 490, are not blocker, or
even we can close
… 489 is errata
… 484 is super long discussion thread
… forcedDisplay and visibility hidden, neither anything we can
do or need to do , but we should again back on this discussion
… nice to have, but no particular point to have
… #82, will not touch image profile, unless it is broken
nigel: it exists and broken already
pierre: unless somebody have absolute need...
nigel: serious question, any issue actually impact to image
profile?
pierre: three issues pointed above - 493, 492, 490
nigel: ok to keep open unless touching these three, or
something need to do?
pierre: fine to keep open
nigel: we only update text profile in 1.3, without image
profile
… people could use image profile from previous ones like 1.2 or
even 1.0, it this correct
pierre: not sure, only objection is working or not, would not
touch anything more on this unless something happens
nigel: ok
pierre: agree on concept, personally, anyway
nigel: all sound good to me
pierre: my three requests, one to decide what to proceed,
create a liaison with ARIB, discussion with APA on 503
<pal> [28]w3c/imsc#524 (comment)
[28] https://github.com/w3c/imsc/issues/524#issuecomment-598299006
[29]Comment regarding APA feedback to resolve
[29] https://github.com/w3c/imsc/issues/524#issuecomment-598299006
Next part of the day's meetings
[30]Joint meeting with APA and MEIG
[30] https://www.w3.org/events/meetings/5d9bf691-6f90-45b9-9c2a-b89faaedb191/
Nigel: [temporary adjournement]
Afternoon session
Nigel: Welcome everyone to the afternoon session.
IMSC (continued)
Pierre: Proposal is to proceed with a minor revision of IMSC
with the primary objective of addressing the lack of support
for superscript and subscript
… and secondarily resolving outstanding editorial and blocking
issues in the backlog.
PROPOSAL: Proceed with a minor revision of IMSC as detailed in
the minutes
Nigel: Any objections?
Pierre: Then I think the other one, assuming we go ahead, is to
get a liaison going to ARIB,
… a last shot at getting this done.
PROPOSAL: Liaise with ARIB about feature improvements in IMSC,
specifically the backlog issues
Nigel: Any objections to either of the above?
RESOLUTION: Proceed with a minor revision of IMSC as detailed
in the minutes
RESOLUTION: Liaise with ARIB about feature improvements in
IMSC, specifically the backlog issues
Nigel: We should think about whether industry needs us to keep
going with multiple
… active versions of IMSC or if we can deprecate any of them.
… Similarly, should we mark it as "allows changes" rather than
continuing with the version numbering approach.
Pierre: Good question. If we allow changes, what's the approach
to removing unused features.
atsushi: Updateable Recs can accept candidate amendments or
candidate new features.
… At any time we can change these candidate changes.
… Once the candidate changes pass the same procedure as normal
Rec, including
… HR, WR, AC Review, everything will be merged into the Rec as
normative parts of the specification.
… In theory everything that is similar to CR even if the spec
is published with a Rec URL.
… Anything can be done.
Nigel: So any candidate change can be made, even deprecation or
removal of features?
atsushi: Yes
Pierre: Even those changed versions of the Rec are dated, is
that right?
atsushi: Yes the publication date changes, even with candidate
amendments
Pierre: Hard to see any reason not to do it
Nigel: @sideshowbarker and I have been discussing this, there
are significant markup requirements
… for the candidate changes, but there is tooling on the way to
help with that.
atsushi: I never want to have to do this for TTML!
Nigel: We can decide about the allows-changes status before we
go to CR.
… Any other points on IMSC for us to think about?
no other points
WebVTT Interop and issues
gkatsev: Thanks for coming, I'd love to hear about the WebVTT
interop stuff
dana: [Presents slides] Hi, I'm Dana, first TPAC, I work on the
WebKit team at Apple
… For some time I've been researching WebVTT and learning about
the problems with it,
… why it's not widely used.
… I'm hear to present what I've learned through my research,
… and some ideas for making it more attractive for websites to
use.
… [slide: Recap of previous years' efforts]
… Will talk about where we are in the discussion, usage of
WebVTT in the wild,
… Web Platform Test interop test results and what they mean,
… bugs and missing features in WebKit and the WPT
infrastructure,
… and then ideas for fixing things.
… [slide: Recap of previous years' efforts]
… In previous years we discussed adding a generic text track
cue class.
… But important to make sure we're building on something that
works, i.e. WebVTT.
… Clear to everyone that WebVTT is not in the state it needs to
be given it's not widely used.
… WebVTT rendering is rarely used on high-traffic websites
… Some sites use WebVTT format, but render cues themselves,
e.g. Hulu.
… Suggests rendering is the reason it's unattractive.
… Problem that sites render it themselves because they don't
have access to the
… accessibility features that are only possible through using
WebVTT.
… Without using WebVTT, cannot get subtitles in PiP or full
screen on the iPhone.
… Users lose out on acccessibility and other features when
sites use don't use WebVTT.
… Anecdata: Interop is a problem.
… WebVTT appears differently on different browsers, so it's not
reliable.
… [slide] WPT results
… Websites tell us they don't like interop of WebVTT between
browsers.
… Backed up by results of the test.
… Rendering scores are critically low.
… No browser passes >30% of rendering tests.
… Chrome, Edge and Safari pass <6% of the rendering tests.
… For the past month I've been going through WebKit's rendering
test failures,
… and categorising the bugs.
… Most are implementation issues,
… 6 are likely bugs in the tests or the test infrastructure.
… I'm also new to WebVTT so I could be wrong.
… This is approximately how it is.
… I'm planning on filing these issues.
… We already have them in the WebKit bug database for our own
tracking purposes.
… Also planning on posting the 6 test bugs into the WPT repo.
… [slide] Stand-out bug is accessibility preferences
… Someone already opened [31]web-platform-tests/wpt#46453
… Problem is on Mac and iPhone user can set accessibility
preferences for captions,
… but the WPT tests don't take those into account, so we fail
almost all the tests.
… The default style we apply via our system settings and the
test infrastructure doesn't know about that.
… Similar issue in Chrome I beleive.
… Couple of solutions:
… 1. Turn off accessibility styling before running tests, in
the test env.
… We don't prefer this because we also want to test the
interaction between user styling and rendering.
… 2. Add the concept of a cue window to the WebVTT spec.
… There's discussion about this under the issue.
… Exposing this concept of a cue window would allow the
websites to override the system
… styling of the cue window, which we can then incorporate into
the tests.
… We prefer the second solution, to ensure the accessibility.
… [slide] I also found other issues with the tests.
… We fail ~80 of them for non-obvious reasons, when it looks to
me as though the two images
… are identical or almost identical, maybe differing by
blurriness, and I'd like to get some help
… to look into why this is the case.
… More obvious bugs: inconsistencies between the expected and
the actual, like border around the video.
… Those are about 6.
… The other 35 are webkit bugs we are tracking and dedicating
time to fixin.
… [slide] We think the solution to WebVTT interop is first
filing and testing the bugs in the
… test infrastructure. We also support proposing a WebVTT focus
area at Interop 2025,
… deadline is Oct 9 2024.
… Would like feedback and suggestions on what a good proposal
would look like.
… Even if we are not successful at getting WebVTT into Interop
2025, we are still committed to fixing
… out bugs to match WebKit's WebVTT implementation to the spec.
… We think it's important to the health of the web that pages
can rely on native subtitle presentation.
… This would allow for more accessibility features to be
accessible by users.
… That's it, thank you!
[31] https://github.com/web-platform-tests/wpt/issues/46453
gkatsev: Thank you very much, that was awesome!
dana: If it's interesting I have the spreadsheet of bugs and
what it impacts.
Pierre: Going back to your first slides, these are great
numbers.
… Have you also collected numbers on IMSC in the process?
dana: No, just WebVTT
pal: Any sense of what the numbers might be for IMSC and TTML?
dana: No I haven't.
pal: I think many of the same results exist for IMSC and TTML
where rendering is done
… by page code, and the same accessibility challenges exist for
TTML
jcraig: Are there WPT tests for TTML?
gkatsev: No, because it's not natively supported
jcraig: That would be the first step before doing this
pal: The same exact issues that you've noted exist for TTML and
there are some platforms
… that only use TTML and are forced to use a polyfill or burn
in after rendering server-side.
dana: It would be preferable to use WebVTT rather than IMSC to
make the accessibility features work
pal: Many of the same considerations apply to both
gkatsev: Making this a baseline for the future, for HTML
fragment cues are both methods of getting
… IMSC support natively, so having really good WebVTT interop
will allow those to work better also.
dana: It could sound a little far-fetched imagining a future
where WebVTT is the most used format.
… We think this is the first step, to improve interop
… Makes that future more of a possibility than it is now.
nigel: I think it's a really good idea to do what you're
suggesting here to look at higher problem spaces
… we need to figure out how to make subtitles accessible and
see how things go from there
… webvtt interop is a good baseline but there's missing
features
… there's things used in practice in ttml that can't be done in
webvtt
… there's also format issues like server-side transforms
… isn't necessarily a good fit for a lot of reasons, speaking
for the BBC
… I've definitely seen problems for mapping from ttml into
webvtt
<Zakim> jcraig, you wanted to mention FCC requirements for
captions and to mention WPT, okay to write WPT tests for any
spec, even if not implemented; allows test-driven development
and to mention overriding the user styles in the LayoutTest or
ideally WPT test
nigel: my vision of an accessible way would be something that
make it more agnostic from webvtt
jcraig: Thank you for the presentation, a few points back.
jcraig: It's ok to write WPT tests for things that have a spec
… for example include .tentative in the test name
… It's common to do that for test driven development on the web
platform.
<Zakim> jcraig, you wanted to react to nigel
jcraig: We already have a CI environment that can run those.
… Additionally, one of the reasons that this test is important
is that there are FCC
… requirements, and have been for almost a decade, about
caption styles,
… that we support on our platforms, and even games platforms
support,
… but I do agree that we should not bypass the accessibility
settings.
… There may be some intermediary settings, like Dean Jackson
had a way to make OS level settings
… before the test run, and write different tests for different
OS level settings.
… Then you can test the rendering part and the customisation
settings.
eric_carlson: I think that's an excellent idea.
… In WebKit's own internal tests we ignore the system styling
completely.
… It's a way to have consistent tests.
… Your idea makes a lot more sense.
jcraig: Dean's tests were something like reduced motion.
eric_carlson: I have lots of ideas about how to do it.
nigel: TTML has large set of test suite, which is not on wpt
<Zakim> nigel, you wanted to react to nigel to mention IMSC and
TTML test suites
<jcraig> jcraig: other tests have been ported to WPT...
pal: On this issue of WPT tests, how do you create tests that
there is no way to expose on a web platform?
… Are you deliberately showing that they're all failing? Is
that useful?
jcraig: TTML you want to render in the browser, right?
… If not then no point in adding to WPT.
… Not sure how familiar you are, it's a massive CI platform
that tests every feature on every web platform.
… It's become the expectation of the place to write the tests
if they're platform independent.
pal: I'm a big fan, just not sure if it's useful to write TTML
tests because they would fail by design.
jcraig: I wouldn't write a lot of them, just write a base level
of coverage.
pal: Not trying to convince anyone, just pointing out that we
have both in the ecosystem.
… Until and unless one gets 100% of the market we are going to
have these issues that were
… highlighted at the beginning.
… Two options are:
… 1. Try to get to agreement to use only one. not worked over
12 years.
… 2. Support both, just like there are lots of image codecs
etc.
… Until we resolve this issue (note zero objection from me for
fixing webvtt),
… we are going to keep having the same issue that you found at
the beginning.
… We need to solve the fundamental issue before we get a great
developer and user experience.
… Maybe now is not the right time.
[32]TTML2 test suite (large set of TTML XML files)
[32] https://github.com/w3c/ttml2-tests
jcraig: Sounds like no objection to more VTT testing or to
proposing interop area for Interop 2025,
… or for fixing the main issues of WebVTT even if it doesn't
solve the wider long term timed text issues.
dana: It can't hurt
gkatsev: I think Interop 2025 would be awesome because it would
feed back into the spec,
… and allow us to get to Rec.
… ON the TTML / IMSC side I think it might be a bit early for
WPT tests, but with the work
… on streaming text tracks and html fragment, when that's
further along, we could write WPT tests
… for them, especially if part of the use case for that is
native IMSC support.
jcraig: Dana, from my experience, I wrote several hundred tests
last year, you're probably right
… that there are bugs.
gkatsev: Definitely bugs, I've seen maybe 10% of tests are
invalid.
jcraig: First implementer finds the bugs!
dana: That would be a goal before interop, that we have a
functioning test suite.
nigel: one of the point in testing difficult, approach is by
design by design
… use of preference setting is pain
… last year we discussed and got some responce on this, web
principle document from TAG
… observation made by me, use of caption is so wide, these days
use cases are widely spread
… not like 20 years ago so can no longer claim that use of
captions implies disability or vulnerability
… broder and too many use cases and users exist now, incl.
disabilities
<jcraig> The TAG's Design Principles issue Nigel mentioned:
[33]https://w3ctag.github.io/
design-principles/#do-not-expose-use-of-assistive-tech
[33] https://w3ctag.github.io/design-principles/#do-not-expose-use-of-assistive-tech
nigel: we need to revisit to these for principle design
… other side, totally different display than native captions
… we would not get usage data
… will not get source of data for improvement of us
… you're harming users allows to gather statistics of use, it
helps
… you can only observe on or off for WebVTT
… not user customisations
???: if in specific site, data could be collected
nigel: talking about is that player page and what is key focus
Jean-Yves Avenard2: definitely agree that captions are widely
used now
… problem with exposing user preference exists
… anybody customize styles are uniquely identifiable
… inject large fingerprinting vector
… absoletely no way to expose
<gkatsev> s/???2/eric_carlson/
nigel: if the world changes and allows access to these
preferences, it cause another side
cyril: try to give TTML and WebVTT information
dana: Netflix produces both styles of captions, the vast
majority that is consumed is TTML.
… A small amount WebVTT and a small amount image.
… There's no way for Netflix to migrate to WebVTT short or long
term that I can see.
… I'm curious, one of the problems we have with WebVTT, which
we don't use on the web,
… even on Safari, is the discrepancy between the Safari support
vs iOS.
… If you use Safari you don't get the same results with Core
Media native playback.
… Do you have a sense - WPT doesn't test native support, right
- do you do internal testing?
jy: They do their testing, we do ours
eric_carlson: That's something we need to address
… They don't support CSS etc.
cyril: The way we use WebVTT in Netflix is lots of p-list
hooks.
… Don't even know how to hook those into CSS
eric_carlson: They do now support CSS in WebVTT files, but they
don't have a full-blown CSS engine.
… We can certainly take files that they support and make sure
that the rendering is the same in
… WebKit and in Core Media.
cyril: As a content provider, the choice of WebVTT is not
obvious because the differences mean we would
… have to provide 2 versions.
eric_carlson: I agree that is a problem for us to fix
jy: Does it matter that there are small differences as long as
the content is there?
Cyril: We don't care about pixel accuracy, but for Japanese you
care about Rubys, vertical text being
… rendered correctly.
… Obviously we have positioning requirements to make sure
captions don't clash with content in the image.
… If the position is off it's a problem.
gkatsev: It's about feature support
pal: I think this is actually the core issue, anecdotal
evidence I've heard is when a platform goes ahead
… and tries to use native, and there is native support for TTML
in a bunch of players, they realise
… that no two players are the same.
… Some diffs subtle, then they decide to render themselves.
… Trying to get to interop is a really good idea.
… Otherwise folks will continue doing it themselves.
gkatsev: A couple of years ago a guy (Dan?) at Demuxed talked
about how and why they decided
… to render everything themselves. Needing to be FCC compliant
made them have to render
… themselves, particularly on set top boxes.
… Interop2025 would be awesome.
… My question: do you need anything from us for that?
… We'd be happy to help, as much as I can, for that.
dana: No I think...
eric_carlson: Signals of support, making it clear, I don't know
how, that if you think it's a good idea
… that you think it's a good idea with the WPT committee.
… Coming from the WG would be extremely helpful.
gkatsev: Is there an existing mechanism for that?
eric_carlson: I don't know but we'll figure it out and let you
know.
jcraig: Typically there are proposals for new focus areas.
… If the WG agrees, one of the Chairs could just comment in
that issue.
… e.g. to say Timed Text supports this.
dana: When you submit proposals to interop they're on GitHub
and you can add discussion comments.
… Once the proposal exists I will share it.
gkatsev: Can you post it to the timed text mailing list?
dana: ok
<jcraig> Current Interop Proposals [34]https://github.com/
web-platform-tests/interop/issues
[34] https://github.com/web-platform-tests/interop/issues
<jcraig> [35]https://github.com/web-platform-tests/interop/
issues?q=is%3Aissue+is%3Aopen+label%3Afocus-area-proposal
[35] https://github.com/web-platform-tests/interop/issues?q=is:issue+is:open+label:focus-area-proposal
gkatsev: I think we're out of time for this topic, technically
out of time for our scheduled session.
gkatsev: On the ATTRIBUTES PR, Nigel and I had the question
about the lang and label and kind
… interact with the track element.
… That would be really good to disambiguate.
[36]new ATTRIBUTES block
[36] https://github.com/w3c/webvtt/pull/523
New ATTRIBUTES block [37]w3c/webvtt#523
[37] https://github.com/w3c/webvtt/issues/523
github: [38]w3c/webvtt#523
[38] https://github.com/w3c/webvtt/pull/523
jcraig: Need to assess, if there's a mismatch, which one wins.
… I don't have a strong opinion.
… Main reason we want it in the VTT is sometimes there's no
HTML as an intermediary,
… so the track information is missing.
gkatsev: That makes sense, my question would be would this also
allow a change in HTML
… if we're adding precedence to those attributes.
jcraig: I would expect that whatever we land on, we could have
the same thing reflected on the track element
eric_carlson: We also would want to add some text to the spec
describing the rules of precedence,
… whichever way we go.
Nigel: How are you going to work out which way to go?
eric_carlson: We'll put a stick in the ground and we'll be
right.
gkatsev: My assumption is the track element takes precedence
eric_carlson: That would be right, it would override what's in
the file.
gkatsev: Can still have the precedence rules in the VTT spec
but would also need them in HTML
eric_carlson: HTML doesn't say anything about how to process
the contents of the VTT file,
… so maybe no change needed.
jcraig: The other editorial suggestions seem fine to me.
… There was a section question, was just my unfamiliarity with
bikeshed.
gkatsev: Should we open an HTML issue now around this to get
the conversation going.
jcraig: I'll take that as an action and write it into PR 523 so
I don't forget it.
pal: On that last point, if you run into any issues, I can
help, please don't hesitate to ask.
jcraig: You mean pushback on HTML?
pal: Sure, ultimately the question is who writes those tests.
… As Nigel pointed out there's a large body of tests to port to
WPT.
… With reference renderers once we've overcome the issue of
should we have that at all in WPT,
… I'm confident that we'll have the resources to make that
happen.
group confusion caused because Pierre misheard!
gkatsev: With that we can, unless there's anything specific to
WebVTT right now we can end
SUMMARY: @cookiecrook to raise issue on HTML about track
attribute precedence
DAPT
nigel: my goal is to get agreement or a plan to get DAPT into
CR
… we have these 5 open issues
[shares screen to show DAPT issues]
… lots of activity lately
… 4 of these have PRs open for them
… What I would like to do is go over these PRs and see if
there's any actions
… and merge these PRs and do a CfC to request transition to CR
… which would be subject to group policy, which would be 2
weeks
… which would give a change to review the spec as a whole
… editorial things can be fixed, the substantive things are the
important bit
cyril: what if there in a big change?
nigel: we can do a new CR snapshot
cyril: the signal is that people can start implementing
nigel: it triggers a couple of things like starting to do a
changelog
… when we met with the APA earlier in the day, it would've been
helpful to see a changelog
… and we can share with other HR groups
… also makes a need for an IR
atsushi: we need to test but not requirement for CR
nigel: yes, but needed to leave CR
… I suggest we have a dapt-tests repo and start putting stuff
there
… we already have a repo
atsushi: you need a link to the IR to pass pub rules
… can be a stub
… can be a wiki page
nigel: we can have a page on the TT wiki
nigel: start from the oldest
nigel: we need to request delta reviews for HR
nigel: the first one [39]dapt#44
… cyril proposed adding a paragraph note
… and there's a PR for that addition
[39] https://github.com/w3c/dapt/issues/44
there's an approval from me
… which we can merge unless there are issues
nigel: next one, [40]dapt#75 define restrictions per script
type
… we have a PR for this
… which has been approved
… I think there's some changes
… but they're small
… if you're in a prerecording script you shouldn't have audio
recordings
… and either synthesized or audio recoding
… I can merge during the break
[40] https://github.com/w3c/dapt/issues/75
nigel: the next one [41]dapt#227, quite a big one
… also a PR, which is approved
… the idea is to capture metadata for script text to know what
it represents
… and we have a top level scriptRepresents and a represents on
every event tag, which is inherited
… the exciting bit is the content-descriptor
… we have a registry of content descriptors
… hierarchical scheme, like audio, audio:dialogue
[41] https://github.com/w3c/dapt/issues/227
cyril: think of it as you have a time text part and you're
annotating whether the text corresponds to dialog or text on
screen
… can be used for translation
… and produce subtitles files or dubbing script
… the origin for the info to produce all types of content
nigel: there's a constraint; if you say a script event has a
descriptor, but at the document level doesn't include it
… it's an error
cyril: document level helps do an early error so there's no
surprises
nigel: you can be more specific in script event but if you
can't use something that wasn't given
… delimiter is . not ;, so the PR needs a smal update
nigel: [42]dapt#232, responding to implementation issue where
people have non DAPT impl files but have smpte times and
there's not enough info to synchronize
… we have a PR which introduces some metadata is section D,
which describes the problem and adds some optional metadata
… origin timecode, and start of programme timecode
… if those two are the same they can synchronize
… if they're different, you can synchronize, but would need
some work for this
… and they're completely optional
… one of the key points is metadata only not intended to be
used to perform direct synchronization offsets during
presentations
[42] https://github.com/w3c/dapt/issues/232
nigel: last one is [43]dapt#233
… no proposed change here
[43] https://github.com/w3c/dapt/issues/233
cyril: break discussion?
nigel: can we close with no action?
cyril: ok, let's do that
… still discuss it during the break
nigel: proposal is to merge PR and then go through the admin
needed to get a working group decision to request transition to
CR
cyril: I support that
atsushi: I also support it
PROPOSAL: Merge the open Pull Requests and then go through the
admin needed to get a WG decision to request transition to CR
RESOLUTION: Merge the open Pull Requests and then go through
the admin needed to get a WG decision to request transition to
CR
Break
TTML2
nigel: first thing with TTML2, we've been in CR, the last
snapshot is 3 years ago
… we should finish that off
… what do we need for that?
… the implementation report
… shows that the test related to new things we have at least 1
implementation for most things
… that means we need a second implementation in order to
proceed to REC
… we also have validation tests
… I'm not sure the best way to have the second passing
implementation?
… there are validators out there
… I don't think we've got any particular spec changes to make
… one option is to back out some changes
… we also have some open issues on TTML2 but we have not
triaged them
… the easy path is just to do implementations
… it's more complicated if we want to make new changes
… because we don't have an active editor
… does anybody have resource availability?
pal: what's absolutely blocking and essential?
nigel: of what?
pal: all the syntax changes require new tests or not?
nigel: all of the tests are done
pal: what's missing?
cyril: implementations
nigel: there is one passing implementation
nigel: there are 2 separate issues
… as the spec stands right now to take to REC we just need a
2nd implementation
… the second problem is if we want to make further changes to
the spec, we need an editor
pal: assuming we have the 2nd implementation, do we know of
changes that are needed?
nigel: no
… but I need to do a serious runtrough of the open issues
… it's not zero work
pal: my inclination if I would be editor would be ignore
everything that is not marked as blocking
nigel: that's fair
cyril: Reverse question to Pierre's: if we don't get the 2nd
implementation how can we move the spec forward,
… if we need to make a new change to TTML2?
nigel: if we edit a new change in that is substantive
… we need to write tests and implementations that pass that
… as it stands we cannot move from CR to REC without second
implementations
atsushi: once we have REC, if we need new changes we need WD,
CR ...
… we can do another CR with just the changes that have 2
passing implementations
… and defer the non-passing tests to a future version
nigel: most of the ones not passing are validation test
… for presentation there are audio and body features
… font embedding family
… font selection strategy
… font shear
… image in body, including png
… region no default
… line shear
… image opacity
pal: we said we need 2 presentation engines?
nigel: yes
pal: for validation, there is a validation tool that can be
checked and modified
… the presentation tests are more challenging
… what was the second implementation for the 1st edition tests?
nigel: TTPE, IMSC.js or Adhere
… but at the time we considered validation and presentation as
2 implementations
… it was and still is adequate
… the requirements haven't changed
pal: ttval is relative easy to update
… so if ttval was added for both presentation and validation,
would that be sufficient?
cyril: not for presentation
nigel: the audio descriptions would pass
nigel: the action is to rearrange the table so that for each
feature you could see if it had a presentation and a validation
test
pal: if all we need is a validator implementation, that can be
done
cyril: .. in my mind, this is really about getting it done so
we don't have to think about it anymore
cyril: The world has moved on without the 2nd implementation,
and probably without the features at all.
pal: It's more about standards hygiene
Nigel: +1
Cyril: Are there any features in TTML2 required by DAPT?
Nigel: Good question I need to check.
Nigel: I will happily nominate Pierre as an editor of TTML2
TTML2 CR snapshot outdated banner
Atsushi: Issue about the out of date banner on TTML2 old CR
snapshot
[44]TTML2 CR 2020-01-28 with banner
[44] https://www.w3.org/TR/2020/CR-ttml2-20200128/
Atsushi: banner points to latest public recommendation but
after that we have CR
… The question is what do we want from the Outdated banner?
… Do we want to point to the latest Rec from /TR/ttml2 or the
latest publication?
… or some link to current history.
… There could be a possibility that everything before the
latest Rec points to the Rec, and
… after the published Rec everything points to the latest
publication.
Nigel: I think that makes sense, before a Rec point to the Rec
that follows,
… and after Rec point to the latest publication.
atsushi: I will raise this and talk to the systems team about
it, if that's okay for everyone?
Cyril: OK sure
atsushi: I will draft the text and then ask for review on the
TTWG mailing list.
pal: Conclusion on TTML2 2nd Edition?
Nigel: My conclusion was that:
… 1. Add Pierre as an Editor
… 2. Encourage folk to work on a 2nd implementation of the
features that have 1 passing implementation in the IR
Pierre: We have a blocker on implementation. Doing more work on
the spec makes that worse
Nigel: Agree, we shouldn't be addressing issues with feature
changes unless there's a path to implementation
… I expect this to be a small editing job, but the focus should
be on implementation
Pierre: Happy to edit, but contingent on their being focus on
implementation
… We need to have resources, people committing to
implementation work
Nigel: I agree, otherwise it's a waste of time
… We all need to consider whether we have resource to commit.
Let's come back to it in a month's time
… I won't do anything about adding you as editor, for the time
being
TTML Media Type Definition and Profile
Nigel: This document is good. It's a Note, but since we
published it, the Process now has a Registry document type
… Should we leave this as is, or formally make it a Registry?
… In general, I want to do as little as possible with it
… And, given we've done work to create a boilerplate for
Registries, it might make sense to move this to the Registry
track
… Atsushi, can we turn a Note into a Registry and keep the same
shortname?
Atsushi: Some Registries are still published as draft Notes, It
should be possible
… Need to check the procedure for publishing as a first draft
… Reusing the same shortname should be fine
Nigel: So I propose to do the work needed to turn this into a
Registry Track document. No commitment to how soon, though
Cyril: We need an identifier for DAPT, so we'll need to update
it. That could be a good incentive to do it, if we'd be blocked
Nigel: I don't think we would, it's more a hygiene issue
cpn: We've done this in Media WG, we have standalone Registries
and Francois has done
… the work to set up the auto publication
atsushi: Coding systems?
cpn: Yes, they were Notes and we moved them over - WebCodecs
and MSE Bytestream Formats
Nigel: Anything else on the Profile Registry?
(nothing)
Joint meeting follow-ups
Nigel: We had 3 joint meetings this week
… Monday was MEIG, talked about DAPT, and there was interest in
that. Wolfgang expressed interest in writing implementation to
convert DAPT to next-generation audio
… We talked about adding text tracks to MSE. MSE supports audio
and video, not subtitles and captions
… There was no negative reaction, some support. We then took it
to Media WG on Thursday, and it was more positive
… I filed an issue to add it. There were technical questions
raised, to be answered.
… Things like switching the number of tracks available when you
go to an advert
… There's currently a requirement to have the same number of
tracks, but a different set of text tracks might be available
… Today we had joint meeting with APA WG. That was generally
positive as well
Gary: Something mentioned in APA was symbolics, could result in
text track additions
Nigel: Relates to not being able to have markup in text content
in TTML
… Media Session only supports a plain text chapter title, no
markup. So no way to embed left-to-right. It's a more general
problem
Gary: WIth Media Session, would we want it to start using the
chapters kind?
… We'd want to make sure the caption formats can add the
symbolics information
Chris: Does getting symbolics into Unicode become a
requirement?
Nigel: There may not be agreement that it belongs in Unicode
Conclusion
Nigel: Thank you all for your contributions. We covered all the
agenda, success!
… We meet next on Thursday 10 October
[adjourned]
Summary of resolutions
1. [45]Proceed with a minor revision of IMSC as detailed in
the minutes
2. [46]Liaise with ARIB about feature improvements in IMSC,
specifically the backlog issues
3. [47]Merge the open Pull Requests and then go through the
admin needed to get a WG decision to request transition to
CR
Minutes manually created (not a transcript), formatted by
[48]scribe.perl version 229 (Thu Jul 25 08:38:54 2024 UTC).
[48] https://w3c.github.io/scribe2/scribedoc.html
Received on Saturday, 28 September 2024 00:56:58 UTC