Minutes from Media Timed Events Task Force call, 21 January 2018

Dear all,

The minutes from Monday's Media Timed Events Task Force call are available [1], and copied below.

Kind regards,

Chris (Co-chair, W3C Media & Entertainment Interest Group)

[1] https://www.w3.org/2019/01/21-me-minutes.html

--

W3C
- DRAFT -
Media and Entertainment IG - Media Timed Events TF
21 Jan 2019

Agenda
    https://lists.w3.org/Archives/Public/public-web-and-tv/2019Jan/0008.html

Attendees

Present
    Steve_Morris, Ali_C_Begen, Chris_Needham, Kaz_Ashimura, Rob_Smith, Francois_Daoust, Nigel_Megitt, Song_Xu

Regrets
    Mark_Vickers

Chair
    Chris

Scribe
    kaz

Contents

Topics
    Status of publishing the IG Note
    Review feedback items
    WICG update
    Timing requirements and cue end times
    WICG update (continued)
    Open discussion on areas to explore
    Next IG call

Summary of Action Items
Summary of Resolutions

<scribe> scribenick: kaz

# Status of publishing the IG Note

Chris: We ran a call for consensus around Christmas, now closed.
.... No comments, so that is approved to go ahead.
.... Would like to propose we review the comments by Mark,
.... then take the snapshot when that's done, rather than publishing the Note right away.
.... I'm working through the comments right now.
.... What do you think?

(no objections)

# Review feedback items

Chris: Mark has left some really helpful comments.

https://github.com/w3c/me-media-timed-events/issues/29 issue 29

Chris: Some of this is to do with the document formatting, some I've addressed already.
.... The main point to discuss is the use case section,
.... and we've not covered important use cases such as ad insertion.
.... Is there existing text that would fit the draft? If not, I can write something.
.... His other comment mention EBIF. Can include if it leads to another use case or requirement.

https://en.wikipedia.org/wiki/Enhanced_TV_Binary_Interchange_Format EBIF

Chris: I'll look at the use cases, to reformulate them.
.... He also mentions the WebVMT case which leads with the technology rather than the use case.

Rob: I can look at that.

Chris: timed interaction event
.... Some of the rest of the comments are about making the document clearer.
.... Would be good to get the item on ad-insertion cues into the doc,
.... one of the big drivers coming from CTA WAVE, so it's important to include.
.... In section 4, he mentions adding CMAF. We talk about emsg in reference to DASH but not CMAF.

https://w3c.github.io/me-media-timed-events/ Media Timed Events draft

<Zakim> nigel, you wanted to propose that we clarify that they are demonstrating requirements described elsewhere in the doc, when we get around to the query about why various docs are in

Nigel: Mark makes some good points, he's not sure why some of those documents are there.
.... Not all of those are standards, more guidelines.
.... They're there because they provide evidence for the recommendations in section 6.
.... But that's not clear. I could suggest a restructuring,
.... it would be helpful to make each major section a use case, describe the evidence for why change is needed,
.... and the recommendations related to that.
.... For example, synchronization, which isn't mentioned as a use case, rather the gap analysis.
.... At that point, we might want to mention BBC subtitle gudelines, quote relevant part, make conclusion.
.... Also a question about why WebVTT is there.
.... HbbTV is about emsg, that's OK.
.... This could make it more readable and clearer, and why the recommendations are there.

Chris: I have made some updates. There is an open PR, where I'm making changes based on the comments.
.... For example, I describe why I included WebVTT, because you can use it for out of band events.
.... There's an example of it in the HTML spec.

https://github.com/w3c/me-media-timed-events/pull/30 PR 30

Chris: looks like a spot of programming
.... In response to Mark's comment, I've updated the section 4.6
.... should be clearer now
.... Regarding BBC subtitle guideline, I've shortened it to the main point,
.... which is about aligning captions with shot changes.

https://w3c.github.io/me-media-timed-events/#bbc-subtitle-guidelines 4.4 BBC Subtitle Guidelines

Chris: It's about extracting from that reference what the motivation is.
.... I'm hoping that I can keep much of the existing structure,
.... because it would take work to reorganise.
.... So I'm hoping that the changes I'm making will make the document more cohesive and clearer.

Nigel: OK, we can review the changes.

Chris: What I want to get to is publishing this doc, then focus on the explainer for WICG,
.... which the browser implementers will need, copy from this doc.

https://github.com/w3c/me-media-timed-events/pull/30/files files changed

Chris: You can review the changes from the PR. When it's ready for merge, we can do another review.
.... (going back to the issue 29)
.... I'm currently studying the time marches on algorithm.

https://github.com/w3c/me-media-timed-events/issues/29 issue 29

Chris: On the last call, Nigel, you mentioned 20ms as a number we could use,
.... being half of the typical frame rate, 25 FPS.
.... Is this simply a case of reducing the 250ms for timeupdate events down to 20ms,
.... or is it a more involved changed, but we're getting into design.

Nigel: It is a complicated algorithm, it's doing a few things at once.
.... The safest thing would be to change as little as possible,
.... simplest to reduce the number from 250ms to 20ms,
.... which would meet most of the use cases.
.... I don't think it's possible to do a lot better.
.... If what you want is for the main events to progress smoothly and handle an arbitrary set of events,
.... each could take any time to process, and it's single threaded, there's a lot of constraints.
.... The best you could do is to say that if you author your page so that nothing takes a long time,
.... at least we'll honour that and process the events with better synchronization than at present.

Chris: Yes, if you have a long running event handler, the more time there is to miss events.
.... I thought maybe a different approach could be to deliver all the events on the timeline since the handler was last invoked.
.... But really what I want to do here is formulate it as a requirement statement.
.... I've got some draft text, so I'll put that into the document for review..
.... The issue that Jon Piesing raised was related to the HbbTV implementation, which is a native DASH player.
.... There, the UA is responsible for extrcting the events and creating the cues.
.... If you do it in a web app, you can add onenter/onexit handlers, and time marches on will guarantee all those will be called.
.... If instead you're using the cuechanged event and inspecting the activeCues list, then you'll possibly miss cues.
.... Maybe this too much detail for now. We can save the detail for the next phase, the WICG incubation.
.... In addition to Mark's comments, there are many "Editor's Notes" within the draft, need addressing.
.... What do you think about the Note right above 5.1.2?

https://w3c.github.io/me-media-timed-events/#synchronization-and-timing 5.1..2 Synchronization and timing

[[

Editor's note

To support DASH clients implemented in web applications, there is therefore either a need for an API that allows applications to tell the UA which schemes it wants to receive, or the UA should simply expose all event streams to applications. Which of these is preferred?

]]

Chris: This could enable optimization. HbbTV use opt-in, where applications subscribe to kinds of events.
.... This could be our recommendation in general.
.... Do we have a point of view for this, or maybe it doesn't matter either way?
.... If so, could possibly remove some of the recommendations.

-> The API should allow web applications to subscribe to receive specific event types.

https://w3c.github.io/me-media-timed-events/#subscribing-to-event-streams 6..1 Subscribing to event streams

Nigel: It could be good practice to subscribe, it's not just optimization for implementations but optimization for network traffic as well.
.... Reduce amount of data on the network. If there are say 20 different event streams available,
.... subscription would make sense, reduce mobile bandwidth.
.... That's an architectural thing. When that API gets built and who wants to use it first is another question.

Chris: The next recommendation is about out of band events,

https://w3c.github.io/me-media-timed-events/#out-of-band-events 6.2 Out-of-band events

Chris: needed to support WebVMT, where the events are delivered separately to the video.
.... The Webkit implementation should be OK for this.
.... For event triggering, there are the different modes coming from DASH-IF..
.... Mode 1, when parsed from container, mode 2, at the right time on the timeline.
.... Not sure if mode 1 is currently supported, but can review that later.

<tidoust> [I note en passant that Out-of-band events could be useful as well for "Bullet Curtain" events that Song raised]

https://w3c.github.io/me-media-timed-events/#event-triggering 6.3 Event triggering

Chris: and in-band event processing

https://w3c.github.io/me-media-timed-events/#in-band-event-processing 6.4 In-band event processing

Chris: In-band tracks to be updated? Or specify elsewhere?

https://dev.w3.org/html5/html-sourcing-inband-tracks/ Sourcing In-band Media Resource Tracks from Media Containers into HTML

Chris: For now, I've said we recommend updating this to handle other kinds of in-band events,
.... and consider splitting it into a registry, with a spec per media format..
.... Another large piece of work to do.
.... Any comments or suggestions?

Ali: I'd like to review those sections.
.... There was an MPEG meeting last week, a lot of time spent on events. There a lot of new recommendations.
.... I would like to make sure we are lined up with your recommendations.
.... I'll ask some others to look at your spec.

Chris: Excellent, thank you.
.... Can I ask you a question? We refer in our document to the Carriage of Web Resources in ISO BMFF
.... Should we still reference this draft, given we're really focused on emsg and out of band events. Or should we follow up separately?

-> https://w3c.github.io/me-media-timed-events/#mpeg-working-draft-on-carriage-of-web-resources-in-iso-bmff

Ali: It may be not so relevant. It's still a working draft, but it's good idea to keep it here unless there's a good reason not to include it.
.... There could be a future edition.

Chris: I'm thinking of Mark's comment about explaining why we have these related standards.
.... I'll add a note about the current status. At this state it's not clear that it brings requirements.

Ali: That's true, and it might be the case in future too, but also a good idea to keep an eye on it.

Chris: I agree. If you want to come and talk with the IG about that, very welcome.

Ali: Sure.

# WICG update

Chris: Rob, would you like to give an update on some of the recent conversations we've had?

Rob: Back at TPAC in Lyon, we made a proposal to take this to WICG and Chris has created a repo for this work.
.... That's been blocked for some time, but now it's moving ahead, we have a first draft of an explainer document.

https://github.com/chrisn/datacue/blob/master/explainer.md DataCue explainer

Rob: I propose expanding the structure, to include use cases from the media timed events document and gap analysis.
.... Also to include placeholder sections to allow others to contribute.
.... I identified two areas as new sections needed are: current implementations
.... (we have WebKit implementation currently), but also another equivalent section for other browsers.
.... The gap analysis leads to identified requirements.
.... I take on board the comment about use cases, so we can come back to WebVMT.
.... Start with use case, lead to requirements. This is a good start.
.... Chris suggested using his GitHub to can assemble the document.
.... The 5 use cases in the Media Timed Event Note are a good start,
.... refine those to simple testable items. In the end we can show we've made progress, tests to show we've satisfied the requirements.

# Timing requirements and cue end times

Rob: Thinking of timing requirements, I don't know how much scope there is to meet those requirements, part of the investigation process.
.... The final thing discussed is the existing TextTrackCue object in HTML, has an end time which is mandatory.
.... In WebVMT streaming use cases you don't know when the end time is.
.... It's helpful to be able to create a cue that extends to the end of the media, and we don't care when that is.
.... Eric has said there's wording already there, but it seems to be for reading rather than writing, a cue with unknown end time.
.... It sounds like a good solution.

Nigel: That's interesting one. A couple of comments.
.... One is not yet knowing the end time, in principle also applies to live subtitles,
.... but are you sure you really need a possibly null end time?
.... Can you architect the system so that even if the cue has a defined end time, that doesn't necessarily mean that you have to do something at that end time.
.... You could either have a null onexit handler whose time is irrelevant, or far in the future,
.... or you could define a semantic where immediately prior to firing the onenter) handler of a cue,
.... make sure all the onexit handlers of previously entered but not yet exited cues are handled first, in that order.
.... I'm just wondering if it's necessary to have an exit time?
.... The time marches on spec wants to do filtering on the start time and end time.

Rob: My understanding is that you specify the start time and end time of the cue.
.... If you don't specify the end time, how do you know when to stop displaying the cue?
.... Or how do you refer back to that event to update it at a later time?

Nigel: There's an issue with live captions, like VTTCue, where appearance/disappearance is managed by the UA.
.... For live captions, the end time is not known at authoring time.
.... Could architect DataCue differently to avoid that problem,
.... where a new cue arriving effectively modifies the previously visible cue's end time.

Rob: Looking at it from a metadata point of view, the use case is drawing a track on a map.
.... A common use case is we have video and location coming in, and we want to update the line on the map.
.... We don't want it to disappear, it should continue for the duration of the media clip.
.... We want the whole journey to be displayed at the end of the media clip.
.... The cues are successive sections of a line.
.... We could either continually update the end time of a cue, or just say that it extends to the end.
.... Seems simple to me, but I'm open to other suggestions.

Nigel: If your cue represents where you are in space within a period of time, and you make all the end events the same,
.... that suggests you're at lots of places simultaneously. But that's not what you mean.
.... There's another model, where you could update some data state, so each new event ends the previous event.
.... This could apply in other cases as well, where each cue automatically ends the previous cue.
.... I think that the issue of identifying the location applicable at each moment during the presentation is separable from the issue of whether or not to draw the line.

Rob: This is a discussion for offline?

Chris: Another example is chapterisation of live video, where you don't know where a chapter ends at the time it starts,
.... so you want the ability to start the event and update when it ends.
.... Can we pick this up separately?

Nigel: Yes

# WICG update (continued)

Chris: There's discussion thread between Rob and myself and people at Apple..
.... I'm keen to continue discussions in public.
.... Ideally this should go under the WICG repo, but for now it's under my personal GitHub account,
.... following Marcos's suggestion. Need to follow up with him about that.
.... He's raised it with Mozilla.

<cpn> https://github.com/mozilla/standards-positions/issues/122

Chris: My overall plan is continuing this as an IG document, continue on the explainer.
.... When the IG doc is ready, circulate that among the browser guys.
.... Next phase is incubation and spec development, needs their involvement.
.... Who will write the API definitions?
.... Need to have the right information written down to have discussions.
.... This DataCue explainer is just a copy of the issue we posted to the WICG Discourse forum,
.... feel free to raise issues there.

# Open discussion on areas to explore

https://github.com/w3c/media-and-entertainment/issues/4 Frame accurate seeking of HTML5 MediaElement

Chris: There's a long discussion thread on frame accurate seeking, then goes into frame accurate rendering,
.... interest from non-IG participants as well. I'd like to follow up on it, but it's not really in the scope specifically for the current DataCue work..

<nigel> +1 to what cpn said - cue timing precision has no impact on seeking accuracy, the latter is a separate problem.

Chris: Is this something any of you have strong feelings about?

Nigel: You're right, it's a separate problem. Not sure how much work is really needed.
.... There are web apps that do frame accurate seeking, it may be painful.
.... May not be a strong requirement if there are workarounds.

<tidoust> [I don't think that's limited to "seeking" accuracy, though]

Chris: Discussion led to rendering issues.

<tidoust> [I believe I said I'd take a first stab at a document that summarizes the discussion. I still haven't done that but I'm still happy to!]

Chris: AOB?

(none)

Chris: Let's continue with the calls until the docs are done and we can close the TF.
.... Once the MTE discussion moved to the WICG side, maybe we still want to have calls. Depends how people want to work.

# Next IG call

Rob: When is the next call?

Chris: We meet on the 3rd Monday of each month, so Feb. 18.

Rob: OK

Chris: The document should be done by that time.

[adjourned]

Summary of Action Items
Summary of Resolutions
[End of minutes]
Minutes formatted by David Booth's scribe.perl version 1.152 (CVS log)
$Date: 2019/01/24 05:13:35 $


-----------------------------
http://www.bbc.co.uk
This e-mail (and any attachments) is confidential and
may contain personal views which are not the views of the BBC unless specifically stated.
If you have received it in
error, please delete it from your system.
Do not use, copy or disclose the
information in any way nor act in reliance on it and notify the sender
immediately.
Please note that the BBC monitors e-mails
sent or received.
Further communication will signify your consent to
this.
-----------------------------

Received on Thursday, 24 January 2019 09:58:23 UTC