Minutes from Media Timed Events Task Force call, 20 May 2019

Dear all,

The minutes from last Monday's Media Timed Events Task Force call are available [1], and copied below.

The next call is planned for Monday 17th June.

Kind regards,

Chris (Co-chair, W3C Media & Entertainment Interest Group)

[1] https://www.w3.org/2019/05/20-me-minutes.html

--

W3C
- DRAFT -

Media and Timed IG - Media Timed Events TF

20 May 2019

Agenda

Attendees

Present
	Kaz_Ashimura, Chris_Needham, Rob_Smith, Steve_Morris, Nigel_Megitt

Regrets

Chair
	Chris

Scribe
	kaz

Contents

Topics

Agenda
	IG Note
	Wrap up
	Next TF call
	Summary of Action Items
	Summary of Resolutions

# Agenda

<scribe> scribenick: kaz

Chris: Review the open issues against the IG note,
.... then look at the WICG explainer.
.... We'll use this to get into the API design, I want to capture some open issues.
.... This call is under the WICG rules, so please join WICG if you haven't already.
.... I propose continuing to have calls, while it's useful.
.... We're moving from the requirements phase to the design phase.

# IG Note

Chris: Thanks to Kaz, we've published the IG Note.

https://www.w3.org/TR/2019/NOTE-media-timed-events-20190516/ IG Note

Chris: There are a few issues to go through, before the document is complete.
.... We can address the issues, then publish the Note again, probably as a final version.

https://github.com/w3c/me-media-timed-events/issues open issues

<cpn> https://github.com/w3c/me-media-timed-events/issues/43 Issue 43

Chris: Issue 43 asks adding some detail on events supported in HLS.
.... It's on my list to do.
.... What kinds of events to be standardized?
.... Our focus has been emsg for DASH, do they want to standardize the HLS events?

<scribe> ACTION: Chris to add detail on events supported in HLS to the IG Note

Chris: Issue 41 is about terminology

<cpn> https://github.com/w3c/me-media-timed-events/issues/41 Issue 41

Chris: We haven't defined what "event" means. I hope to add clarification without rephrasing the document in terms of cues,
.... add some suitable definition. We're talking about cue onenter and onexit events, so we do mean JS events.
.... Need to review the document for consistent definition.

Nigel: It's worth making a distinction between when the event is added to the queue and when it's actually handled.
.... For example, a TextTrackCue with enter/exit time associated with it.
.... Time marches on describes preparing the event,
.... where it gets added to the queue at one moment in time, some time after its begin time, then its onenter hander is called.
.... Then the onexit handler is called sometime after that.
.... These distinct concepts need to be clear, so that when we ask for a change, it's clear what we're changing.

Chris: Is this document the right place for that?
.... We have the explainer document, and we'd need to propose a change to HTML.
.... I wonder if we should put that detail into the HTML issue, or into this document?

Nigel: Good point. The gap analysis talks about timeupdate events, that's separate to cues.
.... What it's really talking about are JavaScript events, add to terminology.

https://w3c.github.io/me-media-timed-events/ Editor's draft

https://www.w3.org/TR/2019/NOTE-media-timed-events-20190516/ CR version

Chris: But we also use "event" as meaning DASH emsg event,
.... so there's two meanings depending on the context.

Nigel: We should not do that. Are the ISO BMFF emsg really events, or messages?

Chris: They're more like messages, but the terminology they use is "event", and we've carried that term into this document.
.... I want to minimise the changes to the Note, but also resolve ambiguity.

<scribe> ACTION: Chris to review the IG Note, add a definition for "event", and disambiguate where needed

Chris: Issue 39, cues with unknown end time

<cpn> https://github.com/w3c/me-media-timed-events/issues/39 Issue 39

Chris: There is a PR, adds text to Gap Analysis and Recommendations sections.

<cpn> https://github.com/w3c/me-media-timed-events/pull/46 Changes for PR 46

Chris: Rob, does this PR capture your points?

Rob: Will look through it and get back to you.

Chris: We're making two recommendations: cues with unbounded end time, also changing a cue's end itme.

Rob: No objections, want to distingush between cues with end of media as end time, and cues with unknown to-be-determined end time.
.... For example, chapter end points. Would require an update event to set the end time.

Chris: What I've written is allowing the end time be to set to infinity, making it valid to end of media stream.
.... Can we use the same mechanism?

Rob: An idea I had was to use -Infinity to signal the unknown to-be-determined case, not thought it through fully.

Nigel: It's fair enough to use a keyword for the not-yet-defined concept,
.... but some concern with using such a numerical value for an unrelated concept.

Rob: I have a concern that negative numbers are allowed, and have special meaning.
.... You can have cues that are executed before the start of the media, using negative cue times. I came across it in the HTML spec.

Chris: I agree with Nigel about using special sentinal values like that.
.... We can introduce the concept, then figure out the mechanism.

Rob: I have no objection to that, just wanted to flag it at this stage.

Chris: I want to start opening issues in the DataCue repo for each question..

Kaz: I also think it's natural to use "unknown" as the keyword given the purpose is unknown end time.

Steve: It seems far safer, rather than trying to use a magic number.

<scribe> ... "unknown" is different, "infinity" is less magic.

Chris: Rob, would you capture the point on the issue?

Rob: yes, sure

<scribe> ACTION: Rob to raise an issue in the DataCue repo for unknown to-be-determined cue end times

Chris: Issue 36, add more drivers for better synchronisation

<cpn> https://github.com/w3c/me-media-timed-events/issues/36 Issue 36

Chris: Is this a new use case, to do with rate of speech and captioning?

Nigel: Actually it's the rate of display and reading.
.... Errors in a presentation time of subtitle captions, the effective reading speed needed is very high.
.... A good reason for requiring better synchronization.
.... The initial comment was about timing accuracy,
.... the other thing is existing requirements in other places, e.g., in Nordig DVB receivers,
.... timing precision between two frames.
.... With the current in-spec tolerance, you can have one subtitle just under 250ms late,
.... the next could be just 1ms late. They'd be authored with time between them,
.... being a quarter second less in practice, and still valid per the current spec.

Chris: This seems like additional information to 3.4 use case?

Nigel: That's fine.
.... BTW, on this repo, could add a link to the draft document at the top?

Chris: There is one on the README.md, but would be handy to have the link at the top.

kaz: do you mean at the top of https://github.com/w3c/me-media-timed-events/?

<scribe> ACTION: Kaz to give Chris access to the Settings for the Media Timed Events GitHub repo

Nigel: Your suggestion is adding text to section 3.4? I think that would make sense.

<scribe> ACTION: Nigel to draft text to add to section 3.4 for high rate captions

Chris: Issue 29, Mark's comments

<cpn> https://github.com/w3c/me-media-timed-events/issues/29 Issue 29

Chris: We've resolved all of these, so can close.
.... Issue 28, we don't have an issue template.

https://github.com/w3c/me-media-timed-events/issues/28 Issue 28

Chris: Do you have one we can borrow, to use as inspiration?

Nigel: Yes, maybe not quite appropriate.

<nigel> TT-Requirements template

Nigel: It's optional, but helps people structure their issues.

Chris: Issue 22, timing requirements for different use cases.

<cpn> https://github.com/w3c/me-media-timed-events/issues/22 Issue 22

Chris: The MEIG issue is broader than Media Timed Events.

https://github.com/w3c/media-and-entertainment/issues/4

Chris: We've captured the frame accurate requirement, but not quantified timing requirements for other use cases.
.... If we use the same underlying mechanism for all events, maybe not needed.
.... Other use cases, e.g, map tracks or ad overlays may have different requirements.
.... Maybe we can leave this for external feedback,
.... if different timing guarantees for different kinds of events is needed.

Rob: In Lyon there was discussion about audio events,
.... something like a beep at an appropriate time, and 20ms accuracy needed for perception of hearing an audio cue and seeing something on screen.
.... Something to investigate in incubation.

Chris: Let's close this as is, no changes. Can add detail, if needed.

Rob: I agree.

Chris: Issue 16, WICG submission.

<cpn> https://github.com/w3c/me-media-timed-events/issues/16 Issue 16

Chris: Can close this.
.... Issue 6, add more use cases

<cpn> https://github.com/w3c/me-media-timed-events/issues/6 Issue 6

Chris: I don't think we need more at this stage. Maybe need to review the 3GPP service interactivity.
.... Issue 2, DataCue or a more specific API for DASH and emsg events

<cpn> https://github.com/w3c/me-media-timed-events/issues/2 Issue 2

Chris: This belongs more with the explainer than the requirements document.
.... Should DataCue simply expose an ArrayBuffer with raw bytes of the in-band event,
.... or should we have specific interfaces for specific kinds of events?
.... This was debated when DataCue was originally proposed, push-back from implementers.
.... This issue is to revisit the debate. Pros and cons each way.
.... For out of band events the DataCue proposal we have now is clear, you can give it any shape data you want.
.... We can open the same issue in the DataCue repo and close this.
.... (Discussion about migrating issues)
.... Issue 1, ISO BMFF Byte Stream Format spec

https://github.com/w3c/me-media-timed-events/issues/1 Issue 1

Chris: It was pointed out that emsg handling may not belong there, as it's not MSE specific.
.... This points to the question of how to specify the DataCue.
.... In the DataCue explainer, I mention updating the Sourcing In-band Media Resource Tracks document.

<cpn> https://github.com/WICG/datacue/blob/master/explainer.md#in-band-event-processing

Chris: There'd be a spec for DataCue, and another describing how to extract events from the different kind of media containers, using a registry.
.... Sourcing In-band Media Resource Tracks could be a home for this.
.... It's a single document, could be split into a registry.
.... There's a section for DASH and other media formats.
.... It describes sourcing text track cues, but I don't know how widely it's used,
.... e.g., how much test coverage in WPT there is, and how much implementation coverage there is.
.... I wouldn't want to make writing an emsg in CMAF spec dependent on getting this document into a normative state.
.... Nigel, are you familiar with it?

Nigel: No, not particularly. I'm familiar with how TTML goes into ISO BMFF.
.... Are we talking about ISO BMFF mainly, or other formats as well?

Chris: Jer's comment suggests they may want to do something around HLS.

Nigel: There are several potential sources of events, then: manifests (HLS and DASH), then the payload data itself.
.... HLS supports fragmented MPEG-2 TS and fragmented MP4, lots of permutations.

Chris: This raises some questions.
.... Is there somebody from community who has expertise and is willing to do the work?
.... Should we update this Sourcing In-band Media Resource Tracks document to cover everything, and split it by format,
.... and move it to the Rec track?
.... I'd want to decouple the work so we can proceed with emsg and DataCue without getting into a much bigger piece of work around in-band text tracks.
.... This document is referenced from the HTML spec, and I'm not sure what its status is.

# Wrap up

Chris: I have a lot of questions about the shape of the API, to discuss on the next call.
.... Hope to have some discussion at the Media Web Symposium at Fraunhofer Fokus this week.

# Next TF call

kaz: June 17?

Chris: Yes

[adjourned]

Summary of Action Items

[NEW] ACTION: Chris to add detail on events supported in HLS to the IG Note
[NEW] ACTION: Chris to review the IG Note, add a definition for "event", and disambiguate where needed
[NEW] ACTION: Kaz to give Chris access to the Settings for the Media Timed Events GitHub repo
[NEW] ACTION: Nigel to draft text to add to section 3.4 for high rate captions
[NEW] ACTION: Rob to raise an issue in the DataCue repo for unknown to-be-determined cue end times
 
Summary of Resolutions
[End of minutes]
Minutes formatted by David Booth's scribe.perl version 1.152 (CVS log)
$Date: 2019/05/27 17:57:48 $

Received on Tuesday, 28 May 2019 07:00:44 UTC