Minutes from Media Timed Events Task Force call, 20 August 2018

Dear all,

The minutes from yesterday's Media Timed Events Task Force call are available [1], and copied below. Many thanks to Charles for presenting.

The next Task Force call will be on Monday 17th September.

Kind regards,

Chris (Co-chair, W3C Media & Entertainment Interest Group)

[1] https://www.w3.org/2018/08/20-me-minutes.html

---

W3C
- DRAFT -
Media and Entertainment IG - Media Timed Events TF

20 Aug 2018

Agenda
        https://lists.w3.org/Archives/Public/public-web-and-tv/2018Aug/0015..html

Attendees

Present
        Kaz_Ashimura, Giri_Mandyam, Charles_Lo, Chris_Needham, Ali_C_Begen, John_Luther, Steve_Morris, Will_Law, Francois_Daoust

Regrets

Chair
        Giri

Scribe
        cpn, tidoust

Contents

Topics

        Introduction
        Background
        Architecture
        APIs
        DASH processing model
        Work status
        Questions and discussion
        Timing and synchronization

Summary of Action Items
Summary of Resolutions

<cpn> scribenick: cpn

Introduction

Giri: I asked my colleague Charles Lo to present an initiative from DASH-IF..
.... It's work in progress, so we can only go into so much detail.
.... We'll distribute the slides after the call.

Charles: Thanks, Giri
.... I'm Charles, from Qualcomm.

<kaz> Charles's slides on DAInty API: https://www.w3.org/2011/webtv/wiki/images/4/44/DASH-IF_DAInty_API_Overview_August_20_2018.pdf

Charles: I see others from DASH-IF here, please jump in to clarify anything, if needed.

Background

Charles: There's an interop group at DASH-IF, DAInty: APIs for interactivity.
.... A client could be implemented as a JavaScript library in a web page, or a streaming client in a particular device platform.
.... The origin of this work is a related work item at 3GPP on service interactivity.
.... The goal is to allow users to engage with auxiliary content alongside live or time-shifted content delivered over 3GPP, either broadcast or unicast network.
.... DAInty provides hooks for interactivity, and also measurement on the use of interactivity.
.... There's a work item description document and a technical report.
.... DASH-IF was requested by 3GPP to support API specification.

Architecture

Charles: The APIs are between a DASH client and a service application. The interactivity events could be delivered over broadcast or unicast HTTP.
.... The 3GPP's focus is more on the broadcast case. Event data and media data is passed over the 3GPP network to the interactive application.
.... Event-type information is handled by the DASH client.
.... At 3GPP there's interest in leveraging the DASH client to record measurement of interactivity usage.
.... It's possible for the application to provide this directly, but 3GPP already has a reporting framework, so we want to use this for application level events.

APIs

Charles: There are 3 APIs to be developed. The first one allows an application to subscribe to DASH stream messages, which carry interactivity information.
.... This could be in-band (emsg) or MPD events.
.... An application subscribes to a scheme id and value, and the client dispatches matching events to the application.
.... The second API allows an application subscription to an ISO BMFF timed metadata track.
.... The third is interactivity usage measurement reporting. The usage measurement is delivered to the DASH client, which delivers it to the network server.

DASH processing model

Charles: The timeline diagram shows an in-band event message, carrying some app specific information.
.... It's been subscribed to by some application.
.... An emsg box, which is defined to be at the beginning of a segment.
.... The payload has a presentation time for the start of the event.
.... The duration could span multiple segments.
.... The emsg event must arrive no later than the beginning segment to which it applies, for the event start and duration to be fully usable.
.... The lower picture shows "event dispatch mode", passing from the DASH client to the application.
.... Two cases are possible: dispatch at the time the event arrives (mode 1), or dispatch at the event start time (mode 2).
.... The application can register with the DASH client which mode is preferred.
.... Mode 1 gives the application more time, but in either case the application should be able to respond in time.

Work status

Charles: The DAInty APIs are intended both for web apps, and native apps on a particular device platform, as there are both application environments in the 3GPP model.
.... For the web app case, the desire is for interactivity apps to be defined in DASH.js, an open source DASH player supported by DASH-IF.
.... There are lots of other clients available, but the one developed under the auspices of the DASH-IF is DASH.js.
.... There is already support in DASH.js for SCTE-35, but we want support for more generic events.
.... We're writing in IDL, so not implementation specific. This allows the IDL to be mapped to a specific language or platform.
.... The present focus of work is on the DASH event API, then later on the timed metadata track and interactivity usage reporting.
.... Does W3C see value in standardising an IDL description? Could use WebIDL.
.... We have an agressive timeline, aligned with 3GPP, to complete by end of the year.
.... We're well on the way with the API definition.

Questions and discussion

Kaz: Is it OK to distribute the slides to the M&E IG public mailing list?

Giri: Yes, I will share them after the call.

Kaz: You're working on a concrete API defintion in IDL. Are you interested not only in WebIDL but also TypeScript? Other W3C groups have been struggling to define event APIs in WebIDL.

Charles: We'd be happy to look into that, if W3C has some guidance.
.... We also looked at OMG IDL. Could you provide some more information?
.... We can pick up offline with Giri.

Giri: We did talk about the choice of IDL. The one experience I had was in Fido Alliance, where we used it to define native APIs.
.... I asked Charles to present on this, it's because of a percieved need that the web standards have not met.
.... We want to feed this into WICG.

Charles: I'm not sure how much WebIDL expertise we have in DASH-IF.

Giri: I'd recommend that when this work is in a sufficient state, DASH-IF should send this to W3C under liaison.

Will: IDL isn't our strength, certianly, but we will send under liaison.

Francois: I'm trying to understand how the event model differs from the usual Web eventing model.
.... I understand your mode 2 is similar to the media element's event model, but mode 1 differs.

Charles: Yes, emsg events could arrive at the beginning of a segment. The emsg box describes the start time and duration
.... Notifying the application when the event is received gives it additional time.

Will: I can confirm that. We have a preference for mode 1, so that we can put the events into a helper library to dispatch at the right time.
.... It's never wrong to get an event early, but it is wrong to get it late.
.... We've had a long thread on this. My concern with the document is with some of the language, such as event API.
.... I want to make sure that what's proposed fits, it is specific to certain players?
.... Need to avoid creating a confusing API surface for developers.

Charles: 3GPP's interest is to have some reference implementation, could be implemented in DASH.js

Will: another comment on language: interactivity is a broad term. Web developers think of click events on UI controls. We need a clear definition.

Charles: Absolutely.

<tidoust> scribenick: tidoust

Chris: You say the event carries a start time and a duration. Is it possible for an event to be unbounded?
.... So you don't know when it ends in the future.

Charles: In principle, it is. We're basing that on the DASH specification. I don't remember exactly what's in there.

Giri: Internally, we've struggled with TTML on ways to define an unbounded metadata cue. "Display once, keep it on the display until I tell you to get rid of it". We haven't found something in TextTrack that lets us do that out of the box.

Chris: OK, I would have to go back and read the DASH spec.
.... My next question is related to the work we're doing in this Task Force. We're producing a use cases & requirements document to gather requirements for user agents that would support these kind of events.

<kaz> Media Timed Events Use Cases and Requirements document: https://w3c.github.io/me-media-timed-events/

Chris: We have agreement on generic use cases, I believe, but need more details on requirements that you may bring here.
.... For example, does the application have to specify the scheme id and value to receive events?

Charles: There may be many types of events and the application may want to know about specific events that it's interested in, so yes, there's a way to subscribe to events based on the scheme ID URL.

Chris: I think the HbbTV specification has a similar API for exposing events that operates in a same way.

Charles: Is this the part that transfers the event to some TextTrack?

Chris: Yes, that's right.
.... In HbbTV, it's always an opt-in from an application perspective.
.... And then the event itself is defined as a DataCue that exposes the binary data from the event message.
.... Is that what you're looking for as well, a generic blob that the application knows how to interpret?

Charles: [looking up examples of events in the DASH spec]
.... Looking at the DASHEventMessageBox definition, the intent is to provide the timing information as well as the data to the application so that the application can make use of it.

Chris: Related to MPD events. Because they're delivered through the MPD, you need an API that allows the application to pass the events into the media player?

Charles: Yes, not explained here, but we have a set of slides on that as well.
.... Both types should be supported.

Chris: I guess it's just that there would need to be an API to allow an application to pass in objects with start time and end time and data, rather than having that extracted from the media segments by client.

Charles: Yes, the DASH client extracts that info, the only difference is where the info resides.

Chris: I'm thinking of this from a Web platform API point of view, we would want the user agent to take care of the emsg box parsing, whereas it's application-level JavaScript that would parse the events embedded in the MPD. The requirement is the same, but done by different things.
.... Another question: If we were to push for a Web platform API to do this, is the preference still for the Mode 1? As I understand it, the current model that we have, using TextTracks, is closer to Mode 2.

Charles: I don't know which one is the ideal mode. We would want to allow both modes, and allow the client to subscribe to the mode it's interested in..
.... I don't know that we want to specify one or the other.

<cpn> scribenick: cpn

Francois: There's no event when a change is made to the list of cues, even when not active.
.... Maybe it's just a matter of adding an event when the list changes.

<tidoust> scribenick: tidoust

Chris: Interesting that we don't have something for mode 1 on the Web.

<cpn> scribenick: cpn

Will: I'd support Francois on that.

<tidoust> scribenick: tidoust

Chris: We shouldn't be going too deep into API design, but what you've been describing Charles really helps shape the specific API requirements.

Timing and synchronization

<kaz> https://github.com/w3c/media-and-entertainment/issues/4

Chris: What kind of timing requirements did you specify for event dispatch? For example, Mode 2 event dispatch is probably more timing sensitive.

Charles: I think the idea is that the application developer should be cognizant of the fact that there may be some sort of delay.
.... It probably cannot be done with guarantees within the right millisecond..
.... It would be important that the DASH player provides the current presentation time when it performs the dispatch.
.... This allows synchronization.

Giri: We're almost out of time for this call. We can continue the discussion offline.
.... Thank you, Charles, for this presentation.

<kaz> [adjourned]

Summary of Action Items
Summary of Resolutions
[End of minutes]
Minutes formatted by David Booth's scribe.perl version 1.152 (CVS log)
$Date: 2018/08/21 08:33:41 $


-----------------------------
http://www.bbc.co.uk
This e-mail (and any attachments) is confidential and
may contain personal views which are not the views of the BBC unless specifically stated.
If you have received it in
error, please delete it from your system.
Do not use, copy or disclose the
information in any way nor act in reliance on it and notify the sender
immediately.
Please note that the BBC monitors e-mails
sent or received.
Further communication will signify your consent to
this.
-----------------------------

Received on Wednesday, 22 August 2018 09:22:59 UTC