Minutes from W3C M&E IG monthly call 4 August 2020

Dear all,

The minutes from today's Media & Entertainment Interest Group call are available at [1] and copied below. The slides are at [2].

Following today's discussion, we have two items to plan into our TPAC agenda:

- Meeting with Web and Machine Learning CG
- Meeting with Web Transport WG

Our next meeting will be on Tuesday 1st September.

Kind regards,

Chris

[1] https://www.w3.org/2020/08/04-me-minutes.html
[2] https://docs.google.com/presentation/d/1xLz-ssVs7Qu72GKTX9-ByAOOM35dzmChLnFlHQOg23Q/edit

W3C
- DRAFT -
Media and Entertainment IG
05 Aug 2020
Agenda

Attendees

Present
  Chris_Needham, Takio_Yamaoka, Kazuhiro_Hoya, Tatsuya_Igarashi, Pierre_Lemieux, Rob_Smith, Barbara_Hochgesang, John_Riviello, Will_Law, Kaz_Ashimura, Gary_Katsevman, Andreas_Tai, Franco_Ghilardi, Nigel_Megitt, So_Vang, Ali_C_Begen

Regrets

Chair
  Chris, Pierre, Igarashi

Scribe
  kaz, cpn

Contents

Topics
  Agenda
  AOB items
  WoT media use cases
  TPAC plans
  Media Integration Guidelines
  Web codecs

Summary of Action Items
Summary of Resolutions

<kaz> scribenick: kaz

# Agenda

Chris: WoT, TPAC plans, Media Integration Guidelines, Web Codecs

# AOB items
  
Chris: A few quick updates. There's a call for review for MediaStream Track Content Hint
.... MediaStreamTrack is used in WebRTC and MediaRecorder.
.... The content hints allow web app to say if the content is expected to be speech or music (for audio), or motion or text (for video).
.... It allows the browser to configure the encoding appropriately.
.... Please take a look, file issues if you have feedback.

https://github.com/w3c/mst-content-hint/blob/gh-pages/explainer.md explainer

Chris: Also liaison letters from ITU-T SG16

https://lists.w3.org/Archives/Member/member-web-and-tv/2020Jul/0001.html LS212

https://lists.w3.org/Archives/Member/member-web-and-tv/2020Jul/0000.html LS213

Chris: The first one is an announcement of VVC codec.
.... The liaison on immersive live experiences could be a collaboration with the Immersive Web groups.
.... Before moving ahead, any other topics for today?

(none)

# WoT media use cases

<cpn> scribenick: cpn

Kaz: The WoT WG are working on use cases and have published on April 9th, possibly media related use cases
.... NHK have submitted one use case for TV synchronization using Hybridcast.. There are more possible use cases, including BBC's grand prix video
.... The WoT would like more possible use cases from the media industry, and input on their existing proposals
.... They invite you to provide your ideas. You don't need to provide use case descriptions in detail, app software, media streams, and devices

Chris: What's the next milestone?

Kaz: I'd like to send a request for contribution or ideas to our MEIG participants, and individually

<kaz> scribenick: kaz

Chris: happy to help
.... media in home, media devices within IoT environment, etc.
.... if it's part of your/your organization's business arias, please let us know

<cpn> scribenick: cpn

Kaz: I'd like to bring any input to the WoT group

<kaz> scribenick: kaz

# TPAC plans

<inserted> scribenick: kaz

Chris: TPAC is a virtual meeting this time.
.... Overall plan is Oct 12-16 for joint meetings and Oct 26-30 for breakouts.
.... Our IG would like to have a joint meeting with TTWG and Media WG
.... to cover text tracks in MSE, Text Track CUE API.
.... We'd like to cover DataCue too.

Nigel: Would make sense to have the joint discussion
.... TTWG produces TTML, WebVTT, IMSC as a profile for TTML.
.... There's a discussion on text track cue to be had. Which group should develop those APIs, Media WG, TTWG perhaps?

Chris: Thanks Nigel.
.... The other group we want to have a joint meeting is the Color on the Web CG.
.... HDR and WCG are topics related to a bunch of groups, Media WG for Capability detection, also CSSWG.
.... There's work happening on color spaces in the Canvas API.
.... We can get updates on progress, and do some cross-group coordination.
.... Also another potential meeting with WoT, depending on how discussions proceed in the meantime.
.... Any other suggestions?

Barbara: Joint meeting with Web Machine Learning Community Group - TPAC or a future monthly meeting. Topic would be reviewing their use case targets before going to a WG.

Chris: Good point, let's follow it up.

Will: Web Transport WG? We'd want to get input from media companies.

<BarbH> +1 on Transport

Will: We don't have an official date yet since the group is not yet created..

Chris: OK
.... We'll make sure to send an invitation to them,
.... and ensure our meetings don't overlap so people who are interested can join both.
.... For the MEIG meetings, we propose two sessions over two days.
.... Oct 19, Monday at 4pm Europe
.... Oct 21, Wednesday at 7am Europe
.... Different times to let Asian people join easier.
.... We are planning to have discussion on CTA WAVE Web Platform tests, a report from Fraunhofer Fokus.
.... If you have any suggestions for the the agenda, please let us know.
.... It would be good to have discussion with CTA WAVE on other topics too, e.g, Content spec, Capabilities, Web Media API.
.... Olease let us know if you want to have other joint meetings as well.
.... Does the plan sound good to you? Any possible conflicts?

(none at the moment)

<cpn> scribenick: cpn

# Media Integration Guidelines

John: We discussed on a call a while ago, but I'll recap.
.... In CTA WAVE, there a discussion around interoperability pain relating to A/V decoders,
.... e.g., devices with only one decoder. How best to handle?
.... The scope is to create an IG note that seeks to describe the major browers, how to handle the APIs, discrepancies.
.... We started with issues to collect info. Categorised into 3 main topics: claiming hardware resources, resource contention, timing.
.... The first step is gathering information on how various UAs handle the issues we've raised.
.... When we have input from all browser vendors, we'll see if there are discrepancies.
.... If it makes sense to bring somethign to WICG or WHATWG to standardize, we'll facilitate that through the IG.
.... It seems likely we'll do that, there will be spec changes needed.
.... We had a call for co-editors. Franco Ghilardi from Comcast will help, but we would welcome more co-editors.
.... We have input from Google (Dan and Chris) on how Chrome handles some of the issues.
.... We're looking for additional input, we've reached out to Mozilla, Microsoft, Apple.
.... We're looking forward to their feedback. Can we use some of the time at TPAC to get their input?
.... Anyone is welcome to file additional issues. Also, if you have input on real world experience on the issues already raised, that would be welcome
.... This would help other developers using the APIs
.... Any questions?

<inserted> scribenick: kaz

Chris: We could set up a dedicated meeting with the browser vendors,
.... Do you have any time schedule for this work?

John: No specific deadline at the moment, but it would be nice to have some discussion during TPAC.
.... Chrome Team did a great job with this.

Chris: Any other questions/comments?

(none at the moment)

Chris: Thanks for your work, John! It's a really good starting point.

# Web codecs

Chris: In previous months we've heard about Microsoft's client-side video editing API proposal, then Web Codecs API. What should the MEIG do next?
.... Links for previous minutes here

https://www.w3.org/2020/06/02-me-minutes.html June-2

https://www.w3.org/2020/07/07-me-minutes.html July-7

Chris: Web Codecs is very powerful.
.... You can use Web Codecs for decoding the media and Canvas for rendering.
.... Is there something we need to think about, e.g., client-side video editing.
.... Not sure about the status of video editing API, is this still needed or can it maybe be implemented on top of Web Codecs?
.... I would like to hear from you. Any particular views on what we should do next?
.... For example, we can gather use cases, evaluate the API against them. Anybody interested in that activity?
.... We heard Google's plan for origin trial during the previous call.
.... I'd like to do a brief walkthrough of the draft spec.

<cpn> https://docs.google.com/document/d/11n6xLkZThJkDc7SBFzUVAynl5kZsxZJtjy_BjbxE8a0/edit#

Chris: Looking at VideoDecoder interface, you initialize with a frame callback you supply.

https://docs.google.com/document/d/11n6xLkZThJkDc7SBFzUVAynl5kZsxZJtjy_BjbxE8a0/edit#heading=h.a235wsc00hud VideoDecoder

Chris: For audio decoding, the AudioFrame contains an AudioBuffer. AudioBuffer comes from the Web Audio API.
.... In ImageFrame, the timestamp information is optional, there's data on the pixel format.
.... A question here is how would this work with WebGPU for further image processing,
.... we'd want to avoid copying fro, GPU memory and CPU memory.
.... This was just a brief overview of the proposed WebCodecs API.
.... We had this kind of idea for media production, editing use cased, but also realtime playback,
.... e.g., a visual mixing editor.
.... So the question is what should we do as the whole MEIG?

<BarbH> Webcodec are working with the WebGPU and the video_shader

John: This is useful, but I don't have any particular use cases myself, though.

Chris: I have one in mind, which is replacing use of WebAudio decodeAudioData, which is a blocking procedure.
.... If we can use Web Codecs instead and stream the audio, we can show progress information in the UI.
.... We're potentially interested in vision mixing as well.
.... I'm wondering how many of us are interested in this from a media production point of view.

Pierre: The client-side video editing idea still on TAG's review. right?

https://github.com/w3ctag/design-reviews/issues/514 TAG review 514

Pierre: This could possibly be done by Web Codecs? Decoding to baseband.

Chris: Web Codecs might not be a direct replacement and possibly need both the proposed APIs

<BarbH> We are looking at Latency-sensitive live streaming. There is alignment with Web Transport

Chris: Individual companies can respond directly.

Barbara: We're looking at latency-sensitive use case, you may want to investigate that.

Chris: Yeah, coupling with Web Transport for low latency media delivery.

Barbara: Right, we're working on PoCs.

Chris: Getting feedback based on implementation is good.
.... We've covered all the topics in the agenda. The next call will be held on Sep. 1 at the same time.

[adjourned]

Summary of Action Items
Summary of Resolutions
[End of minutes]
Minutes formatted by David Booth's scribe.perl version 1.152 (CVS log)
$Date: 2020/08/04 16:37:14 $

Received on Tuesday, 4 August 2020 18:31:20 UTC