[minutes] Media WG Teleconference - 2021-09-14


The minutes of this week's Media WG teleconference are available at:

... and copied as raw text below.

The group discussed MSE issues. Another call will be organized at the 
same time in two weeks, in other words on Tuesday 28 September 2021 at 
9pm UTC.


Media WG Teleconference
14 September 2021

    [2]Agenda. [3]IRC log.

       [3] https://www.w3.org/2021/09/14-mediawg-irc


           Bernard Aboba, Chris Cunningham, Chris Needham, Cyril
           Concolato, Eric Carlson, Francois Daoust, Gary
           Katsevman, Greg Freedman, Jan-Ivar Bruaroey, Jer Noble,
           Joey Parrish, Matt Wolenetz, Peng Liu, Zachary Cava

           Chris, Jer



     1. [4]A couple of media-related joint meetings during TPAC
     2. [5]Media Source Extensions
          1. [6]MSE for WebCodecs (#184)
          2. [7]MSE-in-Workers: {Audio,Video,Text}Track{,List} IDL
             in HTML need additional DedicatedWorker in Exposed
          3. [8]Threading lifetimes (#277)
          4. [9]Timing of initial HAVE_METADATA transition (#275)
          5. [10]Object URLs vs srcObject (#156)
     3. [11]Next meeting

Meeting minutes

   A couple of media-related joint meetings during TPAC

    [12]TPAC 2021 Group meetings

      [12] https://www.w3.org/wiki/TPAC/2021/GroupMeetings

    cpn: focus of today MSE issues
    … first AOB item: TPAC
    … media ig planning meetings to assess future direction of web
    tech for consumer devices
    … in collab w/ CTA WAVE and others
    … date: 25 October, more details coming soon
    … WG members invited to join

    aboba: Also joint meeting w/ WebRTC group
    … on 26 October

   Media Source Extensions

    cpn: today's focus, MSE issues
    … thank you for cfc replies
    … resolved: MSE in workers is ready to integrate

     MSE for WebCodecs (#184)

    [13]Media Source Extensions for WebCodecs - Explainer


    wolenetz: First issue, [14]#184
    … MSE for WebCodecs

      [14] https://github.com/w3c/media-source/issues/184

    wolenetz: looking for authors to try origin trial, and feedback
    on API
    … first promise-based MSE API
    … summary: MSE would take WebCodecs chunks rather than
    containerized bytestreams
    … expecting interest from player libraries, hls.js, etc, who
    could use this to avoid re-muxing

     MSE-in-Workers: {Audio,Video,Text}Track{,List} IDL in HTML need
     additional DedicatedWorker in Exposed (#280)

    wolenetz: [15]#280 how to expose HTMLMediaElement tracks in
    … currently, tracks interfaces exposed only in Window
    … 1) do we need tracks in worker?
    … 2) if yes, how to update html5 to go about it

      [15] https://github.com/w3c/media-source/issues/280

    cpn: changes to html should go to whatwg
    … question: tracks exist both on HTMLMediaElement and
    SourceBuffer. What's the difference?

    wolenetz: w/ SourceBuffer API, you can ask SB what tracks it
    specifically provides
    … note: currently Chromium doesn't expose tracks even in Window

    jer: Generally, I prefer MSE in worker to be as close to on
    main thread as possible. To facilitate easier adoption
    … Probably not a big issue for HTML to expose tracks in Workers

     Threading lifetimes (#277)

    wolenetz: [16]#277 what happens to <video> when worker is
    … do we remove the buffers from <video> that MSE-in-worker was
    … or do we let <video> retain what was buffered
    … also, should <video> get some notification that worker
    termination occurred
    … currently, chrome's mse-in-workers drops buffers after worker
    dies, but not sure if it's right

      [16] https://github.com/w3c/media-source/issues/277

    jernoble: I would behave as if source object is removed from
    <video>. return to empty network state

    wolenetz: or potentially trigger EOS behavior?

    jernoble: that sounds more complicated. if source object
    disappears, most transparent thing is to remove the buffers
    … could be detected by network state event
    … similar to how <video> behaves w/ src set to empty string

    cyril: can things be downloaded, buffered, then wasted if
    worker dies?

    wolenetz: yes that could occur, but generally we expect the app
    to be driving the termination, so just a question about the
    short period of time following that

    cyril: fine to remove buffers if it's just for a short period.

    jernoble: "terminating" the worker would be caused by explicit
    terminate(), which in theory is only done by the client of the

    cyril: can the app wait to terminate until it's used all the

    wolenetz: yes

    cpn: what do we need to spec from interop pov

    wolenetz: I like Jer's suggestion to detach as much as

    cpn: can you reuse media element by setting a new src /

    wolenetz: yes

     Timing of initial HAVE_METADATA transition (#275)

    wolenetz: Regarding [17]#275, currently, HAVE_METADATA is
    spec'ed to be fired synchronously w/ arrival of init segment
    … not feasible with workers. any concerns to relax it to be
    … specifically, bottom of init segment received algorithm: when
    all SB's have init segment received = true
    … set's readyState = HAVE_METADATA

      [17] https://github.com/w3c/media-source/issues/275

    jernoble: I think the time where init segment is processed is
    probably already arbitrary

    wolenetz: so it's probably ok to do async here as well? not
    blocking the worker on synchronously setting readyState?

    jernoble: I agree

    janivar: might be good to gather data on current browser

    wolenetz: will do

    jernoble: do we have a wpt test for this? could use that to
    track differences

    wolenetz: test exists, but it's flaky

    jernoble: for webkit, we issue have_metadata before updated

     Object URLs vs srcObject (#156)

    wolenetz: Regarding [18]#156, chromium regrettably doesn't
    support srcObject yet
    … the original object URL was "autorevoking"
    … Chrome changed behavior to let apps explicitly revoke to
    improve GC ability to collect
    … for MSE-in-workers, we need to keep object URLs alive long
    enough to be used by worker (can't immediately revoke).
    … proposal: for workers, what if we don't revoke until after
    first use.

      [18] https://github.com/w3c/media-source/issues/156

    jernoble: I initially favored keeping the URL, but now I like
    srcObject. Still, usage of srcObject is very low because of
    missing support in other UAs

    wolenetz: we haven't decided how src would work w/
    MSE-in-workers yet. so far just looked at using srcObject

    jernoble: for src url, we could maintain autorevoking behavior
    on the main thread while behaving differently for use in

    <cpn> [19]https://github.com/w3c/media-source/issues/28

      [19] https://github.com/w3c/media-source/issues/28

    janivar: I'm supportive of srcObject. Firefox hasn't
    implemented it yet.

    jernoble: it seems simpler to create object in worker, do
    attachment on main thread, and have spec language say <video>
    informs the MSE-in-worker that they're now attached
    … vs earlier proposals that considered creating src url in main

    janivar: beware, as you "transfer" objects, the typical
    language is that the source-side is "detached"

    wolenetz: Up next I'll be addressed Mark Watsons PR comments
    and taking a look at Spectre mitigations

    cpn: to confirm, for workers, we've aligned on srcObject for

    wolenetz: correct

    jernoble: something to consider, should <source> elements have
    srcObject attribute?

   Next meeting

    cpn: scheduled for end of october. do we need anything sooner?

    cyril: I'd like an earlier session to discuss bytestream issues

    jer: how about same time in two weeks?

    cyril: in the meantime, please add your thoughts to the GH

Received on Thursday, 16 September 2021 19:02:24 UTC