[Minutes] Media Working Group Teleconference - 2020-08-11

Hi all,

The minutes of last week's Media Working Group call are available at:
https://www.w3.org/2020/08/11-mediawg-minutes.html

... and copied as raw text below.

Two resolutions taken regarding the possible integration of the 
Picture-in-Picture API within the HTML spec:
1. Keep the Picture-in-Picture specification as-is as it doesn't extend 
the HTML specification apart from partial interface
2. write best practices with regards to media specifications and HTML 
extension

Thanks,
Francois.
-----

Media WG
11 Aug 2020

    [2]Agenda

       [2] 
https://github.com/w3c/media-wg/blob/master/meetings/2020-08-11-Media_Working_Group_Teleconference-agenda.md

Attendees

    Present
           Nigel Megitt, Mounir Lamouri, ChrisNeedham, Jer Noble,
           Andreas Tai, Francois Beaufort, Eric Carlson, Gary
           Katsevman, Greg Freedman, Joey Parrish, Peng Liu,
           Xiaohan Wang, Francois Daoust (first half)

    Regrets

    Chair
           Jer, Mounir

    Scribe
           Jer, Mounir

Contents

      * [3]Topics
          1. [4]Picture-in-Picture API and integration with HTML
          2. [5]EME: handling hardware context reset
          3. [6]TPAC 2020
      * [7]Summary of Resolutions
      __________________________________________________________

Picture-in-Picture API and integration with HTML

    <mounir>
    [8]https://github.com/w3c/picture-in-picture/issues/156

       [8] https://github.com/w3c/picture-in-picture/issues/156

    mounir: Agenda Item 1: PiP API and the HTML WG
    ... We received feedback that the PiP API should move to the
    HTML WG
    ... three options: 1. We move the entire API to the Media
    Element part of the HTML spec
    ... 2. we only move the events to the HTML spec
    ... 3. We do nothing, and leave everything in the existing spec
    ... This is a similar situation to the video quality
    specification, which is small, and we decided to attempt to
    integrate it into the HTML specification.

    chris: What's the technical argument for moving this into HTML?

    mounir: It will avoid fragmentation and "monkey patching" of
    the HTML spec

    cpn: I see that there is some "monkey patching" in PIP. But it
    seems that it should remain in one place wherever it lands.
    ... Leaving it as a standalone document might be the easiest
    for long term maintenance.

    Francois: I don't think we're inserting anything into any
    existing HTML algorithm.

    nigel: If we think that _all_ HTMLVideoElements need to support
    PiP then there might be an argument to insert just the portions
    that extend elements defined in HTML
    ... the general behavior of PiP makes sense to leave in a
    second spec

    mounir: Yes, this is the option 2. The downside is that moving
    the partial .idl into HTML you'll need to move definitions as
    well
    ... That would mean splitting the spec across two different
    documents
    ... moving media playback quality was a simple operation

    nigel: Do we believe that all HTMLVideoElements are expected to
    support PiP?

    Eric: There will definitely be platforms where pip won't be
    possible due to platform limitations

    nigel: In that case we should retain it as an extension in a
    separate document.

    gkatsev: The fullscreen API lives in a separate specification
    and the PiP shares a lot of behavior with Fullscreen.

    jernoble: should there be extension points for HTML to make
    these easier so we don't have to revisit these questions in the
    future?

    mounir: the use of "partial interface" is the only part where
    we extend HTML

    cpn: What would an "extension point" look like in this case?

    jernoble: hanging a single object off of the HTMLMediaElement
    that contain all the existing elements

    xiaohan: That also works in EME, where a single "mediakeys"
    object hangs off the media element, but everything is accessed
    through new objects

    mounir: "Monkey patching" is really where steps are added into
    existing algorithms, rather that just adding new events or
    properties off existing objects
    ... specs will add language to algorithms, such as: "If you're
    going to different things in this algorithm, do it at _this_
    point"
    ... For example, some specifications will say "when you enter
    or leave fullscreen", which is an operation defined in HTML

    <Zakim> gkatsev, you wanted to ask how does the fullscreen api
    spec differ?

    jernoble_: we should define best practices
    ... ... like creating objects instead of adding multiple
    properties to the media elements

    <mounir> PROPOSED RESOLUTION: Keep the Picture-in-Picture
    specification as-is as it doesn't extend the HTML specification
    apart from partial interface

    <mounir> PROPSED RESOLUTION: write best practices with regards
    to media specifications and HTML extension

    <mounir> PROPOSED RESOLUTION: write best practices with regards
    to media specifications and HTML extension

    <cpn> +1 to both of these

    <nigel> +1

    <jernoble_> +1 to both

    RESOLUTION: Keep the Picture-in-Picture specification as-is as
    it doesn't extend the HTML specification apart from partial
    interface

    RESOLUTION: write best practices with regards to media
    specifications and HTML extension

EME: handling hardware context reset

    [9]https://github.com/w3c/encrypted-media/issues/473

       [9] https://github.com/w3c/encrypted-media/issues/473

    Xiaohan: Implementations are seeing issues where the CDM resets
    when, e.g., a device sleeps
    ... The current EME specification has a note where the CDM can
    close itself under certain circumstances
    ... But this doesn't seem to be a well-known behavior, and
    isn't handled well in JS
    ... On Windows, the media engine will return a special HRESULT
    signaling an error, which are elevated to JS, and then the
    player will attempt to resume playback, or will recreate all
    the necessary sessions.
    ... But this behavior is outside the standard. Should we
    document this behavior to the standard, so that we get better
    cross-browser interop?
    ... On the MediaKeySession.closed attribute is a Promise<void>.
    We do not know why the session is closed.
    ... I propose adding an enum here to define why the session is
    closed.
    ... On HTMLMediaElement, there's a "waitingforkey" event, which
    was added for EME. I propose when we have a closed session,
    instead of erroring out, we return to the "waitingforkey"
    state, so that we can restart playback.
    ... Or the player can restart playback entirely including
    creating a new element, etc.

    jernoble_: I don't remember ever coming across a situation
    where the CDM, in this case fairplay, would close itself due to
    a sleep wake state
    ... maybe I'm not careful enough or didn't notice
    ... there are a lot of reasons why a UA would [???] in the
    first place
    ... so it makes sense that something would cause the session to
    close outside of the JS requesting the session to close
    ... I understand it's an issue and has to be resolved
    ... though I'm curious why the close() happens and why it has
    to be a promise instead of an event

    GregFreedman: my understanding was, to respond to jer, that
    there isn't any sort of events to indicate a problem at a
    session level
    ... key status can report error on the key status but for some
    reasons there is no events on the api on the session
    ... the close promise is pretty vague, it is pretty vague as
    there is a close method and a close attribute
    ... that is called when the session is closed by the page or
    the CDN
    ... we could either update the documentation to indicate that
    right now close is very vague and say that an error can also
    cause this
    ... the thing that Xiaohan is proposing is to add an enum to
    specify the reason

    Xiaohan: I'm in support to make this close more clear
    ... it's currently a note
    ... I agree that we should put that in the explanation of the
    attribute
    ... second, to Greg's question, the potential value to this
    "close reason" is to differentiate fatal errors that cannot be
    recovered and recoverable errors like contact hardware reset
    ... whether we need more detailed errors, I don't know but we
    need at least these two
    ... I don't really know why we have these two different ways of
    doing things

    Joey_Parrish: I wasn't involved in the original decision with
    regards to events vs promise
    ... the events may have been seen a more transient while the
    promise is more final
    ... we need a way to differentiate the close reasons
    ... at least Shaka Player would behave differently
    ... it would also be good to have an enum value to explicitely
    indicate that the application decided to close the session
    ... to differentiate from errors

    jernoble_: The web uses promises to indicate state that will
    only ever change once, and that matches, as Joey indicated, the
    behavior of 'close'.
    ... But when a session is unrecoverable, does that mean that a
    new session won't work?

    Xiaohan: "Unrecoverable" error implies that a new session will
    never work
    ... One example is a platform that has a limited number of
    available "slots" for sessions, and the session was closed
    because it was evicted by another application
    ... Another example is for software DRM, the session was closed
    because it detected being "hacked", and would refuse to create
    new sessions

    GregFreedman: We can currently tell whether the session was
    closed by the application, or whether it was closed by the CDM
    ... There's two reasons I can see: lack of resources, or loss
    of hardware
    ... how would a client application react differently to those
    two reasons? Would an application wait until a user action
    occurred to attempt to restart playback
    ... Those two reasons would result in the same behavior

    Xiaohan: The three states seem to be, closed due to application
    request, recoverable error, or unrecoverable/internal error
    ... Why do we need different enums for recoverable cases? It
    might be useful for logging or monitoring.
    ... If they're all aggregated into these three states, that
    would be fine

    GregFreedman: We can already differentiate between app-driven
    close and CDM-driven close. What would adding the enum values
    add to client behavior?

    Xiaohan: This might be useful for CDMs in the future.
    ... The "unrecoverable" error is a placeholder for future
    behavior

    Joey_Parrish: you can know that the application caused the
    close if you control the entire application, but for apps which
    compose libraries, the part of the app dealing with sessions
    may not know for sure that the close is due to the application;
    an enum state for that would still be useful.

TPAC 2020

    mounir: last agenda item: TPAC 2020
    ... We are trying to do cross-group meetings due to the virtual
    nature of this years TPAC
    ... We have gotten a request to do a cross-functional meeting
    about text tracks

    cpn: We may want to request one with Media & Entertainment WG,
    which is looking at interop considerations of resource
    constrained devices

    <Joey_Parrish> CTA WAVE

    jernoble_: the other group we may be asked to talk to is CTA
    Wave, especially around Media Capabilities
    ... they didn't mention TPAC yet and they are not member of
    W3C, but W3C is a member there

    cpn: there have been discussions about setting up a workshop

    mounir: <wrap up>

Summary of Resolutions

     1. [10]Keep the Picture-in-Picture specification as-is as it
        doesn't extend the HTML specification apart from partial
        interface
     2. [11]write best practices with regards to media
        specifications and HTML extension

Received on Thursday, 20 August 2020 09:28:20 UTC