F2F 2021 Minutes for Thu May 20

May 20Attendees

Philippe Milot, Jack Schaedler, Raymond Toy, Ruth John, Hongchan, Paul
Adenot, Christoph Guttandin, Michel Buffa, Matthew Paradis
Minutes

   -

   16:00-16:10 UTC (9:00-9:10 am PDT): Set up calls
   -

   16:10-18:00 UTC (9:10-11:00 am PDT): Priority-2
   <https://github.com/WebAudio/web-audio-api-v2/issues?q=is%3Aopen+is%3Aissue+label%3Apriority-2>
   and others
   <https://github.com/WebAudio/web-audio-api-v2/issues?q=is%3Aissue+is%3Aopen+-label%3Apriority-1+-label%3Apriority-2>
   -

      Issue 14: Real-time pitch adjustment
      <https://github.com/WebAudio/web-audio-api-v2/issues/14>
      -

         Raymond: Any try out the phaze demo?
         -

         Jack: I haven’t but tried a different one.
         -

         Paul: It’s alright, but not super high quality.
         -

         Jack: This makes sense to have it builtin because you have access
         to fast algorithms.
         -

         Paul: Media element has this builtin so browsers have it already.
         Media element has a preserve pitch attribute.  Seems to be
geared towards
         voice.
         -

         Hongchan: Soundtrap uses Autotune (licensed)
         -

         Paul: That’s cool.
         -

         Jack: We have 3-4 different algorithms.  People really like these,
         even if it’s not the most spiffy method.  It would be nice to
have some
         open source version.
         -

         Paul: Looked at phaze.  Pretty basic JS.  Takes 40% cpu on my mac.
         -

         Hongchan: SIMD would help?
         -

         Paul: Of course.
         -

         Paul: Pitch adjustment has been requested several times over the
         years to stretch ABSN.
         -

         Hongchan: Is processing node the right approach?
         -

         Paul: Yeah, you add it to the graph and it’s pitch-shifted..
         -

         Jack: I think this is higher priority than, say, hard-sync.
         -

         Paul: Let’s say it’s a node with an input that outputs a
         time-stretched output.
         -

         Hongchan: It’s pretty complicated this way.  It’s weird to have a
         separate time-stretch and preserve pitch node.
         -

         Paul: Could add an attribute for this in ABSN.
         -

         Jack: That would be a good start.
         -

         Hongchan: But that doesn’t work with streaming.
         -

         Paul: Does it need samples in the future?
         -

         Raymond: Would you just have a delay, like the compressor does
         today?
         -

         Paul:  Summary: Add attribute (preservePitch defaulting to false)
         to ABSN and add pitch-shifting node (as long as it doesn’t
change duration)?
         -

         Jack:
         -

            Here are the docs for the Elastique pitch/time shifter:
            https://licensing.zplane.de/uploads/SDK/ELASTIQUE-PRO/V3/manual/elastique_pro_v3_sdk_documentation.pdf
            -

            • int CElastiqueProV3If::GetFramesNeeded ()
            -

            Returns the required number of input samples for the upcoming
            processing block.
            -

            This function has to be called before each processing step to
            assure correct input
            -

            buffer sizes.
            -

            4.1.4.9 virtual int CElastiqueProV3If::GetMaxFramesNeeded ( )
            [pure
            -

            virtual]
            -

            returns the maximum number of frames needed based on the
            minimum combined factor
            -

            passed on CreateInstance
            -

         Jack: Some interesting points here re: buffering and latency if
         this were to be a pure audio/effect node
         -

         Jack: Memory resident update to ABSN would also be super duper
         useful.
         -

         [Paul updates issue]
         -

      Issue 41: Informing AudioWorklet if output is not connected?
      <https://github.com/WebAudio/web-audio-api-v2/issues/41>
      -

         Raymond: No comments since TPAC, so ok with just closing this.
         -

         Paul: Not high priority.
         -

         Hongchan, Jack: Agree
         -

         Raymond: Just lower priority.
         -

         Paul: Done.
         -

      Issue 16: Worker support for BaseAudioContext
      <https://github.com/WebAudio/web-audio-api-v2/issues/16>
      -

         Raymond: I see that we now do [Exposed=Window]
         -

         Paul: MediaStreams is discussing transferable streams, so we can
         have this in an AudioContext.  Most problems are gone.
         -

         Hongchan: What’s the utility?
         -

         Paul: A breakout box, and transferable media streams makes it work.
         -

         Hongchan: We should link to that in the issue.
         -

         Paul: I’ll find it and link it.
         -

         Jack: I'm still reading through #16. #16 is a very good issue to
         address.
         -

         Jack: I just want to signal boost :D
         -

         Paul: We’re going to do it most likely.
         -

         Jack: Excellent.
         -

         Raymond: I think Hongchan did an experiment.
         -

         Hongchan: Yeah, I did.  It didn’t work because we refer to
         document and such all over.  I stopped there.
         -

      Issue 19: Customizable windowing function for Analyser node
      <https://github.com/WebAudio/web-audio-api-v2/issues/19>
      -

         Raymond: So, 4 window types, but not clear if you can change it.
         -

         Raymond: I prefer setting it in the constructor and not being
         changeable.  Create a new node if you want a new window.
         -

         Ruth: Can't think of a use case where you would want to change
         either.
         -

         Raymond: Makes it really simple to describe and spec.
         -

         Paul: Any expectations on continuity if you change the window?
         -

         Paul: What happens if we change fftsize?
         -

         Raymond: Huh.  Don’t know
         -

         Paul: Ah, everything is reset if the size changes.
         -

         Raymond: Conclusion is to do as proposed and allow changing the
         window, which resets all state.
         -

      Issue 20: ConvolverNode is limited to two channels
      <https://github.com/WebAudio/web-audio-api-v2/issues/20>
      -

         Paul: Do we just say we want it?
         -

         Raymond: Complicated because of the 4-channel matrixing
         operation.  We need to specify how it works.
         -

         Paul: Interesting that Yotam uses 12 channels.
         -

         Raymond: Hongchan’s omnitone does something similar
         -

         Hongchan: 1st order ambisonics uses 8 channels which is split to 4
         stereo convolvers.
         -

         Raymond: I think we should do it, but need to figure out how to
         specify this.
         -

         Hongchan: We could add an option to specify this.
         -

         Raymond: Works for me; just need to find the right name.
         -

         [Raymond to update the issue]
         -

      Issue 21: loadHRTFDatabase for SpatialPanner
      <https://github.com/WebAudio/web-audio-api-v2/issues/21>
      -

         Raymond: Haven’t we discussed this already?
         -

         Hongchan: This is about allowing custom databases.
         -

         Raymond: Looks like it’s up to Matt. :-)  Want to make it work for
         the BBC at least.
         -

         Matthew: I’ll look into it.
         -

         Raymond: Ok.  We’ll wait for new info.
         -

      Issue 25: Configurable sample rate conversion quality
      <https://github.com/WebAudio/web-audio-api-v2/issues/25>
      -

         Paul: I stand by what I said last. But nothing more to contribute.
         -

         Raymond: What about WaveShaper?
         -

         Paul: What’s the benefit there?
         -

         Raymond: Already specced to linear interp for the curve.
         -

         Paul: Oversampling with linear doesn’t seem good.
         -

         Raymond: Yeah. Linear interp for 4x oversampling isn’t good.
         Maybe it's better to have more points.
         -

         Paul: We do sinc resample for ABSN.  Quite difficult to get right.
         -

         Raymond: CHrome does linear.  Pretty awful if rate is far from 1.
         -

         RAymond: I think it makes sense and we should do it, so all
         comments from TPAC are still true.
         -

         Paul. Yeah.
         -

         [Raymond to update issue]
         -

      Issue 11: Storing AudioBuffers in native sample bit depth
      <https://github.com/WebAudio/web-audio-api-v2/issues/11>
      -

         Paul: WebCodecs supports many bit depths.
         -

         Raymond: Do we really want 4-bit DPCM.  Complicates things a lot.
         -

         Paul: Makes sense not to have DPCM.
         -

         Paul: We then have new Audiobuffer(webcodec audio).  Transfers the
         data to the AudioBuffer.  The WebCodec buffer is detached so it has 0
         length.
         -

         Hongchan: Is this proposed?
         -

         Paul: It’s basically agreed on by WebCodec, but spec may not be
         updated yet.  It’s useful.
         -

         Hongchan: Is this priority-2?
         -

         Paul: We get a lot of requests
         -

         Hongchan: So priority-1?
         -

         Paul: I’m ok with that.
         -

         Raymond: Same, but think we want to make it more explicit so I can
         create an Audiobuffer with the desired depth.
         -

         [Discussion on what this means and how to do it]
         -

         [Raymond to update the issue]

Received on Thursday, 20 May 2021 18:48:47 UTC