- From: Dylan Marcus via GitHub <sysbot+gh@w3.org>
- Date: Fri, 03 Dec 2021 16:44:02 +0000
- To: public-device-apis-log@w3.org
@anssiko Sure thing! > Would a Web API that represents head orientation data in quaternion or rotation matrix formats satisfy your requirements? If so, there's a path to extend https://w3c.github.io/orientation-sensor/ that means only small API surface changes, making the process faster. If you have to pick one or the other and to be very safe I would suggest picking quaternion, even though all our examples use rotations typically in degrees because they are much more human usable. If possible use both as we often have to supply developers with advice on how to manage going back and forth when only one is offered. We have learned ways to help explain how we describe rotation since it does differ from platform to platform which is a large pain point for this use case, we have typed out our experiences and opinion on this here: https://research.mach1.tech/posts/describing-3d-motion/ >What would be the "minimum viable" sampling frequency and sensor reading accuracy requirements that'd enable key spatial audio use cases? This group develops Web APIs that carefully mitigate any known privacy and security threats and I'm wondering whether the existing mitigations we have in place would work without compromising the use cases. This is a great question, when it comes to the use case of spatial audio it has to be very frequent. We have developed internal tests to poll when a listener can detect an "orientation delay" however we never finished this poll to make it public. The aim is to get to 200hz or greater but from our findings 100hz and even 50hz are sometimes acceptable for spatial audio usecases but often users can detect that there is a delay. Once a user detects orientation delay, the resulting spatial audio playback becomes highly compromised and ineffective. We have also found that while using devices/platforms that do not allow for steady sampling that just adding an additional smoothing filter can help; for example in this webexample we add a 1Euro filter to help smooth the orientation results and reduce jitters: https://github.com/Mach1Studios/m1-web-spatialaudioplayer#facetracking >What mainstream products that you know of currently support head orientation tracking and expose (preferably cross-)platform/OS-level APIs that could (at least in theory) be integrated into open-source web engines? We have a blog post trying to track public IMU enabled devices here: https://research.mach1.tech/posts/imu-enabled-devices/ However we also know there are many more headphone devices entering market with IMUs embedded on them suitable for headtracking use cases. The challenge is there is not a lot of SDK or interfacing support for developers yet, despite there being a demand for it from the spatial audio community. Right now beyond a few consumer products, most developers are utilizing 3rd party IMU senors and attaching them to any pair of headphones to continue development while anticipating this becoming much more ubiquitous. >Any other considerations we should be aware of. We have a ton of opinions on this topic from our own experiences in trying to create and develop for this use case over the last 5+ years, this has resulted in an [SDK for helping anyone aggregate spatial audio approaches ](https://github.com/Mach1Studios/m1-sdk)and help the community have more control over multichannel audio developments of any kind. We know from experience the current limitations to spatial audio and headtracking in depth, the biggest issues tend to lie in allowing one or two companies "define spatial audio" as a proprietary feature instead of releasing tools to let everyone access orientation from devices and handle multichannel audio in a more agnostic fashion; this is currently the biggest blocker to the medium. Our goal is to help each company that enters this discussion to see the common pain points so they can create open tools that benefit developers and creators alike. -- GitHub Notification of comment by himwho Please view or discuss this issue at https://github.com/w3c/orientation-sensor/issues/68#issuecomment-985667715 using your GitHub account -- Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config
Received on Friday, 3 December 2021 16:44:05 UTC