Re: January 2019 F2F Summary

Sure thing!

On Tue, 26 Feb 2019 at 23:00, Waliczek, Nell <nhw@amazon.com> wrote:

> Thanks for this super thorough write-up, Ada!
>
>
>
> And a heads up so folks don’t miss it… the design I proposed at the F2F
> for gamepad mappings is now up in its own repo and is ready for wider
> review as of **glances at watch** now 😊 There’s still a bunch of work to
> be done (as the repo’s issues list indicates), but please take a look and
> let me know any feedback you have on the general direction.
>
>
>
> https://github.com/immersive-web/xr-gamepad-mappings
>
>
>
> *Ada,* can we put this on the agenda for next week’s call?
>
>
>
> *From: *Ada Rose Cannon <ada@ada.is>
> *Date: *Tuesday, February 26, 2019 at 12:23 PM
> *To: *"public-immersive-web-wg@w3.org" <public-immersive-web-wg@w3.org>, "
> public-immersive-web@w3.org" <public-immersive-web@w3.org>
> *Subject: *January 2019 F2F Summary
> *Resent-From: *<public-immersive-web-wg@w3.org>
> *Resent-Date: *Tuesday, February 26, 2019 at 12:22 PM
>
>
> January 2019 F2F Follow Up
>
> This can be read on Github
> <https://github.com/immersive-web/administrivia/blob/master/newsletter/2919-01-28-Face-to-face-follow-up.md>.
> Feel free to share the URL.
>
> The aims of this are to summarise some of the discussions and resolutions
> from the face to face in January in California. Firstly I would like to
> thank Samsung Research America for hosting the event.
>
> This was largely put together from the minutes:
>
> https://www.w3.org/2019/01/29-immersive-web-minutes.html
>
> https://www.w3.org/2019/01/30-immersive-web-minutes.html
> Input and Controllers #336
> <https://github.com/immersive-web/webxr/issues/336>, #392
> <https://github.com/immersive-web/webxr/isues/392> & Pull #462
> <https://github.com/immersive-web/webxr/pull/462>
>
> There was a discussion about building a custom interfaces for accessing
> the XR controller. A gamepad-based solution (as described in #499) was also
> proposed and was overwhelmingly preferred.
>
> Nell is putting together a repository
> <https://github.com/immersive-web/xr-gamepad-mappings> of Schemas which
> can be maintained as an open source repo for mapping from the gamepad API
> to controllers.
> originOffset behaviour #477
> <https://github.com/immersive-web/webxr/issues/477>
>
> View the issue
> <https://github.com/immersive-web/webxr/issues/477#issuecomment-456565794>
> for more detail and wonderful handdrawn diagrams!
>
> *Interpretation A* thinks of the originOffset as applying to the
> *tracking* origin. * Interpretation B* thinks of the originOffset as
> applying to the *virtual scene's* origin.
>
> Either can be wholly mathematically consistent, and either can be made
> clear by the spec text, I'm just curious which approach the WG as a whole
> finds to be the most intuitive.
>
> After a vote at the face to face *Interpretation B* seemed to be more the
> popular and we will work with in the future.
> Hit Testing and Anchors (#491
> <https://github.com/immersive-web/webxr/pull/491>, #492
> <https://github.com/immersive-web/webxr/pull/492>, #493
> <https://github.com/immersive-web/webxr/pull/493>)
>
> The proposed changes are the introduction of the first real AR parts of
> the WebXR spec. They were met with overwhelming approval after the
> explanations. I have included the introduction from the hit testing
> explainer
> <https://github.com/immersive-web/webxr/blob/master/hit-testing-explainer.md>
> which I recommend reading:
>
> "Hit testing" (aka "raycasting") is the process of finding intersections
> between 3D geometry and a ray, comprised of an origin and direction.
> Conceptually, hit testing can be done against virtual 3D geometry or
> real-world 3D geometry. As WebXR does not have any knowledge of the
> developer's 3D scene graph, it does not provide APIs for virtual hit
> testing. It does, however, have information about the real-world and
> provides a method for developers to hit test against it. Most commonly in
> WebXR, developers will hit test using XRInputSources or the
> XRSession.viewerSpace to track where a cursor should be drawn on hand-held
> devices, or even to bounce a virtual object off real-world geometry. In
> WebXR, 'inline' and 'immersive-vr' sessions are limited to performing
> virtual hit tests, while 'immersive-ar' sessions can perform both virtual
> and real-world hit tests.
>
> Continue reading the hit testing explainer
> <https://github.com/immersive-web/webxr/blob/master/hit-testing-explainer.md>
> FoV / ViewMatrix (#461 <https://github.com/immersive-web/webxr/issues/461>
> /#447 <https://github.com/immersive-web/webxr/issues/447>) Expose
> combined frustrum (#203
> <https://github.com/immersive-web/webxr/issues/203>) Default FoV for
> Magic Window (#272 <https://github.com/immersive-web/webxr/issues/272>)
>
> The next section tackled issues relating to the field of view.
> Intersection of Request Animation Frame and Request Idle Callback
>
>    - Should we provide projection matrices or break them down for the
>    user?
>
> No clear conclusion. Further discussion required.
>
>    - What should the default field of view for magic window mode be?
>
> There isn't really a too bad answer settled on 90degrees vertical (PR #519
> <https://github.com/immersive-web/webxr/pull/519>)
> Handling Tracking Loss (#243
> <https://github.com/immersive-web/webxr/issues/243>)
>
> This began with a summary of Alex Turner's research into how different
> platforms handle losing position tracking:
>
> Handling tracking loss · Issue #243 · immersive-web/webxr
> <https://github.com/immersive-web/webxr/issues/243#issuecomment-454282109>
>
> There was discussion around sensible defaults a PR will be put together
> soon to put these into the spec.
> Handling Request Animation Frames for content of different framerates
> being displayed on displays of different framerates
>
> It's really worth reading the minutes
> <https://www.w3.org/2019/01/30-immersive-web-minutes.html#item04> for
> this topic. Essentially some video content will be slower than the headset
> framerate:
>
>    - Video 24/50/60Hz
>    - DOM content 60 Hz
>
> Which could be displayed on multiple devices at the same time:
>
>    - Headsets 90+ Hz
>    - Monitors/TV 60Hz
>
> Previously browsers largely ignored this and set the assumed it was always
> 60hz on browsers which could support it even on screens which run at higher
> frame rates.
>
> No resolutions were proposed during the session but it's an interesting
> issue to think about.
> Quadlayer - DOM content in XR
>
> Nell gives a great summary of this issue which you can read in the minutes
> <https://www.w3.org/2019/01/30-immersive-web-minutes.html#item05>
>
> The idea that you can create 2d uis in world space in html/css is one of
> the things that we hear frequently -- back at tpac we spent a bunch of time
> talking about the problem space. The traditional ways people use 2d APIs
> don't work if there's no way to draw them to the screen.
>
> This is solvable in the context of XR if the actual composition is done by
> the UA and not the user, this is the concept of a quad layer.
>
> The main benefit of quadlayer is that putting text content on a texture
> looks really blurry but a seperate layer can be able to do reprojection for
> the final pose without any per-pixel warping.
> Long lived XR Apps (proposals/#43
> <https://github.com/immersive-web/proposals/issues/43>)
>
> This was a discussion about the possibilities of running simultaneous XR
> apps which can hang around for a long time even in other experiences
> outside of the browser. For example a semi-permanent information widget
> positioned in a fixed place in your house in AR.
>
> Initial discussion here for a solution based on WebExtensions:
> https://github.com/immersive-web/proposals/issues/43
>
> Analysis on some of the threat vectors of XR:
>
> immersive-web/privacy-and-security
> <https://github.com/immersive-web/privacy-and-security/blob/master/EXPLAINER.md>
> Mirror views - Display WebXR on other sources
>
> Discussion on how important it is to get a video output suitable for flat
> displays from a WebXR scene? Many uses were raised:
>
>    - Debugging
>    - Streaming
>    - Local Audience
>    - Asymmetrical Experiences
>
> Expose a hidden area mesh or a hidden area depth write function (#463
> <https://github.com/immersive-web/webxr/issues/463>)
>
> A simple optimization that VR native applications are already taking
> advantage of is to inhibit fragment shading of regions of the display
> panels that, after warping and optics, are not visible to the user.
>
> OpenVR exposes such information as a mesh:
>
> https://github.com/ValveSoftware/openvr/wiki/IVRSystem::GetHiddenAreaMesh
>
> Essentially, such a mesh is rendered before the rest of the scene geometry
> using either depth buffer or stencil buffer writes to inhibit fragment
> processing of those pixels in subsequent render passes.
>
> This can provide significant performance improvements without relying on
> any special rendering API functions. This is also much easier to integrate
> into rendering engines relative to other optimizations (eg, multiview).
>
> Should probably just be fetched every frame
>