Re: Modules split

Hey folks,

There is an agenda item for tomorrow’s call to discuss this in more detail, but I’ll take a stab now at addressing some of the concerns outlined in this thread.  Hopefully that will enable tomorrow’s discussion to be a productive one.

The idea to break the specification into modules was initially discussed at the face to face in early June<https://www.w3.org/2019/06/05-immersive-web-minutes.html#item10>. The plan for what would be pulled into separate modules was posted here<https://github.com/immersive-web/administrivia/blob/master/modules.md> and subsequently discussed on several different calls.  Notes are here<https://github.com/immersive-web/administrivia/blob/master/meetings/wg/2019-06-11-Immersive_Web_Working_Group_Teleconference-agenda.md>, here<https://github.com/immersive-web/administrivia/blob/master/meetings/wg/2019-06-18-Immersive_Web_Working_Group_Teleconference-agenda.md>, and here<https://github.com/immersive-web/administrivia/blob/master/meetings/wg/2019-07-30-Immersive_Web_Working_Group_Teleconference-agenda.md>.  At the time, we did not believe we would converge on the AR-specific privacy/security issues or a concrete scope for AR by the end of June.  As such, we decided to remove AR references from the WebXR Core spec and address actual scope the work to be covered in the AR module once we’d reached VR complete. This is where we are today.

It’s probably also worth noting that, in addition to the AR module, we have also created a Gamepads module.  This module is what allows developers to access button, trigger, thumbstick, and touchpad data.  It would be a significant problem for most VR form factors (and some AR ones) if this functionality was not available in browsers when the WebXR core “ships”.  However, the expectation is that this module will be completed and published at the same time as the WebXR core.  Which raises the question, “why bother pulling this out of the core?”  As the Gamepad spec evolves, we may choose to design a different module alongside it.  This module was separated to enable flexibility going forward and to decouple delivering new Gamepad-style integrations or features. Essentially, why should future Gamepad-related features be held up by AR or by multilayer rendering, etc.

That same philosophy is why we chose to split AR out of the WebXR core.  The VR-specific features are at a point where they are nearing (but not fully) ready for a CR to be published.  Alternatively, there are a number of key issues at the heart of enabling AR experiences that remain unresolved.  The group discussed, and agreed, that for AR to have viability, we must have addressed privacy/security concerns and be confident it will work effectively across a range of hardware form factors.  The questions of blend-mode and the user perception of camera access are two such issues.  Another is how to address the different interaction needs of a hand-held vs. head-worn device.  And so on.  At the time we decided to do the module split, there were also differences of opinion on whether additional features were necessary for a “baseline” AR module.  Some of these contentious features include hit testing, plane detection, environment meshes, anchors, aligned-camera access, and more.

With the module split now completed, it is time to make decisions around the timetables and associated scope for the AR module(s).  The kickoff for making this decision is scheduled for tomorrow’s CG call, though I’m certain we’ll need to discuss a few more times before reaching a final consensus.  I would imagine a number of folks share my eagerness to get this plan resolved as soon as possible, and I’m hoping we can identify a small core set of AR functionality that we can collectively focus on completing rapidly while allowing for further incubation in less well defined areas. Honestly, I’d love to see a First Published Working Draft by the end of this year which enables the core of AR functionality.  And as the less defined features become more defined, we can then decide if they become their own modules, new levels of the same module, perhaps something else entirely.

So, until we have these next steps better defined, the recommendation during our last discussion was to ship ‘immersive-ar’ behind a feature flag so that misinterpretations of “done-ness” are less likely.  The good news is there is fair amount of flexibility in how User Agents can do this.  For example, Samsung Internet Browser prompted users to enable the WebVR flag when they first visited a site with VR content.  Another option might be prompting the user to enable the flag the first time a browser is launched on a standalone device.

Which actually brings me to another point about User Agent choice.  Based on prior conversations, I’m definitely surprised to hear that Magic Leap isn’t interested in allowing ‘immersive-vr’ sessions on their hardware.  I absolutely understand that there are drawbacks to displaying VR-optimized content on an additive light display. I think I’d just assumed that Helio would support the mode, as Microsoft had expressed the intention to do.  Given that is not the case, perhaps the simplest solution is to add the XREnvironmentBlendMode enum’s ‘additive’ value to those allowed in the ‘optionalFeatures’ array?  This would allow Helio to decline any ‘immersive-vr’ session for which ‘additive’ isn’t requested.  If this approach generally seems like it might work, I’m happy to file an issue in which we can nail down the details.  (Also, as you all accurately pointed out, I totally forgot to remove the ‘alpha-blend’ value from the core and move it to the AR module.  So that will also need to happen.)

Ultimately, the purpose of this module split is to allow us to move faster on the things we understand well, and allow us enough time to iterate on the things we understand less well.  I firmly believe that this approach will bring both AR and VR into the hands of developers and end-users in the fastest way possible.  And y’all know I’m all about that.

Nell


Links (in case they got stripped out)
https://www.w3.org/2019/06/05-immersive-web-minutes.html#item10

https://github.com/immersive-web/administrivia/blob/master/modules.md

https://github.com/immersive-web/administrivia/blob/master/meetings/wg/2019-06-11-Immersive_Web_Working_Group_Teleconference-agenda.md

https://github.com/immersive-web/administrivia/blob/master/meetings/wg/2019-06-18-Immersive_Web_Working_Group_Teleconference-agenda.md

https://github.com/immersive-web/administrivia/blob/master/meetings/wg/2019-07-30-Immersive_Web_Working_Group_Teleconference-agenda.md


From: Klaus Weidner <klausw@google.com>
Date: Monday, August 5, 2019 at 2:41 PM
To: Rik Cabanier <rcabanier@magicleap.com>
Cc: Brandon Jones <bajones@google.com>, "Waliczek, Nell" <nhw@amazon.com>, "public-immersive-web-wg@w3.org" <public-immersive-web-wg@w3.org>
Subject: Re: Modules split
Resent-From: <public-immersive-web-wg@w3.org>
Resent-Date: Monday, August 5, 2019 at 2:39 PM

FYI, I filed https://github.com/immersive-web/webxr/issues/788 "Clarify `environmentBlendMode` semantics" which includes some of the discussions in this thread.

On Mon, Aug 5, 2019 at 1:17 PM Klaus Weidner <klausw@google.com<mailto:klausw@google.com>> wrote:
On Mon, Aug 5, 2019 at 10:55 AM Rik Cabanier <rcabanier@magicleap.com<mailto:rcabanier@magicleap.com>> wrote:
On Mon, Aug 5, 2019 at 9:40 AM Brandon Jones <bajones@google.com<mailto:bajones@google.com>> wrote:
I'm working on a more complete response, but I'm taking some time to make sure that I'm not innacurately representing anyone's position or offering any conflicting messages. But for the moment I want to make sure we're clear on the use and purpose of environmentBlendMode:

We heard from the HoloLens team that they wanted to have the ability to display what was then the equivalent of "immersive-vr" content, but provide developers a hint to determine how it would actually be shown in order to allow them to either tweak the content for that scenario or possibly optimize it by avoiding rendering certain things that wouldn't display properly anyway (ie: certain shadow effects.) Thus by allowing "immersive-vr" sessions to specify an "additive" environmentBlendMode headsets like HoloLens or Magic Leap are allowed to advertise support for VR content if they so choose. It's also perfectly valid choice for a given platform to choose not to do that if they don't feel the user experience will be a positive one, as you indicated.

It would be good to hear from Microsoft why they wanted this (and not just use 'immersive-ar').
Maybe we can continue this on github? I filed an issue there.

FWIW I think environmentBlendMode should only be meaningful in an 'immersive-ar' session. https://www.w3.org/TR/webxr/#xrsessionmode-enum already implies this

Just to avoid confusion, please keep in mind that the environmentBlendMode is a read-only property that's set by the device depending on hardware properties and the requested session type. It is NOT a developer choice that's selected by an application.

There's no way for an application to request "immersive-vr with additive blend mode". The application just asks for immersive-vr, and the UA either picks the most-opaque blend mode that it supports (reporting this back to the application through the blend mode property), or rejects the session if it does not support immersive-vr.