Re: Modules split

On Mon, Aug 5, 2019 at 5:56 PM Waliczek, Nell <nhw@amazon.com> wrote:

> Hey folks,
>
>
>
> There is an agenda item for tomorrow’s call to discuss this in more
> detail, but I’ll take a stab now at addressing some of the concerns
> outlined in this thread.  Hopefully that will enable tomorrow’s discussion
> to be a productive one.
>
>
>
> The idea to break the specification into modules was initially discussed
> at the face to face in early June
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.w3.org_2019_06_05-2Dimmersive-2Dweb-2Dminutes.html-23item10&d=DwMGaQ&c=0ia8zh_eZtQM1JEjWgVLZg&r=jahSgznxrAL5kPgsRvs7bhKUEd9M5X0d-NE2WJg7VT0&m=8MIY46cPkjOCkz6cJ6m5R_8DJMclWdn0w1qbPVNQaYQ&s=8eVuJcj5T5AdsOFLTUmRNYC4ybD8z5jW3tC3Zu5gri0&e=>.
> The plan for what would be pulled into separate modules was posted here
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_immersive-2Dweb_administrivia_blob_master_modules.md&d=DwMGaQ&c=0ia8zh_eZtQM1JEjWgVLZg&r=jahSgznxrAL5kPgsRvs7bhKUEd9M5X0d-NE2WJg7VT0&m=8MIY46cPkjOCkz6cJ6m5R_8DJMclWdn0w1qbPVNQaYQ&s=m9cG66N3axVrtmoGNKfPJPq0BPOqnqzvMnJ23eP5cjU&e=>
> and subsequently discussed on several different calls.  Notes are here
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_immersive-2Dweb_administrivia_blob_master_meetings_wg_2019-2D06-2D11-2DImmersive-5FWeb-5FWorking-5FGroup-5FTeleconference-2Dagenda.md&d=DwMGaQ&c=0ia8zh_eZtQM1JEjWgVLZg&r=jahSgznxrAL5kPgsRvs7bhKUEd9M5X0d-NE2WJg7VT0&m=8MIY46cPkjOCkz6cJ6m5R_8DJMclWdn0w1qbPVNQaYQ&s=MTooKt2cmQYpYWE1c4h8vMnyKXyxjijVl9-vdFYXoZA&e=>
> , here
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_immersive-2Dweb_administrivia_blob_master_meetings_wg_2019-2D06-2D18-2DImmersive-5FWeb-5FWorking-5FGroup-5FTeleconference-2Dagenda.md&d=DwMGaQ&c=0ia8zh_eZtQM1JEjWgVLZg&r=jahSgznxrAL5kPgsRvs7bhKUEd9M5X0d-NE2WJg7VT0&m=8MIY46cPkjOCkz6cJ6m5R_8DJMclWdn0w1qbPVNQaYQ&s=vdNK1T356j8EnmRQmupmlPOJs2cXIacI6suVcVPlECg&e=>,
> and here
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_immersive-2Dweb_administrivia_blob_master_meetings_wg_2019-2D07-2D30-2DImmersive-5FWeb-5FWorking-5FGroup-5FTeleconference-2Dagenda.md&d=DwMGaQ&c=0ia8zh_eZtQM1JEjWgVLZg&r=jahSgznxrAL5kPgsRvs7bhKUEd9M5X0d-NE2WJg7VT0&m=8MIY46cPkjOCkz6cJ6m5R_8DJMclWdn0w1qbPVNQaYQ&s=vYBF1HeT1rYCMThbFQSk8SrPSMuKdWVYQlJSWtadwOQ&e=>.
> At the time, we did not believe we would converge on the AR-specific
> privacy/security issues or a concrete scope for AR by the end of June.  As
> such, we decided to remove AR references from the WebXR Core spec and
> address actual scope the work to be covered in the AR module once we’d
> reached VR complete. This is where we are today.
>

Thank you Nell!
I apologize for not noticing that modularization was going to remove the
'immersive-ar' mode.
Since it was in the spec for so long, implemented and not marked as "not
stable", I didn't think it was going to be removed but it is totally my
fault for bringing it up this late.


> It’s probably also worth noting that, in addition to the AR module, we
> have also created a Gamepads module.  This module is what allows developers
> to access button, trigger, thumbstick, and touchpad data.  It would be a
> significant problem for most VR form factors (and some AR ones) if this
> functionality was not available in browsers when the WebXR core “ships”.
> However, the expectation is that this module will be completed and
> published at the same time as the WebXR core.  Which raises the question,
> “why bother pulling this out of the core?”  As the Gamepad spec evolves, we
> may choose to design a different module alongside it.  This module was
> separated to enable flexibility going forward and to decouple delivering
> new Gamepad-style integrations or features. Essentially, why should future
> Gamepad-related features be held up by AR or by multilayer rendering, etc.
>
>
>
> That same philosophy is why we chose to split AR out of the WebXR core.
> The VR-specific features are at a point where they are nearing (but not
> fully) ready for a CR to be published.  Alternatively, there are a number
> of key issues at the heart of enabling AR experiences that remain
> unresolved.  The group discussed, and agreed, that for AR to have
> viability, we must have addressed privacy/security concerns and be
> confident it will work effectively across a range of hardware form
> factors.  The questions of blend-mode and the user perception of camera
> access are two such issues.  Another is how to address the different
> interaction needs of a hand-held vs. head-worn device.  And so on.  At the
> time we decided to do the module split, there were also differences of
> opinion on whether additional features were necessary for a “baseline” AR
> module.  Some of these contentious features include hit testing, plane
> detection, environment meshes, anchors, aligned-camera access, and more.
>
>
>
> With the module split now completed, it is time to make decisions around
> the timetables and associated scope for the AR module(s).  The kickoff for
> making this decision is scheduled for tomorrow’s CG call, though I’m
> certain we’ll need to discuss a few more times before reaching a final
> consensus.  I would imagine a number of folks share my eagerness to get
> this plan resolved as soon as possible, and I’m hoping we can identify a
> small core set of AR functionality that we can collectively focus on
> completing rapidly while allowing for further incubation in less well
> defined areas. Honestly, I’d love to see a First Published Working Draft by
> the end of this year which enables the core of AR functionality.  And as
> the less defined features become more defined, we can then decide if they
> become their own modules, new levels of the same module, perhaps something
> else entirely.
>
>
>
> So, until we have these next steps better defined, the recommendation
> during our last discussion was to ship ‘immersive-ar’ behind a feature flag
> so that misinterpretations of “done-ness” are less likely.  The good news
> is there is fair amount of flexibility in how User Agents can do this.  For
> example, Samsung Internet Browser prompted users to enable the WebVR flag
> when they first visited a site with VR content.  Another option might be
> prompting the user to enable the flag the first time a browser is launched
> on a standalone device.
>
>
>
> Which actually brings me to another point about User Agent choice.  Based
> on prior conversations, I’m definitely surprised to hear that Magic Leap
> isn’t interested in allowing ‘immersive-vr’ sessions on their hardware.  I
> absolutely understand that there are drawbacks to displaying VR-optimized
> content on an additive light display. I think I’d just assumed that Helio
> would support the mode, as Microsoft had expressed the intention to do.
>

It would be great if someone from Microsoft could chime in. Maybe they are
not concerned about presenting a full frame to the user?


> *Given that is not the case, perhaps the simplest solution is to add the
> XREnvironmentBlendMode enum’s ‘additive’ value to those allowed in the
> ‘optionalFeatures’ array?  This would allow Helio to decline any
> ‘immersive-vr’ session for which ‘additive’ isn’t requested.*
>

I don't quite follow. If it's an optional feature, why would we deny the
session?
Simplest is to define 'immersive-ar' as content that is designed to be
presented in the real world (as opposed to 'immersive-vr which is supposed
to draw the entire world) and leave it at that.
Additional features (meshing, planes, gestures, etc) can be done as modules
and might also apply to VR sessions.


> If this approach generally seems like it might work, I’m happy to file an
> issue in which we can nail down the details.  (Also, as you all accurately
> pointed out, I totally forgot to remove the ‘alpha-blend’ value from the
> core and move it to the AR module.  So that will also need to happen.)
>

I think we should completely remove the environmentBlendMode from the spec
(or mark it as "at risk").
It's only useful for AR content and it's unclear what values it should have
or what an author should do with it. I can file this as an issue if you
want.


> Ultimately, the purpose of this module split is to allow us to move faster
> on the things we understand well, and allow us enough time to iterate on
> the things we understand less well.  I firmly believe that this approach
> will bring both AR and VR into the hands of developers and end-users in the
> fastest way possible.  And y’all know I’m all about that.
>

Yes, we want that too!
However we don't want our users to have a bad experience by allowing
regular VR experiences and we don't want to tell our developers to use a
non-standard option.


> Links (in case they got stripped out)
>
> https://www.w3.org/2019/06/05-immersive-web-minutes.html#item10
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.w3.org_2019_06_05-2Dimmersive-2Dweb-2Dminutes.html-23item10&d=DwMGaQ&c=0ia8zh_eZtQM1JEjWgVLZg&r=jahSgznxrAL5kPgsRvs7bhKUEd9M5X0d-NE2WJg7VT0&m=8MIY46cPkjOCkz6cJ6m5R_8DJMclWdn0w1qbPVNQaYQ&s=8eVuJcj5T5AdsOFLTUmRNYC4ybD8z5jW3tC3Zu5gri0&e=>
>
> https://github.com/immersive-web/administrivia/blob/master/modules.md
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_immersive-2Dweb_administrivia_blob_master_modules.md&d=DwMGaQ&c=0ia8zh_eZtQM1JEjWgVLZg&r=jahSgznxrAL5kPgsRvs7bhKUEd9M5X0d-NE2WJg7VT0&m=8MIY46cPkjOCkz6cJ6m5R_8DJMclWdn0w1qbPVNQaYQ&s=m9cG66N3axVrtmoGNKfPJPq0BPOqnqzvMnJ23eP5cjU&e=>
>
>
> https://github.com/immersive-web/administrivia/blob/master/meetings/wg/2019-06-11-Immersive_Web_Working_Group_Teleconference-agenda.md
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_immersive-2Dweb_administrivia_blob_master_meetings_wg_2019-2D06-2D11-2DImmersive-5FWeb-5FWorking-5FGroup-5FTeleconference-2Dagenda.md&d=DwMGaQ&c=0ia8zh_eZtQM1JEjWgVLZg&r=jahSgznxrAL5kPgsRvs7bhKUEd9M5X0d-NE2WJg7VT0&m=8MIY46cPkjOCkz6cJ6m5R_8DJMclWdn0w1qbPVNQaYQ&s=MTooKt2cmQYpYWE1c4h8vMnyKXyxjijVl9-vdFYXoZA&e=>
>
>
> https://github.com/immersive-web/administrivia/blob/master/meetings/wg/2019-06-18-Immersive_Web_Working_Group_Teleconference-agenda.md
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_immersive-2Dweb_administrivia_blob_master_meetings_wg_2019-2D06-2D18-2DImmersive-5FWeb-5FWorking-5FGroup-5FTeleconference-2Dagenda.md&d=DwMGaQ&c=0ia8zh_eZtQM1JEjWgVLZg&r=jahSgznxrAL5kPgsRvs7bhKUEd9M5X0d-NE2WJg7VT0&m=8MIY46cPkjOCkz6cJ6m5R_8DJMclWdn0w1qbPVNQaYQ&s=vdNK1T356j8EnmRQmupmlPOJs2cXIacI6suVcVPlECg&e=>
>
>
> https://github.com/immersive-web/administrivia/blob/master/meetings/wg/2019-07-30-Immersive_Web_Working_Group_Teleconference-agenda.md
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_immersive-2Dweb_administrivia_blob_master_meetings_wg_2019-2D07-2D30-2DImmersive-5FWeb-5FWorking-5FGroup-5FTeleconference-2Dagenda.md&d=DwMGaQ&c=0ia8zh_eZtQM1JEjWgVLZg&r=jahSgznxrAL5kPgsRvs7bhKUEd9M5X0d-NE2WJg7VT0&m=8MIY46cPkjOCkz6cJ6m5R_8DJMclWdn0w1qbPVNQaYQ&s=vYBF1HeT1rYCMThbFQSk8SrPSMuKdWVYQlJSWtadwOQ&e=>
>
>
>
> *From: *Klaus Weidner <klausw@google.com>
> *Date: *Monday, August 5, 2019 at 2:41 PM
> *To: *Rik Cabanier <rcabanier@magicleap.com>
> *Cc: *Brandon Jones <bajones@google.com>, "Waliczek, Nell" <nhw@amazon.com>,
> "public-immersive-web-wg@w3.org" <public-immersive-web-wg@w3.org>
> *Subject: *Re: Modules split
> *Resent-From: *<public-immersive-web-wg@w3.org>
> *Resent-Date: *Monday, August 5, 2019 at 2:39 PM
>
>
>
> FYI, I filed https://github.com/immersive-web/webxr/issues/788
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_immersive-2Dweb_webxr_issues_788&d=DwMGaQ&c=0ia8zh_eZtQM1JEjWgVLZg&r=jahSgznxrAL5kPgsRvs7bhKUEd9M5X0d-NE2WJg7VT0&m=8MIY46cPkjOCkz6cJ6m5R_8DJMclWdn0w1qbPVNQaYQ&s=lJNMKxAE_ztIJnj2CbUcLMgVrjEIawbMVDIuBpZlfUc&e=>
> "Clarify `environmentBlendMode` semantics" which includes some of the
> discussions in this thread.
>
>
>
> On Mon, Aug 5, 2019 at 1:17 PM Klaus Weidner <klausw@google.com> wrote:
>
> On Mon, Aug 5, 2019 at 10:55 AM Rik Cabanier <rcabanier@magicleap.com>
> wrote:
>
> On Mon, Aug 5, 2019 at 9:40 AM Brandon Jones <bajones@google.com> wrote:
>
> I'm working on a more complete response, but I'm taking some time to make
> sure that I'm not innacurately representing anyone's position or offering
> any conflicting messages. But for the moment I want to make sure we're
> clear on the use and purpose of *environmentBlendMode:*
>
>
>
> We heard from the HoloLens team that they wanted to have the ability to
> display what was then the equivalent of *"immersive-vr"* content, but
> provide developers a hint to determine how it would actually be shown in
> order to allow them to either tweak the content for that scenario or
> possibly optimize it by avoiding rendering certain things that wouldn't
> display properly anyway (ie: certain shadow effects.) Thus by allowing
> *"immersive-vr"* sessions to specify an *"additive"*
> *environmentBlendMode* headsets like HoloLens or Magic Leap are allowed
> to advertise support for VR content if they so choose. It's also perfectly
> valid choice for a given platform to choose not to do that if they don't
> feel the user experience will be a positive one, as you indicated.
>
>
>
> It would be good to hear from Microsoft why they wanted this (and not just
> use 'immersive-ar').
>
> Maybe we can continue this on github? I filed an issue there.
>
>
>
> FWIW I think *environmentBlendMode *should only be meaningful in an
> 'immersive-ar' session. https://www.w3.org/TR/webxr/#xrsessionmode-enum
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.w3.org_TR_webxr_-23xrsessionmode-2Denum&d=DwMGaQ&c=0ia8zh_eZtQM1JEjWgVLZg&r=jahSgznxrAL5kPgsRvs7bhKUEd9M5X0d-NE2WJg7VT0&m=8MIY46cPkjOCkz6cJ6m5R_8DJMclWdn0w1qbPVNQaYQ&s=y2acvF0xhBtpc8Co3Gupfu-IiUI7YW0eGfA7_DOsqus&e=> already
> implies this
>
>
>
> Just to avoid confusion, please keep in mind that the environmentBlendMode
> is a read-only property that's set by the device depending on hardware
> properties and the requested session type. It is NOT a developer choice
> that's selected by an application.
>
> There's no way for an application to request "immersive-vr with additive
> blend mode". The application just asks for immersive-vr, and the UA
> either picks the most-opaque blend mode that it supports (reporting this
> back to the application through the blend mode property), or rejects the
> session if it does not support immersive-vr.
>
>
>
>