Re: [csswg-drafts] [mediaqueries-5] Introduce a media feature to detect an immersive or virtual environment

I made some comments over on the PR to answer some of these questions.  (@astearns I'm part of the WebXR community, so while I don't speak for them, I'm coming from that viewpoint).

I like this media query. There are two kinds of things we need in the web to deal with XR devices:

- A way to know what kind of environment the page is being _presented_ in
- A way to _present 3D content_ from a "web page" (web app) in XR (AR, VR, etc)

The WebXR API is concerned with the later.  Regardless of how the page itself is presented (e.g., on a 2D desktop, on a computer with a Vive/WindowsMR/Rift attached;  in 3D in VR in WindowsMR's "Cliff House" home space;  in 3D in AR on Hololens or Magic Leap, etc), WebXR is about telling the web page what displays are available and allowing them to draw 3D graphics with WebGL on that display in a "full 3D immersive" mode.   

This media query, and (I think the motivation behind it) is concerned with the former:  what is the sort of display environment in which we are rendering a web page (HTML content). 

In the case of a display attached to a computer (e.g., the Vive), the web page is still 2D on a traditional display, but the VR display used by WebXR is 3D VR.  So this media query would return something indicating a traditional 2D display.

I mention this because things like stereoscopic seem outside the scope of this, without much more dramatic changes to HTML/CSS:  there is no way to cause the page to appear differently to the 2 eyes from the _viewpoint of the web developer_.  That is something the browser itself would deal with (e.g., you could imagine the text or images being rendered slightly differently, not just in slightly different locations, to give the best combined appearance).  

What the dev cares about, as highlighted by the original post, is much more mundane:  what is the sort of display environment?

- traditional:  desktops, phones, tablets.  Content is being rendered on a flat screen, with the pixels being what you request   
- VR: opaque 3D displays.  The web page is rendered in a "rectangle" in a 3D world.  The rectangle is opaque, and it's very likely that notions like pixels will no longer map to pixels on a display.   So, while such a page may have a virtual size in CSS pixels, the relationship to actual rendered pixels is difficult (and may change continuously as the user moves relative to the page).  The "place in the 3D world" of the page is probably not available to the developer (since it's controlled by the user and browser).  Knowing that it's being presented in 3D could cause the developer to change the content presented, and knowing that text entry might be hard or that scrolling might be controlled by more awkward inputs than a mouse might change how they present.
- mixed/AR:  video-mixed or optically transparent displays that render a web page in 3D like VR, but where the world around the page is uncontrolled, and where the user may be moving and need to see things in the real world.  for video-mixed (graphics overlaid on video in the 3D renderer by the browser), the page itself is still opaque, but may want to do things based on context (same as VR, plus perhaps reduce distracting high-motion content, or change its content when not in focus to distract less).   On optically-seethrough displays (like Magic Leap, Hololens, etc), the dev may also want to increase contrast because the "page" is not fully opaque, so the world may show through a bit.  I have done things in the past where I used CSS shadows on text to put a faint white shadow up+left and a faint black shadow down+right, so that the text is legible on both light and dark backgrounds.

This media query is "the most basic" one and needed as a starting point to handle the big-picture adaptations across these 4 sorts of different environments:  traditional, VR, video-mixed, optically-mixed.  

Beyond that, there are other things we might want to add, but they start to tell pages about the world, and have privacy implications:

- a measure of the brightness of the room (so I can decide how bright/contrasty to make the content)
- a measure of the display-angle of the page relative to the viewer.  Am I looking at it head-on or from an angle.   My student Gheric Spieginer (@speigg) explored this in an environment where we could set the background of the page transparent and then control the style and content of elements based on simple queries like aspect ratios, etc.  See this video, where the content of DOM elements is changed based on angles, how big they are on the screen, etc https://www.dropbox.com/s/ecppit80o71u49p/video.mov?dl=0

These both feel beyond the scope of this particular query, and it's less clear how they would work.  

There are also ideas around "popping 3D objects out of the page into the world" (e.g., that the Meta CEO showed demos of in TED talks, for example).  These would require more substantial changes to CSS or new HTML elements.  But, you could imagine wanting to change tagging automatically based on a query (e.g., if you had a DOM element that somehow could pop out into the 3D world, you might want to enable it in VR/Mixed but not on a traditional 2D display).

-- 
GitHub Notification of comment by blairmacintyre
Please view or discuss this issue at https://github.com/w3c/csswg-drafts/issues/2723#issuecomment-393898809 using your GitHub account

Received on Friday, 1 June 2018 14:31:50 UTC