Re: Raw Camera Access module - quick update

> Is your intention to have the current Chrome raw-camera-access
implementation available without the feature flag?

Yes, that's the ultimate goal.

> I understand the easiest implementation to ship is the one that’s already
there, so fully understand if you don’t have the resource or time to commit
to any alternatives at the moment, but it would be good to understand the
level of interest.

I think it's unlikely that we'd be able to justify driving the efforts for
alternative APIs considering Chrome's current priorities in the AR / XR
space. :( My goal with the current proposal is twofold: a) release the API
that plenty of people are asking for, and b) don’t close the door for
future iterations that would expose this API on devices where the current
proposal falls short due to the platform differences.

With the above said, I’m personally not opposed to others incubating
various approaches, but I’m wondering if a separate session type is
necessary here. I think you’ve already been proposing an “inline” session
with AR support - would “camera-ar” differ significantly from an inline AR
session that has raw camera access enabled on it? What I’m trying to
understand here is if this part of the discussion is more about tweaking
raw camera access API vs more about either enabling AR for inline sessions
or configuring the immersive-ar session to not take over the display on the
smartphones. If it’s the latter, then I’d like to still move forward with
the current proposal.

Thanks!
-Piotr



On Thu, 14 Jul 2022 at 06:56, Simon Taylor <simon@zappar.com> wrote:

> Hi Piotr,
>
> Thanks for the update, and your efforts to move this one forward!
>
> Is your intention to have the current Chrome raw-camera-access
> implementation available without the feature flag?
>
> I know we had a discussion some months ago at the face-to-face about
> various ways to expose raw camera access, in brief:
>
> - Synchronous getter in rAF callback - current raw-camera-access
> implementation
> - getUserMedia + new constraints - some support in the discussion, but I
> have some practical concerns
> - A new “camera-ar” session type - would sit in IW’s sphere of influence
> (unlike gUM) and could share all the pose / anchor / etc functionality
>
> I discussed the gUM and new session type options in the issue here:
> https://github.com/immersive-web/webxr-ar-module/issues/78
>
> I think someone in the discussion at the f2f (Alex Turner IIRC) suggested
> writing up some pros/cons for various options on issues. I think I made a
> decent start on that in that issue, but I’m happy to do more work in that
> direction if there is value in it and if there is implementor interest.
>
> I understand the easiest implementation to ship is the one that’s already
> there, so fully understand if you don’t have the resource or time to commit
> to any alternatives at the moment, but it would be good to understand the
> level of interest.
>
> The camera-ar session feels relatively straightforward an implementation
> for mobile platforms, would completely address my concerns around giving up
> layout control to the “immersive” sessions, be easier to polyfill (eg on
> iOS Safari which lacks full screen atm), would support either “rendering
> effects” or “computer vision” use cases, and would be implementable on
> headsets too.
>
> There’s likely a small performance cost on Android, as you’d need to blit
> the SurfaceTexture from ARCore to a framebuffer to support the more async
> access model through an ImageBitmap. I think the performance cost would be
> worth the benefits, and I believe the ARCore wrappers for various engines
> do this anyway (ref an old ARCore issue I opened in 2018:
> https://github.com/google-ar/arcore-android-sdk/issues/668)
>
> Simon
>
> On 13 Jul 2022, at 18:31, Piotr Bialecki <bialpio@google.com> wrote:
>
> Hey Immersive Web!
>
> Just a quick update regarding WebXR's Raw Camera Access module. Latest
> state is that I've issued a PR
> <https://github.com/immersive-web/raw-camera-access/pull/12> that should
> address some concerns from TAG. After that lands, I'm planning on kicking
> off blink launch process which requires gathering signals from other
> browser vendors, so don't be surprised if you see some activity there!
>
> Thanks,
> -Piotr Bialecki
>
>
>