Re: AR Web

On Sat, Aug 18, 2018 at 2:06 PM James Baicoianu <james_w3c@baicoianu.com>
wrote:

> On 08/18/2018 12:12 PM, holykoolala wrote:
>
> Is this discussion setting off red flags for anyone else? Official
> standards seem premature for something so undeveloped and not well
> understood.
>
>
> I think that settling on one specific standards-dictated model type might
> be problematic, yes, given the number of different formats out there, and
> their applicability to different use-cases.  But what I think would be
> interesting to explore is how we could standardize an internal
> representation of 3d model/scene data without constraining to specific file
> formats.  I think this could be accomplished by looking at the loading
> process separately from the process of working with the loaded model data.
>
> For example, perhaps the standard could specify a set of geometry
> primitives - vertices, faces, and all of those basic underlying concepts
> that modern engines understand, and most file formats represent in some
> form.  The <model> tag spec could then represent a collection of these
> primitives, in a clearly-specified way, but instead of requiring browsers
> to implement one specific model format as the only allowable filetype, it
> could provide hooks in the form of API calls and events which would make it
> possible to implement or extend the loading process at the app level.
> Maybe this would look something like:
>
> class MyCustomModelLoader extends HTMLModelLoaderElement {
>   function canLoad(data) {
>     ...
>   }
>   function load(data) {
>     ...
>   }
> };
> let model = document.createElement('model');
> model.addHandler('obj', MyCustomModelLoader);
> model.addEventListener('load', (modeldata) => { console.log('model loaded!', modeldata); })
> model.src = 'mymodel.obj';
>
>
> As long as this or some other means of extending the loaders is possible,
> then standardizing on a common internal representation of the primitives
> benefits all apps, by making it easier to share data between them.  I could
> still load data from esoteric file formats and turn them into
> HTMLModelElement objects, which could then be used anywhere which
> understands how to work with that element type.
>
> From there, the spec could maybe call out some formats which browsers must
> implement, like glTF, without dictating that it's the only supported file
> format.  So developers would have the benefit of knowing that at least one
> format is supported out of the box by all browsers, but that they're not
> constrained in the case where they want to load an existing file that
> hasn't been/can't easily be converted into glTF format.
>
FYI, we did not extend the HTML namespace by adding new element types.
Instead, we created a set of lower level JavaScript functions and then
built Prismatic to provide a declarative interface using that API.
We use custom elements [1] to create our custom <ml-model> and <ml-quad>
tags.

We went this way because it's very flexible (easier to iterate in
JavaScript) and because we suspect that creating new tags would be a
drawn-out process.

> On the specific question of 3d favicons, I think as Blair and Justin said,
> at this level it would be up to the user agents to decide which formats
> they support - the interpretation of favicons and the ability to support
> different formats has always been up to the useragent, so it isn't really a
> big leap to allow for 3d model files here.
>
1:
https://developer.mozilla.org/en-US/docs/Web/Web_Components/Using_custom_elements


>
> On 08/18/2018 01:42 PM, Justin Rogers wrote:
>
> Technically it is almost spec’ed already.
>
> https://html.spec.whatwg.org/multipage/links.html#rel-icon
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__html.spec.whatwg.org_multipage_links.html-23rel-2Dicon&d=DwMDaQ&c=0ia8zh_eZtQM1JEjWgVLZg&r=jahSgznxrAL5kPgsRvs7bhKUEd9M5X0d-NE2WJg7VT0&m=mkfDdK3sAlkkN0Hp1Xt9qU7_QSMyYjK93MUAbO7Mjf8&s=lKyPgMic3wgMYUbfyuDvvN768m8_J44dGHqMR2kHlMo&e=>
>
> Only issue is that our type is model/gltf+... which is not an image type.
> The extension to the specification would be that:
> 1) The type can be other than an image type such as model. We could say
> image or model for now. Which formats of models is not important really
> since we probably want to be restrictive but until the mass web agrees it
> would be better to simply accept any supported popular 3d format.
> 2) We should consider if we want to amend the default lookup behavior when
> an icon is not present. I propose that we do not as that is legacy behavior
> and in our case we can do this more completely. That said it will be a pain
> to have every page on a site get updated and I’m betting a large majority
> of fav icons today are served from the legacy behavior path. It would be
> good to get Edge, Chrome and FireFox to supply Browser stats on how most
> favicons are served.
>
> On Aug 18, 2018, at 1:10 PM, Blair MacIntyre <bmacintyre@mozilla.com>
> wrote:
>
> Not really red flags for the 3D favicon;  this seems like something that
> is pretty “small”, would be really useful, and could be worked out and
> implemented by a few browsers, especially since the bigger questions of
> “what to do with the favicon” is up to the UA.
>
>
> --
> Blair MacIntyre
> Principal Research Scientist
> bmacintyre@mozilla.com
> https://pronoun.is/he/him
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__pronoun.is_he_him&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=aiUMbGW4WL3JSYxR2Qm1uYACjq-bqRegNtlyhvV4xew&m=3Z7xD8YCqOmOMRGC9Q3JHJS0fK-JkA0AaGLo79sZ5Ew&s=BCCL9IrpI1a5RX9-gcRnBnYN6JuAq5mbWANP_xlaFkI&e=>
> https://blairmacintyre.me
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__blairmacintyre.me&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=aiUMbGW4WL3JSYxR2Qm1uYACjq-bqRegNtlyhvV4xew&m=3Z7xD8YCqOmOMRGC9Q3JHJS0fK-JkA0AaGLo79sZ5Ew&s=SPZ2St3v6SaAsAFB6siCaEPk3xFtFJrZ7AhhVSV85qE&e=>
>
>
> On August 18, 2018 at 3:15:31 PM, holykoolala (holykoolala@gmail.com)
> wrote:
>
> Is this discussion setting off red flags for anyone else? Official
> standards seem premature for something so undeveloped and not well
> understood.
>
> -Brett
>
> On Sat, Aug 18, 2018 at 10:38 AM Chris Wilson <cwilso@google.com> wrote:
>
>> On Fri, Aug 17, 2018 at 8:31 PM Rik Cabanier <rcabanier@magicleap.com>
>> wrote:
>>
>>> On Fri, Aug 17, 2018 at 4:46 PM Chris Wilson <cwilso@google.com> wrote:
>>>
>>>> Definite +1 to driving this in the IWCG.  The focus on getting WebXR
>>>> Device API out of the door will move to the soon-to-be-open-for-business
>>>> Working Group, which should free up some time and focus in the CG.  I'm
>>>> personally pretty interested in driving some discussion in the CG for how
>>>> we can get some model interop - i.e. rough standardization on asset type
>>>> support,
>>>>
>>>
>>> Yes, that is a good idea. We will likely have to talk to several group
>>> within W3C to get a standard model type for the web.
>>>
>>
>> Yup.  And make no mistake, I don't expect we'll limit the web to one
>> model type, but it would be good to get a baseline.
>>
>> As for the AR content, we can present what we currently implemented if
>>> there's interest from the group.
>>>
>>
>> I think that would be useful.
>>
>>
>>> and I've poked in to what you've done in Helio and Prismatic.
>>>>
>>>
>>> I'd love to hear what your thoughts are on our approach. Did you already
>>> find the web inspector? :-)
>>>
>>
>> I saw the support but haven't hooked it in yet.  I did get a hub with
>> mine, though, so I'm preparing.  :)  The browser UX had some... interesting
>> choices.  I think the ML-model design has some potential, though it seems
>> pretty specific to headset AR scenario at first glance, and I think
>> bridging across device scenarios will be one of the harder challenges here.
>>
>>
>>
>