Re: Joint mtg followup re XAUR and Avatars

Hi Janina,

Yes...I agree that conveying emotion and the nuance of expressive language is one of the areas where automated avatar technology has some significant limitations. Quite a few years ago, I worked on a project led by TERC to create a series of Signing Math and Science Dictionaries<https://www.terc.edu/terc_products/signing-math-and-science-dictionaries/#:~:text=For%20Signing%20Math%20%26%20Science%2C%20TERC,same%20learning%20opportunities%20that%20hearing>. At the time, this was done with cutting edge 3D avatar technology. I guess some would argue it could have been done just as well by simply recording human signers, but one of the goals of the project was to explore how automation could be used, as well as perhaps determining if there might be a sense in which a 3D view provided by the avatar might at times be better than a flat 2D video of a human. So, in the case of canned factual offerings like a dictionary, avatar technology might be a viable alternative. But the conveyance of emotion and other nuances of expressive language is likely some of the most difficult aspects of automated signing.

--Steve




Steve Noble
Instructional Designer, Accessibility
Psychometrics & Testing Services

Pearson

502 969 3088
steve.noble@pearson.com<mailto:steve.noble@pearson.com>

[https://ci3.googleusercontent.com/proxy/xFjftXlwMzpdFeTtDgc4_IwyMYm8ThtQHIsgElkS8fyiCO2M7ZM0WaO7r2uy-bmKAe5S2sIcg7d-mwbD4ArkJhyafHke-SgJ2ui8DoGoBhZw4YIyWeK3LUozNMwBff4JR2tdu8nZ2fvoNvkkA06KNw9-s3P9UvYsHSTphHss6X0=s0-d-e1-ft#http://accessibility4school.pearson.com/access/4c49fe02-e204-46b4-b6f0-82f5a3f159cb/pearson-accessibility.jpg]


________________________________
From: Janina Sajka <janina@rednote.net>
Sent: Wednesday, May 19, 2021 3:05 PM
To: Joshue O'Connor <joconnor@w3.org>
Cc: public-rqtf@w3.org <public-rqtf@w3.org>
Subject: Re: Joint mtg followup re XAUR and Avatars

Yeah, so looking a bit further into conveying emotion ... If one googles
for:

artificial emotional intelligence

One gets some very interesting results. Not sure how much has made into
deployable libraries yet, but there's definitely such a field of study,
and it's implications are likely very relevant across disability as well
as cross-cultural application. I'm reading a book by one scientist in
the field that I pulled from NLS in the last month.

Of course this also further illustrates how different our opportunities
are with canned vs real-time media.

Joshue O Connor writes:
> Thanks for the input Janina, and yes, if we are to continue/progress with
> user need, we need to be clear around what represents quality vs an 'out of
> the box ersatz avatar'.
>
> I also wish we didn't have to use that term avatar, the budding Sanskritian
> in me objects!
>
> Josh
>
> > Janina Sajka <mailto:janina@rednote.net>
> > Wednesday 19 May 2021 16:17
> > I thought I should say for the record what I raised my hand to say when
> > we ran out of time ...
> >
> > Regarding avatars ...
> >
> > I was highly impressed by the detailes that emerged about why avatars
> > tend to fail SL users. I'm thinking we should capture a high level
> > description of what would be required to create a successful avatar in
> > the XAUR by way of answering any engineering interest in moving to their
> > use prematurely.
> >
> > I believe the explanation is that SL captures far more than the words
> > which are captured in a text transcript of what's being said. SL
> > attempts to communicate more of the conversation than just the verbal
> > language content we've learned to capture with paper and ink.
> >
> > Facial expression -- there are some 43 muscles that control facial
> > expression, though if one googles this question the answers vary, 43,
> > 42, 33 ...
> >
> > Implication: anyone building a signing avatar should provide a face and
> > 43 functioning muscular variables.
> >
> > Similarly, there's the challenge to understand the nuance of vocal
> > expression. Consider the word "O:"
> >
> > O (as in startled surprise)
> > O? (as in really?)
> > O (as in oops, which sometimes comes out as "o, o)
> >
> > There are more for just this one word, but I believe I've made my point.
> >
> > If we take this tack we avoid a perscription against engineering
> > development and supplant it with the far more meaningful challenge of
> > what it takes to design a satisfying avatar.
> >
> > Thoughts?
> >
> > Janina
> >
> >
>
>
> --
> Emerging Web Technology Specialist/Accessibility (WAI/W3C)

--

Janina Sajka
https://urldefense.com/v3/__https://linkedin.com/in/jsajka__;!!LtJ5xwj-!c32tNjkENaUPuhxzoyRyGNKyNCWJfktfXKOW_iCVxoAK2l9sAYSd61dhyhbAryCkKA$

Linux Foundation Fellow
Executive Chair, Accessibility Workgroup:       https://urldefense.com/v3/__http://a11y.org__;!!LtJ5xwj-!c32tNjkENaUPuhxzoyRyGNKyNCWJfktfXKOW_iCVxoAK2l9sAYSd61dhyhZfbEcdUg$

The World Wide Web Consortium (W3C), Web Accessibility Initiative (WAI)
Co-Chair, Accessible Platform Architectures     https://urldefense.com/v3/__http://www.w3.org/wai/apa__;!!LtJ5xwj-!c32tNjkENaUPuhxzoyRyGNKyNCWJfktfXKOW_iCVxoAK2l9sAYSd61dhyhZ-s_1Lig$