- From: John Paton <John.Paton@rnib.org.uk>
- Date: Fri, 5 Mar 2021 17:05:45 +0000
- To: "White, Jason J" <jjwhite@ets.org>, Janina Sajka <janina@rednote.net>
- CC: Joshue O'Connor <joconnor@w3.org>, Scott Hollier <scott@hollier.info>, "public-rqtf@w3.org" <public-rqtf@w3.org>
- Message-ID: <LO2P265MB3547DA6F2968033598DB79D4C0969@LO2P265MB3547.GBRP265.PROD.OUTLOOK.COM>
I should say that I am ok with the idea of collecting user requirements for a text natural language interface that we haven’t seen in the wild just as I am ok with doing it for sign language interfaces despite not having seen any of them working. For language learning a conversational textual natural language application would work well so we have a use case there. The restriction to text is more functionality based than accessibility based but there will still be accessibility aspects. It could always form a useful (if hugely unpopular) tool for big D Deaf people to improve their written language skills. VCO and captioned telephony require text output, and a conversational companionship application for speech impaired users is likely to use written natural language input (even if it will need improved AI to handle the conversing part). Googling natural language shows how much work is going on in this field so we may be creating the necessary user requirements just in time. I used to know teams of engineers who conversed largely by instant messenger so the concept isn’t totally unrealistic. A guy with hearing loss joined their team and they needed very little change to their normal working practice in that respect. I believe Microsoft has some bots built into their instant messaging productivity tools and an AI PA might use a combination of interface and conversation parsing to follow commands such as “could you set up that meeting?”. By disallowing hybrid interactions of natural language and multiple-choice such as the Word example we may be limiting the use cases but outside the NLI/conversational space I think hybrid systems may be picked up by other standards. John From: John Paton Sent: 05 March 2021 16:09 To: White, Jason J <jjwhite@ets.org>; Janina Sajka <janina@rednote.net> Cc: Joshue O'Connor <joconnor@w3.org>; Scott Hollier <scott@hollier.info>; public-rqtf@w3.org Subject: RE: [EXTERNAL] Natural language interfaces and conversational agents Hi Jason, With the example of a word processor that you give however, after the initial action of typing in a query the user normally chooses from a selection of potential options which to my understanding brings that interface out of scope of this work. Would this work focus entirely on the user requirements of that initial search box? Cheers, John From: White, Jason J <jjwhite@ets.org<mailto:jjwhite@ets.org>> Sent: 05 March 2021 16:00 To: Janina Sajka <janina@rednote.net<mailto:janina@rednote.net>> Cc: Joshue O'Connor <joconnor@w3.org<mailto:joconnor@w3.org>>; John Paton <John.Paton@rnib.org.uk<mailto:John.Paton@rnib.org.uk>>; Scott Hollier <scott@hollier.info<mailto:scott@hollier.info>>; public-rqtf@w3.org<mailto:public-rqtf@w3.org> Subject: RE: [EXTERNAL] Natural language interfaces and conversational agents CAUTION: External. Do not click links or open attachments unless you know the content is safe. ________________________________ Thank you, Janina, for raising important considerations here. Besides the "hands-off" nature, another advantage is efficiency. Consider a complex application such as a modern word processor. In my experience, it's often faster to type a natural language query (into the application itself or into a Web search engine) to find what one is looking for than to traverse a complex structure of graphical menus and dialogues. Microsoft Office applications let you type a keystroke (Alt-Q, if I remember rightly) and enter a search query to find what you are looking for. At least for a screen reader user with a keyboard interface, that's often faster than having to search menus/dialogues. Thus I suspect that even if a natural language interface is hands-on (requires touch/keyboard input for the particular user) there are interfaces for which it's still going to be more efficient than to switch to a GUI-based alternative. Efficiency matters in the workplace, in education, and elsewhere, even if the alternative facilitation is technically "accessible" and could still be used. Of course, there's also the case in which the same functionality is not provided via any other user interface than natural language. However, that isn't the case Janina describes. -----Original Message----- From: Janina Sajka <janina@rednote.net<mailto:janina@rednote.net>> Sent: Friday, 5 March 2021 10:15 To: White, Jason J <jjwhite@ets.org<mailto:jjwhite@ets.org>> Cc: Joshue O'Connor <joconnor@w3.org<mailto:joconnor@w3.org>>; John Paton <John.Paton@rnib.org.uk<mailto:John.Paton@rnib.org.uk>>; Scott Hollier <scott@hollier.info<mailto:scott@hollier.info>>; public-rqtf@w3.org<mailto:public-rqtf@w3.org> Subject: Re: [EXTERNAL] Natural language interfaces and conversational agents I've been sitting back and reading this conversation, and I'm appreciating some good progress in scoping potential work. I want to play the devil's advocate a bit. I'm unconvinced it's helpful to say the universe of natural language interfaces needs to be accessible to all pwd, because some of the edge cases are, imo, too cumbersome to pursue when alternative technologies exist that those users might find far more attractive. This is the concept known in U.S. disability regulation as "equivalent facilitation." If one can achieve the same functionality in a more usable interface, why would we insist they should work harder to do so in natural language environments? Seems to me the key function of natural language interfaces is not the language, but the hands off nature of the interaction. One is able to work with the technology without recourse to keyboard or mouse, and able to do so while one's hands may be otherwise occupied. Thus, one is able to wkr even from some physical distance. To my mind this explains the recent success of these agents, and it explains why the same general idea went nowhere in earlier days--think of the old MIT natural language phonomen Eliza: https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FELIZA&data=04%7C01%7Cjjwhite%40ets.org%7C5f3db94e4ac443afd88308d8dfe96fe5%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C637505540989980996%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=14VYqmQ0pQug%2F%2FY3ju%2FPlOF9ZHuDiy5YNLBj1OcDbac%3D&reserved=0<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FELIZA&data=04%7C01%7Cjjwhite%40ets.org%7C5f3db94e4ac443afd88308d8dfe96fe5%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C637505540989980996%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=14VYqmQ0pQug%2F%2FY3ju%2FPlOF9ZHuDiy5YNLBj1OcDbac%3D&reserved=0> I suggest we need to find a way to honor suitability for purpose into our scoping somehow. Best, Janina White, Jason J writes: > Thank you, Josh, for your thoughtful commentary. I think everyone agrees there are challenging scope boundaries in this area that we haven’t yet resolved. A good example of the problem that I’ve read in the research literature is as follows. > Consider a navigation application with a natural language interface and a graphically displayed map. The user points to a place on the map and says “send the ambulance here” to the voice agent. It’s the combination of the utterance and the pointing that determines what the user is referring to, but the pointing gesture and the map aren’t strictly speaking part of the natural language aspect of the design. > > From: Joshue O'Connor <joconnor@w3.org<mailto:joconnor@w3.org>> > Sent: Friday, 5 March 2021 9:30 > To: White, Jason J <jjwhite@ets.org<mailto:jjwhite@ets.org>> > Cc: John Paton <John.Paton@rnib.org.uk<mailto:John.Paton@rnib.org.uk>>; Scott Hollier > <scott@hollier.info<mailto:scott@hollier.info>>; public-rqtf@w3.org<mailto:public-rqtf@w3.org> > Subject: Re: [EXTERNAL] Natural language interfaces and conversational > agents > > Hi Jason and all, > > White, Jason J wrote on 05/03/2021 13:45: > > • Focus our work on the accessibility of the natural language interaction itself. As far as I know, no one has documented the accessibility requirements for it elsewhere. > • Refer to other guidance (WCAG, XAUR, RAUR, etc.) for the accessibility of other aspects of the user interface. > • Note that natural language interaction can occur as part of a larger interface and that the whole interface needs to be accessible. > > +1 from me, with qualifying comments to signal to you all my (ever) shifting perspective on this. As I commented in a private mail to Jason, the situation we are in regarding scope, and various challenges can be broadly broken into: > > 1) The I/O aspect > 2) The service (or agent) behind it > > There are also options on these approaches/perspectives, on these aspects which are 'narrow' - focusing initially on Speech/Voice User Interfaces only or much broader. My two cents are that starting from the narrow perspective would give us a basis to add other modalities later on, but there is push back on that, which I also appreciate and understand. If we were to then take the broader approach and try to widen the scope we can get into very muddy and indistinct water super quickly. For the broader scope approach my current thinking is that we may avoid confusion, mixing streams etc if we took up the idea of 'Natural Language Interface Accessibility User Requirements'. Thinking of Michael's sensible suggestion to have clearly defined terms etc this one if my fave, as it is already well defined, isn't just a marketing term etc. I prefer this to Smart Agents, which potentially pushes us into a sea of IoT and related services. One one level this may not be a bad thing, but we don't have infinite time either. > > To me, if we want to realise a user requirements document with a broader scope - this really nicely covers the need for a multi-modal, device independent descriptor for the I/O side and we can add a strapline or <h2> etc saying ' Accessibility infrastructure and supporting services' or similar. I'm thinking this would allow us to cover VUIs, and other I/O modalities for other groups, the kind of things that Jason refers to as 'Conversational' etc as well as look at the services behind them. > > This is really helpful Jason, and please lets continue to discuss these options, and indeed any more we may be missing. If we were to go down the broader road, then I find this terminology is the most suitable nomenclature that I've seen yet. > > HTH > > Josh > -- > Emerging Web Technology Specialist/Accessibility (WAI/W3C) > > ________________________________ > > This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender; do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited. > > > Thank you for your compliance. > > ________________________________ -- Janina Sajka https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flinkedin.com%2Fin%2Fjsajka&data=04%7C01%7Cjjwhite%40ets.org%7C5f3db94e4ac443afd88308d8dfe96fe5%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C637505540989990947%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gE2cKdz1Zc2yo0qq0ZbZITGV7TqBh7QqFVOqAAzoGoE%3D&reserved=0<https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flinkedin.com%2Fin%2Fjsajka&data=04%7C01%7Cjjwhite%40ets.org%7C5f3db94e4ac443afd88308d8dfe96fe5%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C637505540989990947%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=gE2cKdz1Zc2yo0qq0ZbZITGV7TqBh7QqFVOqAAzoGoE%3D&reserved=0> Linux Foundation Fellow Executive Chair, Accessibility Workgroup:https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fa11y.org%2F&data=04%7C01%7Cjjwhite%40ets.org%7C5f3db94e4ac443afd88308d8dfe96fe5%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C637505540989990947%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=kZbR2X3J%2BEUYTvZa41lCf5lcRIto4ZwmzqNr%2FcvTJLs%3D&reserved=0<https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fa11y.org%2F&data=04%7C01%7Cjjwhite%40ets.org%7C5f3db94e4ac443afd88308d8dfe96fe5%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C637505540989990947%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=kZbR2X3J%2BEUYTvZa41lCf5lcRIto4ZwmzqNr%2FcvTJLs%3D&reserved=0> The World Wide Web Consortium (W3C), Web Accessibility Initiative (WAI) Co-Chair, Accessible Platform Architectureshttps://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.w3.org%2Fwai%2Fapa&data=04%7C01%7Cjjwhite%40ets.org%7C5f3db94e4ac443afd88308d8dfe96fe5%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C637505540989990947%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=KIVPMylYRMJSdBnhdcoaipWhRufG4OaVarUzt%2FzoKA0%3D&reserved=0<Architectureshttps://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.w3.org%2Fwai%2Fapa&data=04%7C01%7Cjjwhite%40ets.org%7C5f3db94e4ac443afd88308d8dfe96fe5%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C637505540989990947%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=KIVPMylYRMJSdBnhdcoaipWhRufG4OaVarUzt%2FzoKA0%3D&reserved=0> ________________________________ This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender; do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited. Thank you for your compliance. ________________________________ -- Every day, 250 people in the UK begin to lose their sight, that’s why we need you to Take on 250 for RNIB. Walking, running, cycling or swimming; baking, singing, dancing or knitting. It’s all up for grabs – and you complete 250 of whatever you decide. Join us and make a difference for people facing sight loss. Join us at https://www.rnib.org.uk/donations-and-fundraising/challenge-events/take-250-rnib and make a difference for people facing sight loss. -- DISCLAIMER: NOTICE: The information contained in this email and any attachments is confidential and may be privileged. If you are not the intended recipient you should not use, disclose, distribute or copy any of the content of it or of any attachment; you are requested to notify the sender immediately of your receipt of the email and then to delete it and any attachments from your system. RNIB endeavours to ensure that emails and any attachments generated by its staff are free from viruses or other contaminants. However, it cannot accept any responsibility for any such which are transmitted. We therefore recommend you scan all attachments. Please note that the statements and views expressed in this email and any attachments are those of the author and do not necessarily represent those of RNIB. RNIB Registered Charity Number: 226227 Website: https://www.rnib.org.uk
Received on Friday, 5 March 2021 17:06:07 UTC