RE: Signed natural language

I should have watched the video while trying to describe it rather than going from memory. One video shows the signer signing "you-how" which is converted to "How are you?" and the other shows him signing "Hello-my-name_D-A-V-I-D" which is converted to "Hello, my name is David".

Happy Audio Description Awareness Day!

John

From: John Paton
Sent: 16 April 2021 09:35
To: public-rqtf@w3.org
Subject: Signed natural language

Hi all,

I was forwarded a link on LinkedIn about a sign language interface SDK<https://developers.googleblog.com/2021/04/signall-sdk-sign-language-interface-using-mediapipe-now-available.html> that can use a single camera interface to a computer and enable sign language input (in a sign language word order rather than in the word order of the spoken language).

I think a lot is conveyed through the video so I'll give a description. The video shows a man sitting in a chair. He signs "name-my-D-A-V-I-D" in ASL while the video places small white marker circles around the parts of the video it deems relevant [assumption, the reason for the circles is not explained]. As he makes each sign the video shows the sign added to the sentence it is constructing and the words are rearranged into an English word order and grammar as they are added. Once the computer has a sentence it presents it's best guesses for the meaning in boxes overlayed on the screen which the user selects by moving their hand into the box on the video and making a tapping motion.

This is, in effect, a sign language natural language interface.

There's still some work to do for them and unfortunately they had to create their own dataset rather than use any of the pre-existing sign language libraries. That means using it to support other sign languages is not a simple affair.

Cheers,

John


--

Whether you’re miles apart or side-by-side, pair up with a mate and run, walk, jog or skip 26.2 miles between you during the month of May. Register for free at https://marathonmates.rnib.org.uk and set your own fundraising target.

--


DISCLAIMER:

NOTICE: The information contained in this email and any attachments is confidential and may be privileged.  If you are not the intended recipient you should not use, disclose, distribute or copy any of the content of it or of any attachment; you are requested to notify the sender immediately of your receipt of the email and then to delete it and any attachments from your system.

RNIB endeavours to ensure that emails and any attachments generated by its staff are free from viruses or other contaminants.  However, it cannot accept any responsibility for any  such which are transmitted.

We therefore recommend you scan all attachments.

Please note that the statements and views expressed in this email and any attachments are those of the author and do not necessarily represent those of RNIB.

RNIB Registered Charity Number: 226227

Website: https://www.rnib.org.uk