RE: The User Input of Mathematics with Recognition Hints

Educational Exercises and Activities Community Group,
Math Working Group,

To those interested in the user input of mathematics in HTML5 documents via keyboard and mouse as well as speech and handwriting recognition,

<input type="math" />

<matharea />

please feel free to express your interest and opinions at: https://github.com/w3c/html/issues/1611 .


Best regards,
Adam Sobieski


________________________________
From: Adam Sobieski <adamsobieski@hotmail.com>
Sent: Tuesday, August 21, 2018 4:45:59 AM
To: public-exercises-and-activities@w3.org; www-math@w3.org
Subject: RE: The User Input of Mathematics with Recognition Hints

Educational Exercises and Activities Community Group,
Math Working Group,

I’m working on the markup and clarifying the explanation of the metadata-based system for multimodal mathematics recognition. These ideas are discussed at: https://www.w3.org/community/exercises-and-activities/wiki/User_Input_of_Mathematics .

<metadata id="hints1">
  <meta name="subject" content="msc2010:12D05" />
  <meta name="grapheme" content="x" />
  <meta name="grapheme" content="y" />
  <meta name="expected">
    <math>
      <msup>
        <mi>x</mi>
        <mn>2</mn>
      </msup>
      <mo>+</mo>
      <msup>
        <mi>y</mi>
        <mn>2</mn>
      </msup>
    </math>
  </meta>
  <meta name="expected">
    <math>
      <msup>
        <mi>x</mi>
        <mn>2</mn>
      </msup>
      <mo>-</mo>
      <msup>
        <mi>y</mi>
        <mn>2</mn>
      </msup>
    </math>
  </meta>
</metadata>

<input type="math" metadata="hints1" />

<matharea metadata="hints1" />


Best regards,
Adam Sobieski

From: Adam Sobieski<mailto:adamsobieski@hotmail.com>
Sent: Saturday, August 18, 2018 3:21 AM
To: public-exercises-and-activities@w3.org<mailto:public-exercises-and-activities@w3.org>; www-math@w3.org<mailto:www-math@w3.org>
Subject: The User Input of Mathematics with Recognition Hints

Educational Exercises and Activities Community Group,
Math Working Group,

The user input of mathematics is an important topic. With respect to implementation, Web browsers could make use of operating system platform features (see also: touchscreen keyboards and handwriting recognition widgets) as well as third-party services. In addition to user interface widgets utilizable with a mouse and keyboard, mathematics input modalities include handwriting and speech recognition.

Contextual recognition hints could enhance the accuracy of both handwriting and speech recognition. Hints for mathematics recognition could include specifying the type of mathematics to be inputted, for instance arithmetic, algebra, geometry, trigonometry or calculus. Hints for mathematics recognition could include specifying expected variables or symbols. Hints for mathematics recognition could include specifying MathML alternatives for expected user inputs.

One approach to providing mathematics user input in hypertext documents is to extend the HTML5 <input> element. Form submission data from such <input> elements would be MathML. Mathematics recognition services could make use of the MathML annotation framework to provide recognition results in other formats as well (e.g. AMS-LaTeX).

In the approach of extending the HTML5 <input> element while providing hints for mathematics recognition, what might the ideal hypertext document markup resemble?

Markup for hints could be nested in <input> elements:

<input type="math">
  <hint … />
  <hint … />
</input>

<input type="text-and-math">
  <hint … />
  <hint … />
</input>

<textarea math="true">
  <hint … />
  <hint … />
</textarea>

<matharea>
  <hint … />
  <hint … />
</matharea>

Also possible is a technique for hints resembling the HTML5 <datalist> element:

<hintlist id="hints1">
  <hint … />
  <hint … />
</hintlist>

<input type="math" hints="hints1" />

<input type="text-and-math" hints="hints1" />

<textarea math="true" hints="hints1" />

<matharea hints="hints1" />

What do you think about the approach of expanding the <input> element to include a mode for mathematics user input? What do you think about the indicated possibilities for providing contextual recognition hints in hypertext document markup? Are there any other approaches and possibilities to consider?

Thank you. I look forward to discussing approaches for providing the user input of mathematics with recognition hints.


Best regards,
Adam Sobieski
http://www.phoster.com/contents/