RE: The User Input of Mathematics with Recognition Hints

Nick Felker,

Thank you. I’m a proponent of some sort of recognition hints system for mathematics handwriting and speech recognition.

Were a recognition hints markup to be, for example, metadata-based, we might envision something resembling:

<hints id="hints1">
  <meta name="subject" content="msc2010:97H20" />
  <meta name="grapheme" content="x" />
  <meta name="grapheme" content="y" />
</hints>

<input type="math" hints="hints1" />

With regard to mathematical subjects, we could make use of some subset of the Mathematics Subject Classification ( https://en.wikipedia.org/wiki/Mathematics_Subject_Classification , http://msc2010.org/mediawiki/index.php?title=MSC2010 , http://msc2010.org/mediawiki/index.php?title=97-XX ).
Recognition hints markup could require expressiveness for, beyond plain text literals, XML literals, to indicate multiple user input possibilities in MathML. In some educational exercises and activities scenarios, multiple user input possibilities can be described a priori (for exercises, multiple expected user inputs might be provided as recognition hints; for quiz and exam items, they might not be provided). We might envision something resembling:

<hints id="hints1">
  <meta name="subject" content="msc2010:97H20" />
  <meta name="grapheme" content="x" />
  <meta name="grapheme" content="y" />
  <meta name="expected"><math><msup><mrow><mi>x</mi></mrow><mrow><mn>2</mn></mrow></msup><mo>+</mo><msup><mrow><mi>y</mi></mrow><mrow><mn>2</mn></mrow></msup></math></meta>
</hints>

<input type="math" hints="hints1" />


Best regards,
Adam


________________________________
From: Nick Felker <handnf@gmail.com>
Sent: Saturday, August 18, 2018 3:01:50 PM
To: adamsobieski@hotmail.com
Cc: public-exercises-and-activities@w3.org; www-math@w3.org
Subject: Re: The User Input of Mathematics with Recognition Hints


I think the input type would be interesting, but I don't know that the hints would be the most standard way of stating the expected format. Other input types<https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input> will just concatenate the specific format at the end. So it'd probably look more like

<input type='math-arith'>

<input type='math-calc'>

etc.

Even with this, what data format would the web application receive? That would also need to be standard within the browser so it can be processed.

On Sat, Aug 18, 2018 at 12:23 AM Adam Sobieski <adamsobieski@hotmail.com<mailto:adamsobieski@hotmail.com>> wrote:
Educational Exercises and Activities Community Group,
Math Working Group,

The user input of mathematics is an important topic. With respect to implementation, Web browsers could make use of operating system platform features (see also: touchscreen keyboards and handwriting recognition widgets) as well as third-party services. In addition to user interface widgets utilizable with a mouse and keyboard, mathematics input modalities include handwriting and speech recognition.

Contextual recognition hints could enhance the accuracy of both handwriting and speech recognition. Hints for mathematics recognition could include specifying the type of mathematics to be inputted, for instance arithmetic, algebra, geometry, trigonometry or calculus. Hints for mathematics recognition could include specifying expected variables or symbols. Hints for mathematics recognition could include specifying MathML alternatives for expected user inputs.

One approach to providing mathematics user input in hypertext documents is to extend the HTML5 <input> element. Form submission data from such <input> elements would be MathML. Mathematics recognition services could make use of the MathML annotation framework to provide recognition results in other formats as well (e.g. AMS-LaTeX).

In the approach of extending the HTML5 <input> element while providing hints for mathematics recognition, what might the ideal hypertext document markup resemble?

Markup for hints could be nested in <input> elements:

<input type="math">
  <hint … />
  <hint … />
</input>

<input type="text-and-math">
  <hint … />
  <hint … />
</input>

<textarea math="true">
  <hint … />
  <hint … />
</textarea>

<matharea>
  <hint … />
  <hint … />
</matharea>

Also possible is a technique for hints resembling the HTML5 <datalist> element:

<hintlist id="hints1">
  <hint … />
  <hint … />
</hintlist>

<input type="math" hints="hints1" />

<input type="text-and-math" hints="hints1" />

<textarea math="true" hints="hints1" />

<matharea hints="hints1" />

What do you think about the approach of expanding the <input> element to include a mode for mathematics user input? What do you think about the indicated possibilities for providing contextual recognition hints in hypertext document markup? Are there any other approaches and possibilities to consider?

Thank you. I look forward to discussing approaches for providing the user input of mathematics with recognition hints.


Best regards,
Adam Sobieski
http://www.phoster.com/contents/