# Fwd: [Filter Effects][css3-transforms] Using MathML for formulas

From: Davide P. Cervone <dpvc@union.edu>
Date: Mon, 21 May 2012 12:21:38 -0400
Message-Id: <69880729-2240-4D62-B0FD-549A35115F8B@union.edu>
To: Cameron McCormack <cam@mcc.id.au>, Dirk Schulze <dschulze@adobe.com>, Vincent Hardy <vhardy@adobe.com>, "public-fx@w3.org" <public-fx@w3.org>
Cameron, Dirk and Vincent:

Peter forwarded your message to me, as he felt I might be able to
contribute to this discussion.  Let me preface my remarks with a
disclaimer:  I am not an expert in assistive technology, and have only
worked with a few screen readers, so I don't have a great deal of
experience with the needs to those who are using assistive
technology.  What little I do know, however, suggests that screen
that it will be difficult to produce a solution that will work for all

Cameron's code samples are interesting ones, and I appreciated getting
to see them.  My experience with screen readers, however, suggests
that text that is visually understandable (like the equation
"miterLimit / stroke-width = 1 / sin ( theta / 2 )") is only going to
be appropriate for screen readers only in the simplest of cases.  What
needs to be spoken for mathematical text is quite different from what
is needed for ordinary English text, and different again from a visual
representation of mathematics as a text string (as you have found
out).  The reason that the parens are not spoken, for example, is that
parens in a sentence are not spoken -- they affect the pitch and tone,
but produce no sounds themselves.  Similarly, the comma in the
mathematics "(x,y)" probably should be spoken, but commas in sentences
aren't.  Special care must be taken to indicate where quantities are
begin and end, to distinguish between 1/(x+1) and 1/x + 1, for
example, or to indicate the extent of a square root's operand, or
where a matrix cell starts and stops.  There are a whole range of
issues of this sort that makes it difficult to use a visual
representation for spoken purposes, unless the screen reader has
special knowledge of mathematical notation.  For a screen reader with
knowledge of mathematics, that would most likely be a knowledge of
MathML, not of some plain-text notation for mathematics, and so trying
to use such plain-text strings for screen readers at the same time
they are used as visual alternatives for browser that don't support
MathML is going to be very difficult.

mathematics (at least I am not aware of one), the knowledge of how to
speak mathematics has to come from another source.  This is why the
current accessibility options for mathematics all involve MathPlayer,
as it is that component that tells the screen readers what to say.
The problem, of course, is that MathPlayer is only available for
Internet Explorer, so that limits its applicability.  What you need is
a more widely available component that can perform a similar function.

My feeling is that MathJax can do much more in this area than it
currently does.  Since MathJax does understand the structure of the
mathematics (since it understand MathML), it could produce text
strings that appropriately represent that mathematics and could
present those to the screen readers to read, while still providing a
good visual representation for the sighted.  The idea that has not yet
been discussed here is that these strings can be generated
automatically, and MathJax can do that.  It doesn't currently do this,
but it COULD.

As a proof of concept, I cobbled together a little bit of code that
extends MathJax to do just that (using the CDN copy of MathJax,
unaltered).  It is not complete, but includes enough to see that it
could work.  I have only tested it with Apple's VoiceOver, so I know
it works with that, but don't know about other screen readers.  My
past experience was that not all screen readers handle the ARIA
attributes, and thee are crucial to the functioning of this test
page.  (Without them, for example, the screen reader will try to read
the HTML output directly, as well as the special strings that MathJax
produced for the screen reader, which will produce gibberish (in
addition to the correct spoken string).  There seem to be effective
ways of adding text for screen readers that are not visible to the
sighted, but the reverse is not so universal; we should be able to use
aria-hidden="true", but this doesn't work for all screen readers.
(Even VoiceOver doesn't always respect this setting; for example, I
could not get VoiceOver not to read the pop-up menu in the same page I

That being said, you can try out the sample at:

http://www.math.union.edu/locate/Cervone/transfer/mathjax/speech-lab.html

You can type TeX code into the area at the top, and press the
"Typeset" button (or CTRL-RETURN) to get the math to be typeset.  If
you are using a current enough version of VoiceOver, it should speak
the new math automatically, otherwise you may need to do CRTL-OPTION-A
to get it to speak it (and it may speak the pop-up menu as well).
There are a number of pre-defined examples in the pop-up menu, several
taken from the SVG documentation.  There is a checkbox that lets you
see the actual string that is generated.  Note that this is pretty
worthless for a sighted person, as everything is written out in words,
and there are a lot of commas inserted to introduce pauses that are
crucial for the understanding of the spoken versions.  For example,
your miterLimit to stroke-width ratio expression produces:

fraction, miterLimit, over stroke-width, end fraction, = fraction, 1,
over sine of quantity fraction, theta, over 2, end fraction, end
quantity, end fraction,

Not everything is spoken in an optimal way (since I only spent about 8
hours putting it all together), and the page linked above only knows
the words for a small number of symbols.  It seems that VoiceOver says
nothing for symbols it doesn't understand, so if you use something
like \subset it will probably skip that.  In any case, there is still
work to be done to make something like this production-ready, but this
should give you a sense of what might be possible.

I would be interested in what you think of this approach, and of the
results of this prototype version.

Davide

> ---------- Forwarded message ----------
> From: Cameron McCormack <cam@mcc.id.au>
> Date: Fri, May 18, 2012 at 9:35 PM
> Subject: Re: [Filter Effects][css3-transforms] Using MathML for
> formulas
> Cc: Peter Krautzberger <pkrautzb@umich.edu>, Vincent Hardy <vhardy@adobe.com
> >, "public-fx@w3.org" <public-fx@w3.org>
>
>
> I think some of the existing <pre>-formatted formulae in the spec
> aren't really accessible yet anyway, especially those that render
> say matrices with repeated "[" and "]" characters.  The example that
> I replaced,
>
>  <pre>miterLimit / stroke-width = 1 / sin ( theta / 2 )</pre>
>
> is (at a guess) probably OK.
>
> I think we can use ARIA attributes to mark up the math with
> equivalent descriptions[1].  Here's a first go at representing the
> above formula:
>
>  <div role="math" aria-describedby="a">
>    <math display="block">
>      <semantics>
>        <mrow>
>          <mfrac>
>            <mi>miterLength</mi>
>            <mi>stroke-width</mi>
>          </mfrac>
>          <mo>=</mo>
>          <mfrac>
>            <mn>1</mn>
>            <mrow>
>              <mi>sin</mi>
>              <mo>&#x2061;</mo>
>              <mfrac>
>                <mi>θ</mi>
>                <mn>2</mn>
>              </mfrac>
>            </mrow>
>          </mfrac>
>        </mrow>
>        <annotation-xml encoding="application/xhtml+xml">
>          <pre id="a">miterLimit / stroke-width = 1 / sin(theta / 2)</
> pre>
>        </annotation-xml>
>      </semantics>
>    [/itex]
>  </div>
>
>
> This is not going to work with MathJax -- either with the SVG or HTML
> +CSS renderings -- since as people have pointed out the > element gets replaced with content like this (in the SVG case): > > <div style="text-align: center;" aria-readonly="true" role="textbox" > class="MathJax_SVG_Display"> > <span class="MathJax_SVG" style="display: inline-block;"> > <svg> > ... > </svg> > </span> > </div> > <script type="math/mml"> > (CDATA representing the serialised [itex] tree we started with) > </script> > > so the element with id="a" is gone. > > (I'm not sure why a read only text box is the best way to present > the <svg> representation of the math to ATs.) > > > Regardless, I'm after a practical solution here -- so perhaps we can > just fall back on the old "position some text for the screen reader > off the page where the sighted reader can't see it". > > <style> > div[role="math"] > :first-child + * { > position: absolute; > left: -400px; > width: 400px; > overflow: hidden; > } > </style> > <div role="math" aria-describedby="a"> > <math display="block"> > <mrow> > <mfrac> > <mi>miterLength</mi> > <mi>stroke-width</mi> > </mfrac> > <mo>=</mo> > <mfrac> > <mn>1</mn> > <mrow> > <mi>sin</mi> > <mo>&#x2061;</mo> > <mfrac> > <mi>θ</mi> > <mn>2</mn> > </mfrac> > </mrow> > </mfrac> > </mrow> >
>    <pre id="a">miterLimit / stroke-width = 1 / sin(theta / 2)</pre>
>  </div>
>
> This worked in Safari with VoiceOver -- although surprisingly to me,
> it didn't read the parentheses; I'm not sure if there is a way to
> investigate the exact characters that are in there with VO or other
> ATs.
>
> Can anyone suggest any better way to represent the math content in
> HTML besides a <pre> with content that looks like code?  Especially
> for more complex formulae like the ones in http://www.w3.org/TR/SVG11/implnote.html#ArcConversionEndpointToCenter
> .  Are screen reader users who read math on the web comfortable with
> TeX?  (I notice that Wikipedia uses <img>s of rendered math with TeX
> in the alt="".)
>
>
> Aside: it'd be neat if I could use CSS to provide aria-describedby
> relationships for all the math in the page at once, for example
> something like:
>
>  div[role="math"] {
>    aria-describedby: selector(:scope > :first-child + *);
>  };
>
>
> [1] http://www.w3.org/TR/wai-aria/roles#math
>
>
>

Received on Monday, 21 May 2012 16:24:50 UTC

This archive was generated by hypermail 2.3.1 : Monday, 22 June 2015 03:33:47 UTC