Re: [Filter Effects][css3-transforms] Using MathML for formulas

Thanks for looking into this!

This strengthens the case for going with a script based solution like
MathJax. If Peter Krautzberger can incorporate your suggestion, we will
have better accessibility than if we had to do it manually.

I personally have no issue with using scripts but we still haven't gotten
an official reply from the W3C that confirms that this is an accepted
practice.

Rik

On Mon, May 21, 2012 at 7:21 PM, Davide P. Cervone <dpvc@union.edu> wrote:

> Cameron, Dirk and Vincent:
>
> Peter forwarded your message to me, as he felt I might be able to
> contribute to this discussion.  Let me preface my remarks with a
> disclaimer:  I am not an expert in assistive technology, and have only
> worked with a few screen readers, so I don't have a great deal of
> experience with the needs to those who are using assistive technology.
>  What little I do know, however, suggests that screen readers vary widely
> in their adherence to the various standards, and that it will be difficult
> to produce a solution that will work for all readers that people have
> available.
>
> Cameron's code samples are interesting ones, and I appreciated getting to
> see them.  My experience with screen readers, however, suggests that text
> that is visually understandable (like the equation "miterLimit /
> stroke-width = 1 / sin ( theta / 2 )") is only going to be appropriate for
> screen readers only in the simplest of cases.  What needs to be spoken for
> mathematical text is quite different from what is needed for ordinary
> English text, and different again from a visual representation of
> mathematics as a text string (as you have found out).  The reason that the
> parens are not spoken, for example, is that parens in a sentence are not
> spoken -- they affect the pitch and tone, but produce no sounds themselves.
>  Similarly, the comma in the mathematics "(x,y)" probably should be spoken,
> but commas in sentences aren't.  Special care must be taken to indicate
> where quantities are begin and end, to distinguish between 1/(x+1) and 1/x
> + 1, for example, or to indicate the extent of a square root's operand, or
> where a matrix cell starts and stops.  There are a whole range of issues of
> this sort that makes it difficult to use a visual representation for spoken
> purposes, unless the screen reader has special knowledge of mathematical
> notation.  For a screen reader with knowledge of mathematics, that would
> most likely be a knowledge of MathML, not of some plain-text notation for
> mathematics, and so trying to use such plain-text strings for screen
> readers at the same time they are used as visual alternatives for browser
> that don't support MathML is going to be very difficult.
>
> Since screen readers don't have direct knowledge of how to read
> mathematics (at least I am not aware of one), the knowledge of how to speak
> mathematics has to come from another source.  This is why the current
> accessibility options for mathematics all involve MathPlayer, as it is that
> component that tells the screen readers what to say.  The problem, of
> course, is that MathPlayer is only available for Internet Explorer, so that
> limits its applicability.  What you need is a more widely available
> component that can perform a similar function.
>
> My feeling is that MathJax can do much more in this area than it currently
> does.  Since MathJax does understand the structure of the mathematics
> (since it understand MathML), it could produce text strings that
> appropriately represent that mathematics and could present those to the
> screen readers to read, while still providing a good visual representation
> for the sighted.  The idea that has not yet been discussed here is that
> these strings can be generated automatically, and MathJax can do that.  It
> doesn't currently do this, but it COULD.
>
> As a proof of concept, I cobbled together a little bit of code that
> extends MathJax to do just that (using the CDN copy of MathJax, unaltered).
>  It is not complete, but includes enough to see that it could work.  I have
> only tested it with Apple's VoiceOver, so I know it works with that, but
> don't know about other screen readers.  My past experience was that not all
> screen readers handle the ARIA attributes, and thee are crucial to the
> functioning of this test page.  (Without them, for example, the screen
> reader will try to read the HTML output directly, as well as the special
> strings that MathJax produced for the screen reader, which will produce
> gibberish (in addition to the correct spoken string).  There seem to be
> effective ways of adding text for screen readers that are not visible to
> the sighted, but the reverse is not so universal; we should be able to use
> aria-hidden="true", but this doesn't work for all screen readers.  (Even
> VoiceOver doesn't always respect this setting; for example, I could not get
> VoiceOver not to read the pop-up menu in the same page I link to below).
>
> That being said, you can try out the sample at:
>
> http://www.math.union.edu/locate/Cervone/transfer/mathjax/speech-lab.html
>
> You can type TeX code into the area at the top, and press the "Typeset"
> button (or CTRL-RETURN) to get the math to be typeset.  If you are using a
> current enough version of VoiceOver, it should speak the new math
> automatically, otherwise you may need to do CRTL-OPTION-A to get it to
> speak it (and it may speak the pop-up menu as well).  There are a number of
> pre-defined examples in the pop-up menu, several taken from the SVG
> documentation.  There is a checkbox that lets you see the actual string
> that is generated.  Note that this is pretty worthless for a sighted
> person, as everything is written out in words, and there are a lot of
> commas inserted to introduce pauses that are crucial for the understanding
> of the spoken versions.  For example, your miterLimit to stroke-width ratio
> expression produces:
>
> fraction, miterLimit, over stroke-width, end fraction, = fraction, 1, over
> sine of quantity fraction, theta, over 2, end fraction, end quantity, end
> fraction,
>
> Not everything is spoken in an optimal way (since I only spent about 8
> hours putting it all together), and the page linked above only knows the
> words for a small number of symbols.  It seems that VoiceOver says nothing
> for symbols it doesn't understand, so if you use something like \subset it
> will probably skip that.  In any case, there is still work to be done to
> make something like this production-ready, but this should give you a sense
> of what might be possible.
>
> I would be interested in what you think of this approach, and of the
> results of this prototype version.
>
> Davide
>
>
> ---------- Forwarded message ----------
> From: Cameron McCormack <cam@mcc.id.au>
> Date: Fri, May 18, 2012 at 9:35 PM
> Subject: Re: [Filter Effects][css3-transforms] Using MathML for formulas
> To: Dirk Schulze <dschulze@adobe.com>
> Cc: Peter Krautzberger <pkrautzb@umich.edu>, Vincent Hardy <
> vhardy@adobe.com>, "public-fx@w3.org" <public-fx@w3.org>
>
>
> I think some of the existing <pre>-formatted formulae in the spec aren't
> really accessible yet anyway, especially those that render say matrices
> with repeated "[" and "]" characters.  The example that I replaced,
>
>
>  <pre>miterLimit / stroke-width = 1 / sin ( theta / 2 )</pre>
>
> is (at a guess) probably OK.
>
> I think we can use ARIA attributes to mark up the math with equivalent
> descriptions[1].  Here's a first go at representing the above formula:
>
>  <div role="math" aria-describedby="a">
>    <math display="block">
>      <semantics>
>        <mrow>
>          <mfrac>
>            <mi>miterLength</mi>
>            <mi>stroke-width</mi>
>          </mfrac>
>          <mo>=</mo>
>          <mfrac>
>            <mn>1</mn>
>            <mrow>
>              <mi>sin</mi>
>              <mo>&#x2061;</mo>
>              <mfrac>
>                <mi>θ</mi>
>                <mn>2</mn>
>              </mfrac>
>            </mrow>
>          </mfrac>
>        </mrow>
>        <annotation-xml encoding="application/xhtml+**xml">
>          <pre id="a">miterLimit / stroke-width = 1 / sin(theta / 2)</pre>
>        </annotation-xml>
>      </semantics>
>    </math>
>  </div>
>
>
> This is not going to work with MathJax -- either with the SVG or HTML+CSS
> renderings -- since as people have pointed out the <math> element gets
> replaced with content like this (in the SVG case):
>
>  <div style="text-align: center;" aria-readonly="true" role="textbox"
>       class="MathJax_SVG_Display">
>    <span class="MathJax_SVG" style="display: inline-block;">
>      <svg>
>        ...
>      </svg>
>    </span>
>  </div>
>  <script type="math/mml">
>    (CDATA representing the serialised <math> tree we started with)
>  </script>
>
> so the element with id="a" is gone.
>
> (I'm not sure why a read only text box is the best way to present the
> <svg> representation of the math to ATs.)
>
>
> Regardless, I'm after a practical solution here -- so perhaps we can just
> fall back on the old "position some text for the screen reader off the page
> where the sighted reader can't see it".
>
>  <style>
>    div[role="math"] > :first-child + * {
>      position: absolute;
>      left: -400px;
>      width: 400px;
>      overflow: hidden;
>    }
>  </style>
>  <div role="math" aria-describedby="a">
>    <math display="block">
>      <mrow>
>        <mfrac>
>          <mi>miterLength</mi>
>          <mi>stroke-width</mi>
>        </mfrac>
>        <mo>=</mo>
>        <mfrac>
>          <mn>1</mn>
>          <mrow>
>            <mi>sin</mi>
>            <mo>&#x2061;</mo>
>            <mfrac>
>              <mi>θ</mi>
>              <mn>2</mn>
>            </mfrac>
>          </mrow>
>        </mfrac>
>      </mrow>
>    </math>
>    <pre id="a">miterLimit / stroke-width = 1 / sin(theta / 2)</pre>
>  </div>
>
> This worked in Safari with VoiceOver -- although surprisingly to me, it
> didn't read the parentheses; I'm not sure if there is a way to investigate
> the exact characters that are in there with VO or other ATs.
>
> Can anyone suggest any better way to represent the math content in HTML
> besides a <pre> with content that looks like code?  Especially for more
> complex formulae like the ones in http://www.w3.org/TR/SVG11/**
> implnote.html#**ArcConversionEndpointToCenter<http://www.w3.org/TR/SVG11/implnote.html#ArcConversionEndpointToCenter>.
>  Are screen reader users who read math on the web comfortable with TeX?  (I
> notice that Wikipedia uses <img>s of rendered math with TeX in the alt="".)
>
>
> Aside: it'd be neat if I could use CSS to provide aria-describedby
> relationships for all the math in the page at once, for example something
> like:
>
>  div[role="math"] {
>    aria-describedby: selector(:scope > :first-child + *);
>  };
>
>
> [1] http://www.w3.org/TR/wai-aria/**roles#math<http://www.w3.org/TR/wai-aria/roles#math>
>
>
>
>
>