RE: [MathML4] Multiple Formats for Presentation and Semantics

Math Working Group,
MathML Refresh Community Group,

I put together some discussion notes on these MathML4 and related Web technology topics. Attached is a PDF version. I hope this brainstorming is useful to the MathML4 endeavor and, in the event of interest, I could open a Google Documents document of this content.


Presentation and Semantics

<math id="eq1">

<presentation>

<annotation-xml encoding="application/xhtml+xml">...</annotation-xml>

<annotation-xml encoding="application/svg+xml">...</annotation-xml>

<annotation encoding="image/png" src="data:..." />

<annotation-xml encoding="MathML-Presentation">...</annotation-xml>

</presentation>

<semantics>

<annotation-xml encoding="application/openmath+xml">...</annotation-xml>

<annotation-xml encoding="MathML-Content">...</annotation-xml>

</semantics>

</math>

Presentation, Semantics and Metadata

<math id="eq1">

<presentation>

<annotation-xml id="eq1-p-xhtml" encoding="application/xhtml+xml">...</annotation-xml>

<annotation-xml id="eq1-p-svg" encoding="application/svg+xml">...</annotation-xml>

<annotation id="eq1-p-png" encoding="image/png" src="data:..." />

<annotation-xml id="eq1-p-mathmlp" encoding="MathML-Presentation">...</annotation-xml>

</presentation>

<semantics>

<annotation-xml id="eq1-s-om" encoding="application/openmath+xml">...</annotation-xml>

<annotation-xml id="eq1-s-mathmlc" encoding="MathML-Content">...</annotation-xml>

</semantics>

<metadata>

<annotation-xml encoding="application/rdf+xml">...</annotation-xml>

<annotation encoding="application/json+ld">...</annotation>

</metadata>

</math>

Multiple Notations

Expression “metadata” could be utilized to describe and interrelate multiple “presentation” annotations, e.g. to describe “presentation” annotations as utilizing distinct notations. With ontology for describing “presentation” annotations’ notations in “metadata”, one or more JavaScript libraries could be authored to facilitate navigating notation, e.g. selecting which notations are displayed for expressions in documents.

<math id="eq1">

<presentation>

<annotation-xml id="eq1-p-n1" encoding="MathML-Presentation">...</annotation-xml>

<annotation-xml id="eq1-p-n2" encoding="MathML-Presentation">...</annotation-xml>

<annotation-xml id="eq1-p-n3" encoding="MathML-Presentation">...</annotation-xml>

</presentation>

<semantics>

<annotation-xml id="eq1-s" encoding="MathML-Content">...</annotation-xml>

</semantics>

<metadata>

<annotation-xml encoding="application/rdf+xml">...</annotation-xml>

</metadata>

</math>



Should, instead, notation be an attribute on “presentation” annotations?

<math id="eq1">

<presentation>

<annotation-xml notation="Notation 1" encoding="MathML-Presentation">...</annotation-xml>

<annotation-xml notation="Notation 2" encoding="MathML-Presentation">...</annotation-xml>

<annotation-xml notation="Notation 3" encoding="MathML-Presentation">...</annotation-xml>

</presentation>

<semantics>

<annotation-xml id="eq1-s" encoding="MathML-Content">...</annotation-xml>

</semantics>

</math>

Multiple Presentation Formats and Multiple Notations

This example shows a combination of multiple presentation formats with multiple notations.

<math id="eq1">
<presentation>
<annotation-xml notation="Notation 1" encoding="application/xhtml+xml">...</annotation-xml>
<annotation-xml notation="Notation 2" encoding="application/xhtml+xml">...</annotation-xml>
<annotation-xml notation="Notation 3" encoding="application/xhtml+xml">...</annotation-xml>
<annotation-xml notation="Notation 1" encoding="application/svg+xml">...</annotation-xml>
<annotation-xml notation="Notation 2" encoding="application/svg+xml">...</annotation-xml>
<annotation-xml notation="Notation 3" encoding="application/svg+xml">...</annotation-xml>
<annotation notation="Notation 1" encoding="image/png" src="data:..." />
<annotation notation="Notation 2" encoding="image/png" src="data:..." />
<annotation notation="Notation 3" encoding="image/png" src="data:..." />
<annotation-xml notation="Notation 1" encoding="MathML-Presentation">...</annotation-xml>
<annotation-xml notation="Notation 2" encoding="MathML-Presentation">...</annotation-xml>
<annotation-xml notation="Notation 3" encoding="MathML-Presentation">...</annotation-xml>
</presentation>
<semantics>
<annotation-xml encoding="application/openmath+xml">...</annotation-xml>
<annotation-xml encoding="MathML-Content">...</annotation-xml>
</semantics>
</math>

Internationalization

For scenarios where natural language is utilized in the “presentation” annotation content, a BCP47 language attribute can adorn annotation markup and “presentation” annotations can also be described in expression “metadata”.

<math id="eq1">
<presentation>
<annotation-xml id="eq1-p-l1" lang="en" encoding="MathML-Presentation">...</annotation-xml>
<annotation-xml id="eq1-p-l2" lang="fr" encoding="MathML-Presentation">...</annotation-xml>
</presentation>
<semantics>
<annotation-xml id="eq1-s" encoding="MathML-Content">...</annotation-xml>
</semantics>
<metadata>
<annotation-xml encoding="application/rdf+xml">...</annotation-xml>
</metadata>
</math>

Multimodality

“Presentation” annotations can include multimodal content, e.g. SSML or audio, and “presentation” annotations can also be described in expression “metadata”.

<math id="eq1">
<presentation>
<annotation-xml lang="en" encoding="MathML-Presentation">...</annotation-xml>
<annotation-xml lang="fr" encoding="MathML-Presentation">...</annotation-xml>
<annotation-xml lang="en" encoding="application/ssml+xml">...</annotation-xml>
<annotation-xml lang="fr" encoding="application/ssml+xml">...</annotation-xml>
<annotation-xml lang="en" encoding="audio/mpeg" src="...">...</annotation-xml>
<annotation-xml lang="fr" encoding="audio/mpeg" src="...">...</annotation-xml>
</presentation>
<semantics>
<annotation-xml encoding="MathML-Content">...</annotation-xml>
</semantics>
</math>

Quality Scores

With quality score attributes, algorithms for selecting content from alternatives can resemble agent-driven or reactive content negotiation (https://en.wikipedia.org/wiki/Content_negotiation).

<math id="eq1">

<presentation>

<annotation-xml q="0.9" encoding="application/xhtml+xml">...</annotation-xml>

<annotation-xml q="0.9" encoding="application/svg+xml">...</annotation-xml>

<annotation q="0.9" encoding="image/png" src="data:..." />

<annotation-xml q="1.0" encoding="MathML-Presentation">...</annotation-xml>

</presentation>

<semantics>

<annotation-xml encoding="application/openmath+xml">...</annotation-xml>

<annotation-xml encoding="MathML-Content">...</annotation-xml>

</semantics>

</math>

Remote Content

<math id="eq1">

<presentation>

<annotation-xml encoding="application/xhtml+xml" src="eq1.xhtml" />

<annotation-xml encoding="application/svg+xml" src="eq1.svg" />

<annotation encoding="image/png" src="eq1.png" />

<annotation-xml encoding="MathML-Presentation" src="eq1.mmlp" />

</presentation>

<semantics>

<annotation-xml encoding="application/openmath+xml" src="eq1.om" />

<annotation-xml encoding="MathML-Content" src="eq1.mmlc" />

</semantics>

</math>

Content Negotiation

Using agent-driven or reactive content negotiation over HTTP, URL’s can be provided for “presentation”, “semantics” and “metadata” (https://en.wikipedia.org/wiki/Content_negotiation).

<math id="eq1">

<presentation src="eq1.php?content=presentation" />

<semantics src="eq1.php?content=semantics" />

<metadata src="eq1.php?content=metadata" />

</math>

Toward a Single URL per Mathematical Expression

<math id="eq1" src="eq1.php" />

Extensibility

Are there any other varieties of content for a mathematical expression beyond “presentation”, “semantics” and “metadata”? Might we want to include “other” for extensibility?

<math id="eq1">

<presentation>

<annotation-xml id="eq1-p-xhtml" encoding="application/xhtml+xml">...</annotation-xml>

<annotation-xml id="eq1-p-svg" encoding="application/svg+xml">...</annotation-xml>

<annotation id="eq1-p-png" encoding="image/png" src="data:..." />

<annotation-xml id="eq1-p-mathmlp" encoding="MathML-Presentation">...</annotation-xml>

</presentation>

<semantics>

<annotation-xml id="eq1-s-om" encoding="application/openmath+xml">...</annotation-xml>

<annotation-xml id="eq1-s-mathmlc" encoding="MathML-Content">...</annotation-xml>

</semantics>

<metadata>

<annotation-xml encoding="application/rdf+xml">...</annotation-xml>

<annotation encoding="application/json+ld">...</annotation>

</metadata>

<other rel="http://www.<http://www.semantic.com/example-uri/>example.com/semantic-uri/<http://www.semantic.com/example-uri/>">

<annotation-xml id="eq1-o" encoding="...">...</annotation-xml>

</other>

</math>

Interrelating Expressions and Mathematical Proofs

Is content which interrelates mathematical expressions, e.g. for mathematical proofs, expression “metadata” or is it another variety of content?

Clipboarding

Some preliminary thoughts on clipboarding mathematical expressions include a consideration of multipart MIME. A mathematics expression can map to data of type multipart/related which contains multiple, nested contents of type multipart/alternative (https://en.wikipedia.org/wiki/MIME#Multipart_messages). The root part or the main part of the multipart/related message, the part which references content in the other parts, can be the “metadata” content.

Content Negotiation and Semantic Web Formats

In the section Multiple Notations, it was asked whether mathematical notation should be an attribute on “presentation” annotations.

We could also add “notation” to a list of parameters for local and remote content negotiation: encoding, language, quality and notation. Perhaps, the parameters of content negotiation could be extensible. Adding a parameter for “notation” could be as simple as utilizing a URI in content returned during agent-driven or reactive content negotiation.

“Unfortunately HTTP leaves the format of the list of representations and metadata along with selection mechanisms unspecified.” – https://en.wikipedia.org/wiki/Content_negotiation

If the format of the content returned accompanying an HTTP 300 or 406 during agent-driven or reactive content negotiation were a Semantic Web format, e.g. RDF/XML, then these matters would be tractable. We could readily add “notation” as a content negotiation parameter.

We could specify the use of Semantic Web formats, e.g. RDF/XML (application/rdf+xml), during agent-driven or reactive content negotiation. This would facilitate extensibility in terms of the parameters of content negotiation (adding “notation” to encoding, language and quality).



Best regards,
Adam Sobieski