- From: CVS User fsasaki <cvsmail@w3.org>
- Date: Tue, 23 Jul 2013 00:42:59 +0000
- To: public-multilingualweb-lt-commits@w3.org
Update of /w3ccvs/WWW/International/multilingualweb/lt/drafts/its20
In directory gil:/tmp/cvs-serv3021
Modified Files:
its20.html its20.odd
Log Message:
edit implementing clarification of mt confidence score, see http://lists.w3.org/Archives/Public/public-multilingualweb-lt/2013Jul/0037.html
--- /w3ccvs/WWW/International/multilingualweb/lt/drafts/its20/its20.html 2013/07/22 14:50:14 1.482
+++ /w3ccvs/WWW/International/multilingualweb/lt/drafts/its20/its20.html 2013/07/23 00:42:58 1.483
@@ -3014,15 +3014,15 @@
<strong class="hl-tag" style="color: #000096"></html></strong></pre></div><p>[Source file: <a href="examples/html5/EX-locQualityRating-html5-local.html" shape="rect">examples/html5/EX-locQualityRating-html5-local.html</a>]</p></div></div></div><div class="div2">
<h3><a href="#contents" shape="rect"><img src="images/topOfPage.gif" align="right" height="26" width="26" title="Go to the table of contents." alt="Go to the table of contents."/></a><a name="mtconfidence" id="mtconfidence" shape="rect"/>8.18 MT Confidence</h3><div class="div3">
<h4><a href="#contents" shape="rect"><img src="images/topOfPage.gif" align="right" height="26" width="26" title="Go to the table of contents." alt="Go to the table of contents."/></a><a name="mtconfidence-definition" id="mtconfidence-definition" shape="rect"/>8.18.1 Definition</h4><p>The <a href="#mtconfidence" shape="rect">MT Confidence</a> data category is used to
- communicate the self-reported confidence score from a machine translation engine for
+ communicate the confidence score from a machine translation engine for
the accuracy of a translation it has provided. It is not intended to provide a score
that is comparable between machine translation engines and platforms. This data
- category does NOT aim to establish any sort of correlation between the self-reported
+ category does NOT aim to establish any sort of correlation between the
confidence score and either human evaluation of MT usefulness, or post-editing
cognitive effort. For harmonization’s sake, MT Confidence is provided as a rational
number in the interval 0 to 1 (inclusive).</p><div class="note"><p class="prefix"><b>Note:</b></p><p>Implementers are expected to interpret the floating-point number and present it to
human and other consumers in a convenient form, such as percentage (0-100%) with up
- to 2 decimal digits, font or background color coding, etc.</p></div><p>This data category can be used for several purposes, including, but not limited
+ to 2 decimal digits, font or background color coding, etc.</p></div><div class="note"><p class="prefix"><b>Note:</b></p><p id="mt-confidence-score-generation-tools">The value provided by the <a href="#mtconfidence" shape="rect">MT Confidence</a> data category can be 1) the quality score of the translation as produced by an MT engine, or 2) a quality estimation score that uses both MT-system-internal features and additional external features. For this reason it is important that <a href="#mtconfidence" shape="rect">MT Confidence</a> provides additional information about the MT engine (via the <code class="its-attr-markup">annotatorsRef</code> attribute, or in HTML the <code class="its-attr-markup">its-annotators-ref</code> attribute). Otherwise the score on it's own is hard to interpret and to reuse. In the case of 2), <a href="#mtconfidence" shape="rect">MT Confidence</a> potentially conveys information about any additional tools that were used in deriving the score.</p></div><p>This data ctegory can be used for several purposes, including, but not limited
to:</p><ul><li><p>Automated prioritising of raw machine translated text for further processing
based on empirically set thresholds.</p></li><li><p>Providing readers, translators, post-editors, reviewers, and proof-readers of machine translated
text with self-reported relative accuracy prediction.</p></li></ul><p>MT confidence scores can be displayed e.g., on websites machine translated on the
@@ -5962,7 +5962,7 @@
<code class="its-attr-markup">lits-line-break-type</code>?</td></tr></tbody></table></div><div class="div1">
<h2><a href="#contents" shape="rect"><img src="images/topOfPage.gif" align="right" height="26" width="26" title="Go to the table of contents." alt="Go to the table of contents."/></a><a name="revisionlog" id="revisionlog" shape="rect"/>H Revision Log (Non-Normative)</h2><p id="changelog-since-20130521">The following log records major changes that have been made to this document since the <a href="http://www.w3.org/TR/2013/WD-its20-20130521/" shape="rect">ITS 2.0 Working Draft 21 May 2013</a>:</p><ol class="depth1"><li><p>Updated all text of the <a href="#html5-withintext-handling" shape="rect">HTML5 defaults for Element Within Text</a> and added example.
See <a href="https://www.w3.org/International/multilingualweb/lt/track/issues/118" shape="rect">issue-118</a>
- and <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/532" shape="rect">action-532</a>.</p></li><li><p>Added a paragraph about mime type submission to <a class="section-ref" href="#its-mime-type" shape="rect">Appendix B: Internationalization Tag Set (ITS) MIME Type</a>, see step 3 (first bullet point) at <a href="http://www.w3.org/2002/06/registering-mediatype#Planned" shape="rect">Register an Internet Media Type for a W3C Spec</a> and <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/251" shape="rect">action-251</a>.</p></li><li><p>Removed company names from various examples, see <a href="http://www.w3.org/Search/Mail/Public/search?type-index=public-multilingualweb-lt-commits&index-type=t&keywords=action-502&search=Search" shape="rect">CVS commits</a> and <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/502" shape="rect">action-502</a>.</p></li><li><p>Reformatting of various examples, see <a href="http://lsts.w3.org/Archives/Public/public-multilingualweb-lt-commits/2013May/0066.html" shape="rect">CVS commit mail</a> and further CVS commit mails with the same send time.</p></li><li><p>Edits related to <a title="" href="#nif-reference" shape="rect">[NIF]</a>, see <a href="https://www.w3.org/International/multilingualweb/lt/track/issues/125" shape="rect">issue-125</a>.</p></li><li><p>Edit related to <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/527" shape="rect">action-527</a>: put the sentence about foreign elements / attributes also in <a class="section-ref" href="#datacategories-defaults-etc" shape="rect">Section 8.1: Position, Defaults, Inheritance, and Overriding of Data Categories</a>.</p></li><li><p>Copy editing of spec, see <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/422" shape="rect">action-422</a>.</p></li><li><p>Edits related to <a href="https://www.w3.org/International/multilingualweb/lt/track/issues/126" shape="rect">issue-126</a> (Minor ssue with quality types listing).</p></li><li><p>Edits related to <a href="https://www.w3.org/International/multilingualweb/lt/track/issues/127" shape="rect">issue-127</a> (Clarifying HTML5 translate and global rules).</p></li><li><p>Edits related to <a href="https://www.w3.org/International/multilingualweb/lt/track/issues/128" shape="rect">issue-128</a> (Various editorial edits + conformance section fixes).</p></li><li><p>Edits related to <a href="https://www.w3.org/International/multilingualweb/lt/track/issues/129" shape="rect">issue-129</a> (sec1-2 editing (non-normative sec)).</p></li><li><p>Fixes about <a title="" href="#rfc2119" shape="rect">[RFC 2119]</a> statements, see <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/540" shape="rect">action-540</a>.</p></li><li><p>Various checks: spelling (using U.S. English), style, grammar, use of ":" etc.</p></li><li><p>Added a non-normative XML Schema for ITS 2.0 to <a class="section-ref" href="#its-schemas" shape="rect">Appendix D: chemas for ITS</a>, see <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/546" shape="rect">action-546</a>.</p></li><li><p>Updated link to quality issue type mappings in <a class="section-ref" href="#lqissue-typevalues" shape="rect">Appendix C: Values for the Localization Quality Issue Type</a>, see <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/543" shape="rect">action-543</a>.</p></li><li><p>Clarify conflicts of <code class="its-elem-markup">param</code> elements with the same name in <a class="section-ref" href="#selection-precedence" shape="rect">Section 5.5: Precedence between Selections</a> and <a class="section-ref" href="#html5-selection-precedence" shape="rect">Section 6.4: Precedence between Selections</a>, see <a href="https://www.w3.org/International/multilingualweb/lt/track/issues/130" shape="rect">issue-130</a>.</p></li><li><p>Updated editors list.</p></li><li><p>Implemented <a href="http://lists.w3.org/Archives/Public/public-i18n-its-ig/013Jul/0041.html" shape="rect">editorial edits</a> in <a class="section-ref" href="#idvalue" shape="rect">Section 8.14: ID Value</a> and <a class="section-ref" href="#locNote-datacat" shape="rect">Section 8.3: Localization Note</a>.</p></li><li><p>Implemented <a href="http://lists.w3.org/Archives/Public/public-multilingualweb-lt-comments/2013Jul/0056.html" shape="rect">clarification</a> about <code class="its-attr-markup">annotatorsRef</code> attribute in <a class="section-ref" href="#its-tool-annotation" shape="rect">Section 5.8: ITS Tools Annotation</a>.</p></li></ol><p id="changelog-since-20130411">The following log records major changes that have been made to this document since the <a href="http://www.w3.org/TR/2013/WD-its20-20130411/" shape="rect">ITS 2.0 Working Draft 11 April 2013</a>:</p><ol class="depth1"><li><p>Added a reference to the <a href="http://www.w3.org/2005/11/its/rdf#" shape="rect">ITS RDF Ontology</a> and an <a href="#its-rdf-ontology-status" shape="rect">explanatory note</a> aboutits status to <a class="section-ref" href="#conversion-to-nif" shape="rect">Section 5.7: Conversion to NIF</a>, see <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/514" shape="rect">action-514</a>.</p></li><li><p>Updated <a class="section-ref" href="#conversion-to-nif" shape="rect">Section 5.7: Conversion to NIF</a> to reflect <a href="http://www.w3.org/2013/05/08-mlw-lt-minutes#item04" shape="rect">MLW-LT May 2013 f2f discussion</a>: <code>nif:occursIn</code> has changed to <code>nif:sourceUrl</code>, and <code>nif:convertedFrom</code> replaces <code>itsrdf:xpath2nif</code>. See <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/517" shape="rect">action-517</a>.</p></li><li><p>Added <a href="#local-approach-not-applicable-to-attributes" shape="rect">a note</a> to <a class="section-ref" href="#basic-concepts-selection-local" shape="rect">Section 2.2.1: Local Approach</a> expressing that local selection does not apply to attributes,
+ and <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/532" shape="rect">action-532</a>.</p></li><li><p>Added a paragraph about mime type submission to <a class="section-ref" href="#its-mime-type" shape="rect">Appendix B: Internationalization Tag Set (ITS) MIME Type</a>, see step 3 (first bullet point) at <a href="http://www.w3.org/2002/06/registering-mediatype#Planned" shape="rect">Register an Internet Media Type for a W3C Spec</a> and <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/251" shape="rect">action-251</a>.</p></li><li><p>Removed company names from various examples, see <a href="http://www.w3.org/Search/Mail/Public/search?type-index=public-multilingualweb-lt-commits&index-type=t&keywords=action-502&search=Search" shape="rect">CVS commits</a> and <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/502" shape="rect">action-502</a>.</p></li><li><p>Reformatting of various examples, see <a href="http://lsts.w3.org/Archives/Public/public-multilingualweb-lt-commits/2013May/0066.html" shape="rect">CVS commit mail</a> and further CVS commit mails with the same send time.</p></li><li><p>Edits related to <a title="" href="#nif-reference" shape="rect">[NIF]</a>, see <a href="https://www.w3.org/International/multilingualweb/lt/track/issues/125" shape="rect">issue-125</a>.</p></li><li><p>Edit related to <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/527" shape="rect">action-527</a>: put the sentence about foreign elements / attributes also in <a class="section-ref" href="#datacategories-defaults-etc" shape="rect">Section 8.1: Position, Defaults, Inheritance, and Overriding of Data Categories</a>.</p></li><li><p>Copy editing of spec, see <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/422" shape="rect">action-422</a>.</p></li><li><p>Edits related to <a href="https://www.w3.org/International/multilingualweb/lt/track/issues/126" shape="rect">issue-126</a> (Minor ssue with quality types listing).</p></li><li><p>Edits related to <a href="https://www.w3.org/International/multilingualweb/lt/track/issues/127" shape="rect">issue-127</a> (Clarifying HTML5 translate and global rules).</p></li><li><p>Edits related to <a href="https://www.w3.org/International/multilingualweb/lt/track/issues/128" shape="rect">issue-128</a> (Various editorial edits + conformance section fixes).</p></li><li><p>Edits related to <a href="https://www.w3.org/International/multilingualweb/lt/track/issues/129" shape="rect">issue-129</a> (sec1-2 editing (non-normative sec)).</p></li><li><p>Fixes about <a title="" href="#rfc2119" shape="rect">[RFC 2119]</a> statements, see <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/540" shape="rect">action-540</a>.</p></li><li><p>Various checks: spelling (using U.S. English), style, grammar, use of ":" etc.</p></li><li><p>Added a non-normative XML Schema for ITS 2.0 to <a class="section-ref" href="#its-schemas" shape="rect">Appendix D: chemas for ITS</a>, see <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/546" shape="rect">action-546</a>.</p></li><li><p>Updated link to quality issue type mappings in <a class="section-ref" href="#lqissue-typevalues" shape="rect">Appendix C: Values for the Localization Quality Issue Type</a>, see <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/543" shape="rect">action-543</a>.</p></li><li><p>Clarify conflicts of <code class="its-elem-markup">param</code> elements with the same name in <a class="section-ref" href="#selection-precedence" shape="rect">Section 5.5: Precedence between Selections</a> and <a class="section-ref" href="#html5-selection-precedence" shape="rect">Section 6.4: Precedence between Selections</a>, see <a href="https://www.w3.org/International/multilingualweb/lt/track/issues/130" shape="rect">issue-130</a>.</p></li><li><p>Updated editors list.</p></li><li><p>Implemented <a href="http://lists.w3.org/Archives/Public/public-i18n-its-ig/013Jul/0041.html" shape="rect">editorial edits</a> in <a class="section-ref" href="#idvalue" shape="rect">Section 8.14: ID Value</a> and <a class="section-ref" href="#locNote-datacat" shape="rect">Section 8.3: Localization Note</a>.</p></li><li><p>Implemented <a href="http://lists.w3.org/Archives/Public/public-multilingualweb-lt-comments/2013Jul/0056.html" shape="rect">clarification</a> about <code class="its-attr-markup">annotatorsRef</code> attribute in <a class="section-ref" href="#its-tool-annotation" shape="rect">Section 5.8: ITS Tools Annotation</a>.</p></li><li><p>Implemented a <a href="http://lists.w3.org/Archives/Public/public-multilingualweb-lt/2013Jul/0037.html" shape="rect">clarification</a> in <a class="section-ref" href="#mtconfidence-definition" shape="rect">Section 8.18.1: Definition</a> and added <a href="#mt-confidence-score-generation-tools" shape="rect">a note</a> that <a href="#mtconfidence" shape="rect">MT Confidence</a> information can be produced both by MT systems and other tool.</p></li></ol><p id="changelog-since-20130411">The following log records major changes that have been made to this document since the <a href="http://www.w3.org/TR/2013/WD-its20-20130411/" shape="rect">ITS 2.0 Working Draft 11 April 2013</a>:</p><ol class="depth1"><li><p>Added a reference to the <a href="http://www.w3.org/2005/11/its/rdf#" shape="rect">ITS RDF Ontology</a> and an <a href="#its-rdf-ontology-status" shape="rect">explanatory note</a> about its status to <a class="section-ref" href="#conversion-to-nif" shape="rect">Section 5.7: Conversion to NIF</a>, see <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/514" shape="rect">action-514</a>.</p></li><li><p>Updated <a class="section-ref" href="#conversion-to-nif" shape="rect">Section 5.7: Conversion to NIF</a> to reflect <a href="http://www.w3.org/2013/05/08-mlw-lt-minutes#item04" shape="rect">MLW-LT May 2013 f2f discussion</a>: <code>nif:occursIn</code> has changed to <code>nif:sourceUrl</code>, and <code>nif:convertedFro</code> replaces <code>itsrdf:xpath2nif</code>. See <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/517" shape="rect">action-517</a>.</p></li><li><p>Added <a href="#local-approach-not-applicable-to-attributes" shape="rect">a note</a> to <a class="section-ref" href="#basic-concepts-selection-local" shape="rect">Section 2.2.1: Local Approach</a> expressing that local selection does not apply to attributes,
see <a href="https://www.w3.org/International/multilingualweb/lt/track/issues/98" shape="rect">issue-98</a>.</p></li><li><p>Added a <a href="#qa-issue-types-mappings" shape="rect">clarification</a> about the role of mappings from tools to quality issue types,
see <a href="https://www.w3.org/International/multilingualweb/lt/track/actions/493" shape="rect">action-493</a>.</p></li><li><p>Updated the definition of the regular expression to use in the <a class="section-ref" href="#allowedchars" shape="rect">Section 8.19: Allowed Characters</a> data category,
see <a href="https://www.w3.org/International/multilingualweb/lt/track/issues/67" shape="rect">issue-67</a>.</p></li><li><p>Updated explanation of usage in <a title="HTML5" href="#html5" shape="rect">[HTML5]</a> to reflect discussion on <a title="HTML5" href="#html5" shape="rect">[HTML5]</a> defaults, see <a href="https://www.w3.org/International/multilingualweb/lt/track/issues/89" shape="rect">issue-89</a>, <a href="https://www.w3.org/International/multilingualweb/lt/track/issues/97" shape="rect">issue-97</a> and <a href="https://www.w3.org/International/multilingualweb/lt/track/issues/118" shape="rect">issue-118</a>.</p></li><li><p>Clarified <a href="#provenance-records-in-html5-constraint" shape="rect">provenance</a> and
--- /w3ccvs/WWW/International/multilingualweb/lt/drafts/its20/its20.odd 2013/07/22 14:50:14 1.490
+++ /w3ccvs/WWW/International/multilingualweb/lt/drafts/its20/its20.odd 2013/07/23 00:42:59 1.491
@@ -4461,10 +4461,10 @@
<div xml:id="mtconfidence-definition">
<head>Definition</head>
<p>The <ref target="#mtconfidence">MT Confidence</ref> data category is used to
- communicate the self-reported confidence score from a machine translation engine for
+ communicate the confidence score from a machine translation engine for
the accuracy of a translation it has provided. It is not intended to provide a score
that is comparable between machine translation engines and platforms. This data
- category does NOT aim to establish any sort of correlation between the self-reported
+ category does NOT aim to establish any sort of correlation between the
confidence score and either human evaluation of MT usefulness, or post-editing
cognitive effort. For harmonization’s sake, MT Confidence is provided as a rational
number in the interval 0 to 1 (inclusive).</p>
@@ -4473,6 +4473,9 @@
human and other consumers in a convenient form, such as percentage (0-100%) with up
to 2 decimal digits, font or background color coding, etc.</p>
</note>
+ <note>
+ <p xml:id="mt-confidence-score-generation-tools">The value provided by the <ref target="#mtconfidence">MT Confidence</ref> data category can be 1) the quality score of the translation as produced by an MT engine, or 2) a quality estimation score that uses both MT-system-internal features and additional external features. For this reason it is important that <ref target="#mtconfidence">MT Confidence</ref> provides additional information about the MT engine (via the <att>annotatorsRef</att> attribute, or in HTML the <att>its-annotators-ref</att> attribute). Otherwise the score on it's own is hard to interpret and to reuse. In the case of 2), <ref target="#mtconfidence">MT Confidence</ref> potentially conveys information about any additional tools that were used in deriving the score.</p>
+ </note>
<p>This data category can be used for several purposes, including, but not limited
to:</p>
<list>
@@ -5996,6 +5999,7 @@
<item>Updated editors list.</item>
<item>Implemented <ref target="http://lists.w3.org/Archives/Public/public-i18n-its-ig/2013Jul/0041.html">editorial edits</ref> in <ptr target="#idvalue" type="specref"/> and <ptr target="#locNote-datacat" type="specref"/>.</item>
<item>Implemented <ref target="http://lists.w3.org/Archives/Public/public-multilingualweb-lt-comments/2013Jul/0056.html">clarification</ref> about <att>annotatorsRef</att> attribute in <ptr target="#its-tool-annotation" type="specref"/>.</item>
+ <item>Implemented a <ref target="http://lists.w3.org/Archives/Public/public-multilingualweb-lt/2013Jul/0037.html">clarification</ref> in <ptr target="#mtconfidence-definition" type="specref"/> and added <ref target="#mt-confidence-score-generation-tools">a note</ref> that <ref target="#mtconfidence">MT Confidence</ref> information can be produced both by MT systems and other tools.</item>
</list>
<p xml:id="changelog-since-20130411">The following log records major changes that have been made to this document since the <ref target="http://www.w3.org/TR/2013/WD-its20-20130411/">ITS 2.0 Working Draft 11 April 2013</ref>:</p>
<list type="ordered">
Received on Tuesday, 23 July 2013 00:43:00 UTC