RE: MT Confidence definition

Hi Yves and all,

I fully agree with this extension, but the problem is however that it is
extremely important where the score comes from in this case. It has to be
made known to the steps down the stream.

Therefore, the language should at least read:

"The MT Confidence data category is used to communicate the self-reported
confidence score or confidence score reported by a third part evaluation
system from a machine translation engine of the accuracy of a translation it
has provided."

Here comes another question: if the confidence score comes either from MT
engine, or from a third-party system, where is it reflected?

This could be extremely important in the workflows where we have, for
example, an MT engine that gives self-reported confidence score, and then
there's another evaluation that comes from a different system, the steps
that are down the stream should be able to distinguish where the score comes
from, and between them scores, too.

Regards,
Serge




-----Original Message-----
From: Yves Savourel [mailto:ysavourel@enlaso.com] 
Sent: Wednesday, July 17, 2013 9:30 AM
To: public-multilingualweb-lt@w3.org
Subject: MT Confidence definition

Hi all,

I've noticed a minor text issue in the specification:

For the MT Confidence data category we say:

"The MT Confidence data category is used to communicate the self-reported
confidence score from a machine translation engine of the accuracy of a
translation it has provided."

This is very limiting.

I think it should say:

"The MT Confidence data category is used to communicate the confidence score
of the accuracy of a translation provided by a machine translation."

(and later: "the self-reported confidence score" should be "the reported
confidence score").

There could be cases where the confidence score is provided by another
system than the one that provided the MT candidate. The QuEst project is an
example of this
http://staffwww.dcs.shef.ac.uk/people/L.Specia/projects/quest.html)

Cheers,
-ys

Received on Wednesday, 17 July 2013 07:45:52 UTC