Fwd: [action 265] data category specific confidence scores

Hi all,
To try and wrap up this point:

_Summary  of Discussion so far:_
1) text analytics annotation was proposed as a way of offering a 
confidence score for text analytics results. As with mtconfidence score, 
the tools annotaiton is now covered by the itsTool feature, but the 
proposal for confidence scores remains

2) Marcis pointed out, using real world terminology use cases, that we 
may have several annotations operating on the same fragment, so applying 
a confidence score to different text analytics annotations with a single 
data category won't work in these cases because of complete override.

Also, if we used text analytics annotation with annotation from other 
data categories we are breaking our 'no dependencies between data 
category rules'.

3) We could overcome the complete override problems using standoff mark 
up as in loc quality issue and provenance. But as confidence score would 
be different for each annotated fragment, that would result in very big 
stand-off records, and we would still be breaking the data cat 
dependencies rule. So this doesn't seem a realistic option

4) so the suggestion discussed in Lyon was to drop  text analytics 
annotation altogether as a separate data category and focus on adding 
confidence attributes to the existing data categories that would benefit 
from it.

so.....

_Proposal:_
I therefore suggest the following and we need your feedback by friday 
16th Nov so we can wrap this up on the monday call!

For those extended with confidence score (terminology, disambiguation) 
please express your support and any comments by friday - if we don't 
receive any we will definitely drop these suggestions. Marcis, Tadej in 
particular, please consider review these.

For exclusions (domain, localizationQualityissue), this is your last 
chance to counter-argue in favour of including, otherwise assume these 
are dropped also.

i) confidence for terminology: as suggested by Marcis 
(http://lists.w3.org/Archives/Public/public-multilingualweb-lt/2012Nov/0028.html), 
revised data category as word revisions attached (addition to local 
definition, note on its-tools and example  38)

ii) confidence for disambiguation: revised data category as word 
revisions attached (addition to local definition, note on its-tools and 
ex 52)

iii) domain: I suggest _excluding _this as an annotation to which we 
attach a confidence score. Its not clear that the use of text analytics 
to identify domain, while feasible, actually represents a real use case 
for interoperability mark-up. If use it would probably be internalized 
by the MT engine. Also, since there are multiple domain values the 
semantics of a single confidence score is unclear.

iv) localizationQualityIssue: i suggest also excluding this as an 
annotation to which we attach confidence scores. The use of statistical 
text analytics doesn't seem common for QA tasks. One exception is the 
recent innovation by digital lingusitics whose Review Sentinel product 
ranks translation but a TA assessment for QA purposes - but this 
innovative and not current practice, so its probably not yet a concrete 
use case.

cheers,
Dave