Re: [ISSUE 34] Potential problem with high-level quality issues

I assume you mean a matching in processes like

1) Source content
2) MT
3) quality check
4) Automatic revision of output from 2), taking the results from 3) into
account

To be able to do 4), one would need to define in a machine readable manner
what kind of information e.g. a "grammar" error provides.

Felix

2012/8/1 Jan Nelson <Jan.Nelson@microsoft.com>

>  Are your assumptions including MT interop where the matching is through
> automation?
>
>
>
> Thanks,
>
>
>
> Jan
>  ------------------------------
> *From:* Arle Lommel [arle.lommel@dfki.de]
> *Sent:* Wednesday, August 01, 2012 1:09 PM
> *To:* Jan Nelson
> *Cc:* Felix Sasaki; Phil Ritchie; Multilingual Web LT Public List
>
> *Subject:* Re: [ISSUE 34] Potential problem with high-level quality issues
>
>   Hi Jan,
>
>  I'm not too concerned about that since the boundaries of the category
> list I made are pretty clear and there are principles to guide selection in
> the case of ambiguities.
>
> --
> Arle Lommel
> Berlin, Germany
> Skype: arle_lommel
> Phone (US): +1 707 709 8650 <#138e3d8cea9acdf3_>
>
>  Sent from a mobile device. Please excuse any typos.
>
> On Aug 1, 2012, at 18:48, Jan Nelson <Jan.Nelson@microsoft.com> wrote:
>
>    Thoughts about how to map "comparable"?  That fuzziness is where I
> think the QA issue is toughest.
>
>
>
> Jan
>  ------------------------------
> *From:* Felix Sasaki [fsasaki@w3.org]
> *Sent:* Wednesday, August 01, 2012 9:35 AM
> *To:* Phil Ritchie
> *Cc:* Arle Lommel; Multilingual Web LT Public List
> *Subject:* Re: [ISSUE 34] Potential problem with high-level quality issues
>
>  Sure - it's just that we would need to build test cases around this
> mapping table. What happens in the Okapi / QADistiller row doesn't need to
> be the same, but at some general level comparable.
>
>  Felix
>
> 2012/8/1 Phil Ritchie <philr@vistatec.ie>
>
>> Can we introduce some indirection here:
>>
>> Can we make all 26 values normative and publish an annex which to the
>> best of our abilities would list mappings between ours and any
>> publishing/consuming tool?
>>
>> e.g.
>>
>>   ITS 2.0 Okapi QADistiller
>>
>> Phil.
>>
>>
>>
>>
>>
>> From:        Arle Lommel <arle.lommel@dfki.de>
>> To:        Multilingual Web LT Public List <
>> public-multilingualweb-lt@w3.org>,
>> Date:        01/08/2012 16:02
>> Subject:        Re: [ISSUE 34] Potential problem with high-level quality
>> issues
>> ------------------------------
>>
>>
>>
>> Hello all,
>>
>> I was discussing the high-level quality issues with Felix this morning
>> and we have an issue. If they are to be normative, then we will need to
>> find at least two interoperable implementations *for each value*, not
>> just for the mechanism as a whole, and to test those implementations
>> against test cases. While that would not be hard for some like *
>> terminology*, it would be difficult for others like *legal*, because,
>> while they are used in metrics, they are not particularly embedded in tools
>> that would produce or consume ITS 2.0 markup.
>>
>> One solution is to put the issue names in an informative annex and *very
>> strongly recommend* that they be used. That approach is, I realize,
>> unlikely to satisfy Yves, for good reason: if we cannot *know* what
>> values are allowed in that slot, then we cannot reliably expect
>> interoperability. At the same time, if we only go with those values for
>> which we can find two or more interoperable implementations, that list of
>> 26 issues will probably become something like six or eight, thereby leaving
>> future tools that might address the other issues out in the cold.
>>
>> I have to confess that I do not see a solution to this issue right now
>> since we really need the values to be normative but if we cannot test them
>> in fairly short order they cannot be normative. The test cases must be more
>> robust that simply seeing that a tool identifies an issue and passes it on:
>> we also need to see that they do this consistently with each other, which
>> is hard since the set of issues from the various tools only partially
>> overlap.
>>
>> If anyone has any brilliant ideas on how to solve the issue, please feel
>> free to chime in. We're still working on this and hope to find a way to
>> move forward with normative values.
>>
>> Best,
>>
>> Arle
>>
>>
>> ************************************************************
>> This email and any files transmitted with it are confidential and
>> intended solely for the use of the individual or entity to whom they
>> are addressed. If you have received this email in error please notify
>> the sender immediately by e-mail.
>>
>> www.vistatec.com
>> ************************************************************
>>
>
>
>
>  --
> Felix Sasaki
> DFKI / W3C Fellow
>
>


-- 
Felix Sasaki
DFKI / W3C Fellow