Request on QA for ITS 2.0

Szervusztok, Gábor és István!

I hope this mail finds you well. As part of work at the W3C on ITS 2.0 on standardized markup for localization quality data, I have been asked to follow up with some developers of quality assurance tools to get feedback. While we are still working on details, there is one piece I was asked to approach you about as the drivers of XLIFF:doc’s quality model.

(If you want a lot of details about the work going into this, see https://www.w3.org/International/multilingualweb/lt/track/issues/34)

Basically, to support a broad-brush interoperability between tools we decided to create a list of "top-level" quality issue categories. They are not intended to replace internal categories, or describe systems in full detail. Rather the categories are intended to give various tools some idea of what sort of quality issue has been detected by a tool, even if the precise details may not match the consuming tool’s own ontology of issues.

What we are asking you to do is read the following extract from a rough draft of the description of the locQualityType (localization quality type) attribute value that is used for this categorization and let us know if (a) the values proposed would meet your needs (both for Kilgray and for XLIFF:doc) and (b) if the proposed mapping of XLIFF:doc quality categories (in the appendix draft further down) make sense to you. The hardest one for us to map was the XLIFF:doc internationalization category since it is very broad in its meaning, considerably broader than the corresponding category in this list. (The text also refers to locQualityCode, which is used to pass on internal codes, which may be considerably more detailed than the locQualityType values to which they are mapped.)

So, if you can spare a little time to look this over and provide your thoughts, we would appreciate it.

(By the way, I will be in Hungary in a few weeks. Unfortunately I won't be anywhere near Gyula—I'm going to Nógrád megye for a week, so I'm headed the wrong way—but if you would happen to be in Budapest on Monday, August 20, I could meet with you to discuss this in person.)

Best,

Arle
DRAFT CONTENT on locQualityType

The locQualityType subcategory is intended to provide a basic level of interoperability between differing localization quality assurance systems. It provides a list of 26 high-level quality issue types common in automatic and human localization quality assessment. Localization quality assessment tools can map their internal categories to these categories in order to exchange information about the kinds of issues they identify and take intelligent and appropriate action even if another tool does not know the specific issues identified by the generating tool

NOTE: Tools implementing the loc-quality data category that use the locQualityCode subcategory SHOULD also use the locQualityCode to provide this level of interoperability.

The values listed in the following table are allowed for locQualityType. Note that tools implementing locQualityTypeare not required to check for or flag issues for all or any of the types. However, if they implement locQualityType, the values they produce for the attribute MUST match one of the values provided in this table and MUST be semantically accurate. If a tool can map its internal values to these categories it MUST do so and may not use the value of other, which is reserved strictly for values that cannot be mapped to these values.

Allowable values for locQualityType
Value	Description	Examples	Notes
terminology	An incorrect term or a term from the wrong domain was used or terms are used inconsistently	
The localization had Pen Drivewhen corporate terminology specified thatUSB Stick was to be used; The localization inconsistently used Start andBegin.
Should not be confused with the ITS terminology data category.
mistranslation	The content of the target mistranslates the content of the source	
The English source reads “An ape succeeded in grasping a banana lying outside its cage with the help of a stick” but the Italian translation reads “l’ape riuscì a prendere la banana posta tuori dall sua gabbia aiutandosi con un bastone” (“Abeesucceeded…”)
Issues related to translation of specific terms related to the domain or task-specific language should be categorized as terminology issues
omission	Necessary text has been omitted from the localization or source	
One or more segments found in the source that should have been translated are missing in the target
This category should not be used for missing whitespace or formatting codes, but instead should be reserved for linguistic content.
untranslated	Content that should have been translated was left untranslated	
The source segment reads “The Professor said to Smith that he would hear from his lawyer” but the Hungarian localization reads “A professzor azt modta Smithnek, hogy he would hear from his lawyer.”
omission take precedence over untranslated. Omissions are distinct in that they address cases where text is not present, while untranslated address cases where text has been carried from the source untranslated.
addition	The translated text contains inappropriate additions	
The translated text contains a note from the translator to himself to look up a term; the note should have been deleted but was not.
duplication	Content has been duplicated improperly	
A section of the target text was inadvertently copied twice in a copy and paste operation.
inconsistency	The text is inconsistent with itself (NB: not for use with terminology inconsistency)	
The text states that an event happened in 1912 in one location but in another states that it happened in 1812.
grammar	The text contains a grammatical error (including errors of syntax and morphology)	
The text reads “The guidelines says that users should use a static grounding strap.”
legal	The text is legally problematic (e.g., it is specific to the wrong legal system)	
The localized text is intended for use in Thailand but includes U.S. regulatory notices.
A text translated into German contains comparative advertising claims that are not allowed by German law
register	The text is written in the wrong linguistic register of uses slang or other language variants inappropriate to the text	
A financia text translated into U.S. English refers to dollars as “bucks”.
locale-specific-content	The localization contains content that does not apply to the locale for which it was prepared	
A text translated for the Japanese market contains call center numbers in Texas and refers to special offers available only in the U.S.
Legally inappropriate material should be classified as legal
locale-violation	Text violates norms for the intended locale	
A text localized into German has dates in YYYY-MM-DD format instead of in DD.MM.YYYY
A translated text uses American-style foot and inch measurements instead of centimeters.
style	The text contains stylistic errors	
Company style dictates that all individuals be referred to as Mr. or Ms. with a family name, but the text refers to “Jack Smith”.
characters	The text contains characters that are garbled or incorrect or that are not used in the language in which the content appears	
the text should have a
but instead has a ¥ sign
A text translated into German omits the umlauts over ü, ö, and ä
A Japanese localization contains characters like మ and ఊ (from Telugu)
misspelling	The text contains a misspelling	
A German text misspells the word Zustellungas Zustellüng
typographical	The text has typographical errors such as omitted/incorrect punctuation, incorrect capitalization, etc.	
An English localization has the following sentence: The man whom, we saw, was in the Military and carried it’s insignias
formatting	The text is formatted incorrectly	
Warnings in the target text are supposed to be set in italic face, but instead appear in bold face
Margins of the text are narrower than specified
inconsistent-entities	The source and target text contain different named entities (dates, times, place names, individual names, etc.)	
The nameThaddeus Cahillappears in an English source but is rendered as Tamaš Cahillin the Czech version
The date February 9, 2007 appears in the source but the translated text has “2. September 2007.”
numbers	Numbers are inconsistent between source and target	
The source text states that an object is 120 cm long, but the target text says it is 129 cm. long.
Some tools may correct for differences in units of measurement to reduce false positives
markup	There is an issue related to markup or a mismatch in markup between source and target	
The source segment has five markup tags but the target has only two
An opening tag in the localization is missing a closing tag
pattern-problem	The text fails to match a pattern that defines allowable content (or matches one that defines non-allowable content)	
The quality checking tool disallows the regular expression pattern ['"”’][\.,] but the translated text contains A leading “expert”, a political hack, claimed otherwise.
whitespace	There is a mismatch in whitespace between source and target content	
A source segment starts with six space characters but the corresponding target segment has two non-breaking spaces at the start.
internationalization	There is an error related to the internationalization of content	
A line of programming code has embedded language-specific strings
A user interface element leaves no room for text expansion
A form allows only for U.S.-style postal addresses and expects five digit U.S. ZIP codes
There are many kinds of internationalization errors of various types. This category is therefore very heterogeneous in what it can refer to.
length	There is a significant difference in source and target length	
The translation of a segment is five times as long as the source
What constitutes a “significant” difference in length is determined by the model referred to in the locQualityProfile
uncategorized	The issue has not been categorized	
A new version of a tool returns information on an issue that has not been previously checked and that is not yet classified
This category has to uses: (1) a tool can use it to pass through quality data from another tool in cases where the issues from the other tool are not classified (for example, a localization quality assurance tool interfaces with a third-party grammar checker); (2) a tool’s issues are not yet assigned to categories, and, until an updated assignment is made, they may be listed as uncategorized. In the latter case it is recommended that issues be assigned to appropriate categories as soon as possible since uncategorized does not foster interoperability.
other	Any issue that cannot be assigned to any values listed above.		This category allows for the inclusion of any issues not included in the previously listed values. This value MUST not be used for any tool- or model-specific issues that can be mapped to the values listed above. In addition, this value is not synonymous with uncategorized in that uncategorized issues may be assigned to another precise value, while other issues cannot.

If a metric has an “miscellaneous” or “other” category, it should be mapped to this value even if the specific instance of the issue might be mapped to another category.
Annex X: Mapping of Tool-Specific Quality Codes tolocQualityType Values (Non-Normative)

This Annex is informative.

The following table provides mappings of native quality assurance issue codes for a number of common localization quality tools to locQualityType values. Tool developers are free to map their own issue codes to the locQualityTypevalues and are encouraged to make their mappings publicly available. Tools that produce ITS 2.0 loc-qualitymarkup should ensure that the output of their tools matches any publicly available mappings they may produce.

Note: These mappings are provides for example only. In the event of discrepancy between the mapping published by a developer and this annex, the statements from the developer take precedence over this annex.

locQualityType value	Tool/Metric-Specific Values
CheckMate	xliff:doc	QA Distiller	SAE J2450	LISA QA Model (UI)	LISA QA Model (doc)* - language only**
terminology	
TERMINOLOGY
terminology
Consistency
Tag-aware
ID-aware
Untranslatables
wrong term
Terminology
Glossary adherence
Abbreviations
Context
mistranslation			
Mistranslation
Accuracy
Semantics
Accuracy
omission	
MISSING_TARGETTU
MISSING_TARGETSET
EMPTY_TARGETSEG
EMTPY_SOURCESEG
omission
Empty translations
omission
Omissions
untranslated	
TARGET_SAME_AS_SOURCE
Forgotten translations
Skipped translations
Partial translations
Incomplete translation
addition	
EXTRA_TARGETSEG
Additions
duplication	Not addressed in any of these metrics. It may be possible to treat this as a case of addition.
inconsistency		
inconsistency
Source
Target
Consistency
grammar				
syntactic error
word structure or agreement error
Grammar
legal	Not addressed in any of these metrics. However, legal compliance checking is a big deal for regulated industries and forms a core part of their metrics.
register						
Register/tone
Language variants/slang
locale-specific-content						
Local suitability
locale-violation					
Country
Country standards
style					
Style
General style
Company standards
characters	
ALLOWED_CHARACTERS
Corrupt characters, source
Corrupt characters, target
Double/Single Size
Character formatting
misspelling				
misspelling
Spelling
typographical		
punctuation
Consecutive punctuation marks
End of segment punctuation
Non-matching pairs (brackets)
Leading bracket outside of TU
Different count(brackets)
Initial capitalization
Entire capitalization
Non-matching pairs (quotation marks)
Incorrect type (quotation marks)
Different count (quotation marks)
punctuation error
Punctuation marks
formatting					
TOC
Index
Layout
Typography
Graphics
Call Outs and Captions
Alignment
Sizing
Truncation/overlap
(Numerous)
inconsistent-entities		
date
time
numbers		
number
Number values
Incorrect type (measurements)
Check conversions (measurements)
markup	
MISSING_CODE
EXTRA_CODE
SUSPECT_CODE
tags
pattern-problem	
UNEXPECTED_PATTERN
SUSPECT_PATTERN
pattern
whitespace	
MISSING_LEADINGWS
MISSINGORDIFF_LEADINGWS
EXTRA_LEADINGWS
EXTRAORDIFF_LEADINGWS
MISSING_TRAILINGWS
MISSINGORDIFF_TRAILINGWS
EXTRA_TRAILINGWS
EXTRAORDIFF_TRAILINGWS
Consecutive spaces
Inconsistent leading and trailing spaces
Required/forbidden spaces
Different count (tabs)
Required/forbidden spaces
internationalization		
internationalization
(The examples for this code are broader than the type category here.)

Number formatting
length	
TARGET_LENGTH
uncategorized	
LANGUAGETOOL_ERROR
other		
other
miscellaneous error
Hyper text functionality, jumps, popups
Localizable text
Dialogue functionality
Menu functionality
Hotkeys/accelerators
Jumps/links
(** There are significant discrepancies between the categories in the LISA QA Model software and its documentation. The relationship between the two is unclear, so both are listed here.)

(** The LISA QA Model documentation addresses numerous issues related to software formatting that are outside the scope of the ITS 2.0 loc-quality model. For the sake of conciseness and clarity, these are not listed in this document.)


<eof>