Re: On conformance

Hi Christian,

On Fri, 10 Feb 2006 18:20:19 +0900, Lieske, Christian  
<christian.lieske@sap.com> wrote:

> Hi Felix,
>
> Sure. I intend to provide something related to the "term" data category?
> Might it be a good idea in general that the "driver" for each data
> category works on requirements/tests for "his" data category?

We had this action item a while ago

<scribe> ACTION: Christian and Felix to provide examples for tests, and  
see if we can come up with a common set of conformance levels [recorded in  
http://www.w3.org/2006/02/01-i18nits-minutes.html#action09]

I understood this that we provide as many examples as - in each of our  
views - necessary, to test conformance. This is then the start of a  
"bottom up" approach, from tests to conformance levels.

I used below the "translatability" data category, but I could have used  
every other data category as well. So, below is my exaustive list of tests  
for all data categories :)

So I think you should provide all tests which you think which are  
necessary, not only the ones for "terminology".
This might be a very complicated task, *if* you assume a lot of  
conformance levels, and even conformance specific conformance criteria to  
a single data category. So my answer to your question

> Might it be a good idea in general that the "driver" for each data
> category works on requirements/tests for "his" data category?

would be: try to make it as simple as possible, and as uniform as possible  
for all data categories, that is: have possibly only "one" driver for all  
data categories.

Regards, Felix.

>
> Best,
> Christian
>
> -----Original Message-----
> From: public-i18n-its-request@w3.org
> [mailto:public-i18n-its-request@w3.org] On Behalf Of Felix Sasaki
> Sent: Freitag, 10. Februar 2006 02:03
> To: Lieske, Christian; public-i18n-its@w3.org
> Subject: Re: On conformance
>
>
> Hi Christian,
>
>
> Many thanks for your feedback. Just a question: Could you please provide
>
> tests on you own? As we said at last weeks call, we want to start the
> discussion at this low level, so please provide *an exaustive* list of
> the
> tests you are thinking of, and classify them.
>
> Regards, Felix.
>
> On Fri, 10 Feb 2006 02:12:42 +0900, Lieske, Christian
> <christian.lieske@sap.com> wrote:
>
>>
>> Hi Felix,
>>
>> This is great work. Nevertheless, here are some high-level comments
> ...
>>
>> I have got the feeling that we currently do not clearly destinguish
>>
>> 1. conformance clause
>> 2. conformance level
>> 3. conformance testing
>> 4. test suite
>>
>> Here's some prose I would use to talk about these concepts (cf.
>>
> http://www.itl.nist.gov/div897/ctg/conformance/bulletin-conformance.htm)
>> :
>>
>> 1. conformance clause: section in our a document that states which
>> requirement has to be met (e.g. Sec. 7.1.)
>> 2. conformance level: a label attached to a set of conformance clauses
>> 3. conformance testing: verify whether sth. is conformant (e.g. by
>> running an application on a test suite)
>> 4. test suite: used to check whether results produced by an
>> implementation match the expected results
>>
>> Based on these concepts, I would say that what we are currently aiming
>> at is a test suite.
>> Our test suite should provide an initial set of metrics to determine
>> whether or not sth. is
>> conformant (cf. http://www.w3.org/XML/Test/xmlconf-20031210.html).
>>
>> For our test suite I suggest that that each test
>>
>> - has an idenfitier
>> - relates to a requirement/conformance clause
>> - describes its purpose in prose
>> - is classified as binary or output
>> - lists the result which is expected from a conformant
>> application/document
>> (e.g. the string "rejected since ITS markup invalid" or an XML file
> with
>> the contents which should correctly by
>> classified as "to be translated")
>>
>> Felix might know whether we need some kind of official statement that
>> tells people how the test suite is developed/maintained (cf.
>> http://www.w3.org/XML/Test/XMLConformanceTS-Process-20031210.html).
>>
>> Coming back to my original "this is great work". From my
> understanding,
>> Felix already put the most important things
>> in place which would be needed to realize my suggestion.
>>
>> Best regards,
>> Christian
>>
>> -----Original Message-----
>> From: public-i18n-its-request@w3.org
>> [mailto:public-i18n-its-request@w3.org] On Behalf Of Felix Sasaki
>> Sent: Mittwoch, 8. Februar 2006 14:32
>> To: public-i18n-its@w3.org
>> Subject: On conformance
>>
>>
>> Hi all,
>>
>> This is on my action item (or the one of Christian and me) to have
> tests
>>
>> for various conformance levels.
>>
>>
>> I) Testing a schema for conformance as described in sec. 7.1. Example:
>> <!ENTITY % itsmarkup SYSTEM "its.dtd">
>> %itsmarkup;
>> <!ELEMENT book ...>
>> <!ATTLIST book ... %att.datacats.attributes;>
>> each ATTLIST has the att.datacats.attributes entity.
>> Purpose: check the statement "The schema must allow the usage of the
>> attribute group att.datacats at every element which is declared in the
>> schema.".
>>
>> - Testing a schema for conformance as described in sec. 7.2. Example:
>> <!ENTITY % itsmarkup SYSTEM "its.dtd">
>> %itsmarkup;
>> <!ELEMENT book (..., its:documentRules?>
>> <!ATTLIST book ... %att.datacats.attributes;>
>> Purpose: checksthe statement "The schema must allow the usage of the
>> documentRules element in at least one element in the schema".
>>
>> II) Testing interpretation of simple data category attributes as
>> described
>> in sec. 7.1. Example:
>> <book ... its:translate="yes">
>> <head its:translate="no">...</head>
>> </book>
>> This document can be used to test the statement "The interpretation of
>> data category attributes in instance documents must be conformant to
> the
>>
>> data category specific default selections described in Section 4.1:
>> Position and Default Selections of Data Categories." for the
>> translatability data category. The implementations of Sebastian, Yves
>> and
>> me would pass the tests because they select the nodes in accordance
> with
>>
>> the in situ translatability data category definitions.
>>
>>
>> III) Testing Conformance to Dislocated Selection Mechanisms (see sec.
>> 7.2). Example:
>> <book ... its:translate="yes">
>> <head its:translate="no">
>> <its:documentRules>
>> <its:documentRule translate="yes" translateSelector="//p/@comment"/>
>> </its:documentRules>
>> ...</head>
>> </book>
>> This document can be used to test the statement "An application which
>> processes ITS elements and attributes must  process the selection
>> mechanisms described in Section 3.5: Precedence between Selections".
> The
>>
>> implementations of Sebastian, Yves and me would pass the tests because
>> they select the nodes in accordance with the insitu and dislocated
>> translatability data category definitions, and they take precedence of
>> selection mechanisms into account.
>>
>>
>> Results of test:
>>
>> - test type I: Result is a list of schemas which encompass the markup
>> for
>> in situ usage of data categories. We already have enough schemas to
> pass
>>
>> such tests, I would say. But it cannot hurt to have more.
>>
>> - test type II and III: Results are "properly selected nodes".
>> "Properly"
>> can be checked via visualization (as in the case of Sebastians
>> implementation) or as a list of nodes (as in my case, or - I guess -
>> Yves
>> case).
>>
>> Looking forward for your feedback.
>>
>> - Felix
>>
>>
>
>
>

Received on Friday, 10 February 2006 09:50:44 UTC