Re: On conformance

Hi Christian,


Many thanks for your feedback. Just a question: Could you please provide  
tests on you own? As we said at last weeks call, we want to start the  
discussion at this low level, so please provide *an exaustive* list of the  
tests you are thinking of, and classify them.

Regards, Felix.

On Fri, 10 Feb 2006 02:12:42 +0900, Lieske, Christian  
<christian.lieske@sap.com> wrote:

>
> Hi Felix,
>
> This is great work. Nevertheless, here are some high-level comments ...
>
> I have got the feeling that we currently do not clearly destinguish
>
> 1. conformance clause
> 2. conformance level
> 3. conformance testing
> 4. test suite
>
> Here's some prose I would use to talk about these concepts (cf.
> http://www.itl.nist.gov/div897/ctg/conformance/bulletin-conformance.htm)
> :
>
> 1. conformance clause: section in our a document that states which
> requirement has to be met (e.g. Sec. 7.1.)
> 2. conformance level: a label attached to a set of conformance clauses
> 3. conformance testing: verify whether sth. is conformant (e.g. by
> running an application on a test suite)
> 4. test suite: used to check whether results produced by an
> implementation match the expected results
>
> Based on these concepts, I would say that what we are currently aiming
> at is a test suite.
> Our test suite should provide an initial set of metrics to determine
> whether or not sth. is
> conformant (cf. http://www.w3.org/XML/Test/xmlconf-20031210.html).
>
> For our test suite I suggest that that each test
>
> - has an idenfitier
> - relates to a requirement/conformance clause
> - describes its purpose in prose
> - is classified as binary or output
> - lists the result which is expected from a conformant
> application/document
> (e.g. the string "rejected since ITS markup invalid" or an XML file with
> the contents which should correctly by
> classified as "to be translated")
>
> Felix might know whether we need some kind of official statement that
> tells people how the test suite is developed/maintained (cf.
> http://www.w3.org/XML/Test/XMLConformanceTS-Process-20031210.html).
>
> Coming back to my original "this is great work". From my understanding,
> Felix already put the most important things
> in place which would be needed to realize my suggestion.
>
> Best regards,
> Christian
>
> -----Original Message-----
> From: public-i18n-its-request@w3.org
> [mailto:public-i18n-its-request@w3.org] On Behalf Of Felix Sasaki
> Sent: Mittwoch, 8. Februar 2006 14:32
> To: public-i18n-its@w3.org
> Subject: On conformance
>
>
> Hi all,
>
> This is on my action item (or the one of Christian and me) to have tests
>
> for various conformance levels.
>
>
> I) Testing a schema for conformance as described in sec. 7.1. Example:
> <!ENTITY % itsmarkup SYSTEM "its.dtd">
> %itsmarkup;
> <!ELEMENT book ...>
> <!ATTLIST book ... %att.datacats.attributes;>
> each ATTLIST has the att.datacats.attributes entity.
> Purpose: check the statement "The schema must allow the usage of the
> attribute group att.datacats at every element which is declared in the
> schema.".
>
> - Testing a schema for conformance as described in sec. 7.2. Example:
> <!ENTITY % itsmarkup SYSTEM "its.dtd">
> %itsmarkup;
> <!ELEMENT book (..., its:documentRules?>
> <!ATTLIST book ... %att.datacats.attributes;>
> Purpose: checksthe statement "The schema must allow the usage of the
> documentRules element in at least one element in the schema".
>
> II) Testing interpretation of simple data category attributes as
> described
> in sec. 7.1. Example:
> <book ... its:translate="yes">
> <head its:translate="no">...</head>
> </book>
> This document can be used to test the statement "The interpretation of
> data category attributes in instance documents must be conformant to the
>
> data category specific default selections described in Section 4.1:
> Position and Default Selections of Data Categories." for the
> translatability data category. The implementations of Sebastian, Yves
> and
> me would pass the tests because they select the nodes in accordance with
>
> the in situ translatability data category definitions.
>
>
> III) Testing Conformance to Dislocated Selection Mechanisms (see sec.
> 7.2). Example:
> <book ... its:translate="yes">
> <head its:translate="no">
> <its:documentRules>
> <its:documentRule translate="yes" translateSelector="//p/@comment"/>
> </its:documentRules>
> ...</head>
> </book>
> This document can be used to test the statement "An application which
> processes ITS elements and attributes must  process the selection
> mechanisms described in Section 3.5: Precedence between Selections". The
>
> implementations of Sebastian, Yves and me would pass the tests because
> they select the nodes in accordance with the insitu and dislocated
> translatability data category definitions, and they take precedence of
> selection mechanisms into account.
>
>
> Results of test:
>
> - test type I: Result is a list of schemas which encompass the markup
> for
> in situ usage of data categories. We already have enough schemas to pass
>
> such tests, I would say. But it cannot hurt to have more.
>
> - test type II and III: Results are "properly selected nodes".
> "Properly"
> can be checked via visualization (as in the case of Sebastians
> implementation) or as a list of nodes (as in my case, or - I guess -
> Yves
> case).
>
> Looking forward for your feedback.
>
> - Felix
>
>

Received on Friday, 10 February 2006 01:03:12 UTC