Re: What *is* a spec test? (musings on feature interop tests)

On Nov 27, 2013, at 1:57 AM, Mihai Balan <mibalan@adobe.com> wrote:

> Answers inline.
> 
> Mihai Balan | Quality Engineer @ Web Engine team |  mibalan@adobe.com |
> +4-031.413.3653 / x83653 | Adobe Systems Romania
> 
> 
> 
> On 11/26/13 9:11 PM, "Gérard Talbot" <css21testsuite@gtalbot.org> wrote:
> 
>> Le 2013-11-26 09:15, Mihai Balan a écrit
> 
>>> [snip]
>>> 
>>> Or does the build step for test suites includes all the tests that
>>> reference a particular spec (section)?
>> 
>> Yes. Peter Linss should confirm this. In your example, it should create
>> a test in regions and in fragmentation.
> 
> Makes sense and it sounds like good news :)

Confirmed, when building a spec test suite, the build system includes all tests that have help links to that spec.

> 
> However, what about if some of the specs linked to are not CSS specs?
> Would the test still be part of the other spec test suite? (e.g. if I'm
> writing a test for CSS Regions and ShadowDOM)

As I understand the current situation, no. However the metadata will support that if future tooling for the WPT repo allows.

> 
>> 
>> [snip]
>>> @ Alan
>>> ------
>>> Having read the section on Interoperability in the Overview
>> 
>> Can you pinpoint the url where you read this?
> 
> It was in an earlier email from Alan. The document would be over at [1].
> 
> 
> [snip]
>> Many non-basic tests involve more than 1 property in action, being
>> involved in a test. In my mind, property interaction (or property
>> inter-relation) and interoperability are 2 distinct words with 2
>> distinct meanings.
> [snip]
>> 
>> This specification is related to other specifications as described in
>> the references section. In addition, it is related to the following
>> specifications:
>> 
>>    CSS Fragmentation Module Level 3 [CSS3-BREAK]. This module defines
>> the rules for fragmenting content over multiple containers and applies
>> to CSS regions in addition to applying to multi-column and paged media.
>> "
>> CSS3 Regions, §9. Relation to other specifications
>> http://www.w3.org/TR/css3-regions/#relation-to-other-specifications
>> 
>> Therefore, it seems normal to have tests that will involve other
>> properties and other parts of other specs.
> 
> OK, so the main takeaway here is that, apart from my awkward overloading
> of the "interoperability" concept, these tests are useful and they should
> not be separated rather be tracked as parts of test suites they appear in.
> Did I get this right?

Yes. Note that so long as the tests are in the CSS repo, the build system should find them, so their location isn't all that crucial (and we're going to re-org the repo soon anyway).

> 
> Thanks a lot,
> Mihai
> 
> [1] http://www.w3.org/Style/CSS/Test/Overview.en.html
> 
> 
>