Re: Adding the testharness.js subrepo to the csswg-test repo

On 16/10/13 21:30, Peter Linss wrote:
>>> In other words, we wouldn't get the automatic updating of the
>>> resources directory, as we do know by leaving them separate. Is this
>>> what's really wanted?
>>
>> Yes, for consistency with the main repo.
>
> I have no issue with keeping the csswg repo consistent with the main
> repo. But then we need to know which version of the resources repo the
> main is using to keep in sync.

Not really. You need to upgrade when you have tested that the changes 
don't accidentially break your tests. I don't think anyone should have a 
requirement that the two version are in perfect sync.

> Is there a system for notifications when this changes? or perhaps a tag
> in the resources repo that we can automatically sync to?
>
> We also serve a copy of the resources repo at
> http[s]://test.csswg.org/resources <http://test.csswg.org/resources> so
> that references in the tests work. I also need to be able to keep that
> in sync with the version referred to in WPT rather than the tip.

You need to keep that in sync with the version in your repository. That 
isn't really a harder problem than generally keeping any checkout up to 
date.

>>> I haven't seen any discussion about making the resources directory a
>>> submodule in the main WPT repo, so I'm not sure what the goals of
>>> this change are.
>>
>> It's been a submodule on the main repo for a while now.
>
> That doesn't answer my question.
>
> Was this done only so that people working on local clones of WPT (sort
> of) automatically get the resources directory? or is there something
> more to it?
>
> e.g.: Has there also been a change in the way testharness.js gets used?
> i.e. is it sill always '/resources/testharness.js' or are relative urls
> now allowed too? Is the expectation now that the resources directory is
> included in a packaged test suite? Remember that not all test suites are
> simple clones of the WPT repo...

I don't understand what you mean by this.

Going forward, the supported configuration for running the web platform 
tests tests will be by using the provided servers and provided python 
script to initalise a full test environment including the required HTTP 
and websockets servers (in the future we can imagine also initalising 
additional servers such as FTP if we have tests that depend on them). 
Cloning the repository and all submodules will get you this full testing 
environment.

If you are a vendor trying to run the tests in automation it is expected 
that you will take the same approach to running them as just described, 
even if you also run other tests that require a different environment. 
Such other tests can be run seperately in whatever configuration they 
require.

Received on Thursday, 17 October 2013 10:52:21 UTC