Re: Who currently executes the tests in the w3c repos?

On Wed, Aug 7, 2013 at 2:05 PM, Ms2ger <ms2ger@gmail.com> wrote:

> On 08/07/2013 10:49 PM, Dirk Pranke wrote:
>
>> So it sounds like whether a test is in an "approved" directory or a
>> "submitted" directory is a rough guideline at best and probably should be
>> ignored. Does Shepherd expose APIs that would allow me to query the status
>> of a particular test path in the repo? Also, do you have plans to
>> eventually remove the approved/submitted directories? It seems like there
>> is value in having tests be in the main branch of the main repo even if
>> they are not actually approved, so that people can run tests prior to
>> their
>> approval, right?
>>
>> (It seems like this would be valuable for web-platform-tests as well,
>> right?)
>>
>
> That's already possible without requiring those tests to be in the main
> repository; pull requests to the web-platform-tests are mirrored to <
> http://w3c-test.org/web-**platform-tests/submissions/<http://w3c-test.org/web-platform-tests/submissions/>>
> (if they're created by trusted contributors or get a comment containing
> "w3c-test:mirror" from one).
>
>
I wasn't aware of that, thanks.

That said, having taken a quick look, that seems like that is quite a bit
less accessible, being N different copies of the main repo (1 per pull
request / patch), rather than having the specifically submitted files all
available alongside the approved ones.

Put differently, if we had 10 different pending pull requests for tests
from Blink, it seems like I would either have to manually merge the N
repos, or, equivalently and probably easier, apply the N pull requests to a
clone. And, even so, then I lose important traceability aspects from the
filesystem (though they'd be discoverable from the version control history).

It seems like the CSSWG's model works somewhat better if tests persist in
the "submitted-but-not-approved" state for a significant period of time
(which seems quite plausible for some tests, especially for specs that are
still being worked on). Does that sound like I have things right?

-- Dirk