W3C home > Mailing lists > Public > public-html-testsuite@w3.org > December 2012

Re: Notes on test suite reorganisation

From: Ms2ger <ms2ger@gmail.com>
Date: Tue, 11 Dec 2012 16:38:01 +0100
Message-ID: <50C75359.4020308@gmail.com>
To: public-html-testsuite@w3.org
On 12/11/2012 02:53 PM, James Graham wrote:
> On 12/11/2012 01:03 PM, Robin Berjon wrote:
>> Option (B) is interesting, but my concern is getting a view of the
>> entire set of submissions + approved tests. We *could* use the GH API to
>> obtain the full list of pending PRs and extract the content accordingly,
>> but that introduces reliance on the GH API beyond just git (which may
>> not be a problem given http://gitlabhq.com/).
>
> Let's restate Option (B) as "use branches, one per submission (note: not
> one per file)". This then frees us from the idea that this option is
> particularly tied to github; pull requests are effectively just branches
> with a fancy name.
>
> The major advantage of this approach is that good code review tools (and
> even mediocre code review tools like the one built in to github) are
> commonly organised around the paradigm that a branch is the unit of code
> review. As issues are identified in the submission, you keep pushing
> fixes to the branch until you fixed all the problems, then you
> integrate. Indeed if we want git-based code review (rather than
> something custom like the CSSWG has) I think we *have* to use this
> option to some extent.
>
> The net result will be that the approved tests are obvious — they are
> just mainline. The list of unapproved submissions is obvious; it's just
> the list of all open review requests. The only problem is that it's hard
> to run all approved + unapproved tests at once because that means
> creating a new branch that includes all the commits from the review
> branches. I suppose it isn't impossible to imagine automating the
> creation of such a thing, but there will inevitably be problems from
> merge conflicts and so on.

My main issue with this approach is that we've never been good at 
reviewing tests in a timely manner, and I don't see a reason to expect 
that to change. It seems to me like we will end up with dozens of 
branches someone once created, and that will just hang around until 
someone else, unaware of the previous submission, will duplicate them.

If we were to go this route, I would suggest being rather liberal in 
approving tests, rather than waiting until someone reviews them in detail.

HTH
Ms2ger
Received on Tuesday, 11 December 2012 15:38:35 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Tuesday, 11 December 2012 15:38:35 GMT