Re: Adding support for test requests in Shepherd

On Sep 13, 2013, at 8:19 AM, Tobie Langel <tobie@w3.org> wrote:

> On Friday, September 13, 2013 at 5:01 PM, Mihai Balan wrote:
>> (cc'ed public-test-infra, but not yet dropping public-css-testsuite)
>> 
>> The way I see it (and the way I discussed it with Peter), the data will be synchronized to/from gitHub, too.
> Sure. That still won't benefit other groups, though.
>> The reason we settled on doing it via Shepherd is that it has the potential to ease the submission of test requests, without giving write access on w3c/csswg-test to everyone. Right now, one can only submit an issue, but not change its labels, milestone, etc. which makes it rather awkward and error-prone to create a gitHub-only process (for now at least).
> 
> The GH API lets you do that easily.

Yes, but the ability to change labels and milestones is still limited to accounts with write access to the repository, even via the API. Also, using the API still requires the use of a tool outside GitHub itself, hence the decision to build this feature into Shepherd since most of the required infrastructure for the tool is already done.

>> To go on a full disclosure here: the idea for a test-request process/tool came from developers in my team working on Regions/Shapes/Blending (but I think it might apply to people actually implementing the specs in other companies, too). The goal of it was to allow anyone to submit a new test request with minimal-to-none overhead (if that is possible without even logging in, even better). Anything that would require creating (new) accounts or manually editing issues, would decrease the appeal of such a tool/process.
> 
> I understand the premises. I just regret the conclusion. :)

I agree that it's unfortunate that GitHub imposes the issue access restriction and doesn't have a way to control it, requiring an external tool to get around it's limitations. 

Note that while the instance of Shepherd on csswg.org is currently only managing the csswg test repo, the Shepherd code has the ability to manage multiple repositories and is in general, not tied specifically to the csswg repo or testing methodologies (the few cases that are, are from dependencies on the build code which is getting refactored to be generic). If the facilities that Shepherd adds to GitHub are useful to the main repo, there's nothing stopping anyone from setting up another instance to integrate with the main repo, or simply adding the main repo to the existing instance.

Peter