RE: Adding support for test requests in Shepherd

(cc'ed public-test-infra, but not yet dropping public-css-testsuite)

The way I see it (and the way I discussed it with Peter), the data will be synchronized to/from gitHub, too.

The reason we settled on doing it via Shepherd is that it has the potential to ease the submission of test requests, without giving write access on w3c/csswg-test to everyone. Right now, one can only submit an issue, but not change its labels, milestone, etc. which makes it rather awkward and error-prone to create a gitHub-only process (for now at least).

To go on a full disclosure here: the idea for a test-request process/tool came from developers in my team working on Regions/Shapes/Blending (but I think it might apply to people actually implementing the specs in other companies, too). The goal of it was to allow anyone to submit a new test request with minimal-to-none overhead (if that is possible without even logging in, even better). Anything that would require creating (new) accounts or manually editing issues, would decrease the appeal of such a tool/process.

Does it make more sense now? :)

Mihai Balan | Quality Engineer @ Web Engine team |  mibalan@adobe.com | +4-031.413.3653 / x83653 | Adobe Systems Romania

-----Original Message-----
From: Tobie Langel [mailto:tobie@w3.org] 
Sent: Friday, September 13, 2013 4:25 PM
To: Mihai Balan
Cc: Public CSS Test suite mailing list (public-css-testsuite@w3.org)
Subject: Re: Adding support for test requests in Shepherd

Providing solutions to help identify areas which lack coverage or should be prioritized for other reasons would be really useful. Even more so if it's coupled with a coverage analysis tool.  

Tying it to Shepherd, which is only used by the CSS WG, while the rest of the W3C is moving to a GitHub based solution is unfortunate.

Best,

--tobie  

P.S.: discussing test infrastructure related issues is generally best done on public-test-infra@.

On Friday, September 13, 2013 at 4:03 PM, Mihai Balan wrote:

>  
> Hello everybody,
>  
>  
> Recently I have participated in discussions, in different contexts, about the usefulness of recording test suggestions for W3C tests when implementing them on the spot is just not feasible because of resource/priority constraints.
>  
>  
> This becomes even more interesting/useful in the context of events like Test the Web Forward, where simply pointing to a specification might prove discouraging for a person looking to start writing a test, but instead pointing to a list of proposed/requested test cases can be a lot more approachable.
>  
>  
>  
>  
> I initially had some off-list exchanges with Peter about a gitHub-centric process that used issues on the csswg-test repo[1] to do that, but it proved unfeasible for a number of reasons (most important being access control). The proposed solution would be adding a new feature in Shepherd to create/track/edit test request that would sync with the gitHub repository eventually (but without the access control mess).
>  
>  
>  
>  
> I put together a rough draft of the proposed workflows and needed features over at [2], and I’m interested in your opinion about it:
>  
>  
> · Does the whole idea of test requests/test suggestions make sense?
>  
>  
> · Do the proposed workflows/features make sense and/or are enough?
>  
>  
>  
>  
> Let me know what you think,
>  
>  
> m.
>  
>  
>  
>  
> [1] https://github.com/w3c/csswg-test/  
>  
>  
> [2] https://github.com/mibalan/csswg-test/wiki/Tracking-tests-to-be-written-(test-requests)  
>  
>  
>  
>  
> Mihai Balan | Quality Engineer @ Web Engine team | mibalan@adobe.com (mailto:mibalan@adobe.com) | +4-031.413.3653 / x83653 | Adobe Systems Romania
>  
>

Received on Friday, 13 September 2013 15:02:30 UTC