Re: CSS test GitHub repo strategy for TestTWF Shenzhen

On 10/4/13 11:48 AM, "Peter Linss" <peter.linss@hp.com> wrote:

>
>On Oct 3, 2013, at 4:52 PM, Tobie Langel wrote:
>
>> On Thursday, October 3, 2013 at 4:58 PM, Rebecca Hauck wrote:
>>> On 10/3/13 7:25 AM, "Tobie Langel" <tobie@w3.org
>>>(mailto:tobie@w3.org)> wrote:
>>>> On a branding/discoverability perspective, I'm also not keen on
>>>>seeing a
>>>> testthewebforward org pop up on GH that wouldn't also be the canonical
>>>> location of the testtwf-website repo, web-platform-tests repo, etc. As
>>>> moving those repos would just be a major annoyance to everyone
>>>>involved,
>>>> I'm strongly opposed to this. If you absolutely must create a new org
>>>>for
>>>> this, please tie it to the CSS WG and not to Test the Web Forward.
>>> 
>>> Fair enough. FWIW, we created the testhewebforward github user in
>>>April in
>>> preparation for the Seattle event, where we first used GitHub. I only
>>>just
>>> converted it to an organization so I could add more owners. I'm aiming
>>>for
>>> this to be a repeatable process for other events and other organizers
>>>who
>>> may not be CSSWG members, so I'm not sure a CSSWG organization is the
>>>way
>>> to go. Would TestTheWebForward-Events be more acceptable?
>> 
>> No, because that again makes a special case (the CSS WG's reliance on
>>HG as a canonical repository) the perceived norm. One of the key
>>guidelines of this testing effort is discoverability. The existence of a
>>TestTheWebForward branded repo that isn't the canonical location of the
>>tests completely trumps that.
>> 
>> I have another proposal to make. It is probably a bit crazy, there are
>>obvious downsides to it which might end up being deal breakers, but it
>>elegantly solves all of the problems you've been bringing up here, and
>>more.
>> 
>> What if we immediately started to use the main repository for
>>contributions of CSS tests made during TestTWF events?
>> 
>> We could add directories for each CSS WG spec much like we do for the
>>specs of other groups. When the time comes to merge the content of the
>>CSS repo into the main directory, everything would already be setup. In
>>the unfortunate event that the CSS WG would decide not to join the
>>common effort, we could always merge the CSS tests back to the CSS WG's
>>repo at relatively low engineering costs.
>> 
>> This solution obviously completely solves your workflow issues. It also
>>solves the CLA problem I raised and the issues related to creating a
>>dedicated GH org I mentioned above.
>> 
>> An added benefit is that it really simplifies the documentation and the
>>explanation done at TestTWF events and on testthewebforward.org. There
>>is one repo, one workflow, one way to do things, etc. It's also nicely
>>forward-looking.
>> 
>> Now as, I mentioned before, there are obvious downsides to this (and
>>maybe also less obvious ones), but the tradeoff might be worth it.
>>Should we seriously consider this, collect the downsides, and have an
>>informed conversation about it or are there immediate blockers that come
>>to mind and make the exercise moot?
>
>I think the downsides of this approach make it a non-starter. Doing this
>would leave us in one of two possible situations:
>1) The CSS tests are split across two non-related repositories.
>2) We move all the CSS tests from the WPT repo in to the CSS repo after
>the event.
>
>#1 is a situation I wouldn't wish upon my worst enemy. Firstly, trying to
>manage synchronization between two repos will be a nightmare, test suites
>will get split, tests will get out of sync, and no one will be able to
>unravel which is the proper state of the combined tests. Second, the CSS
>test suites are not the contents of the repository, they are the product
>of a build process, modifying our build process to handle source content
>from multiple repos will be problematic to say the least (primarily due
>to the synchronization issues listed previously).
>
>#2 breaks the relationship between the author's original PR and the test
>source. Future modifications to the original PR will be disconnected form
>the tests and lost (or will resurrect the test in the WPT repo,
>re-introducing the sync issues), and to the author it will appear that
>their submissions have simply disappeared. Not a good way to encourage
>future contributions.
>
>The only way out of either of the above situations would be to move the
>CSS tests en-masse into the WPT repo prior to the event, which is
>something we're not willing to do at this point in time. We'd loose all
>of our existing review information (or at the least, disconnect it from
>the repo making the job of ever getting it back in sync a nightmare) and
>the WPT tooling and process is not yet capable of handling the CSS WG's
>testing needs (and the question of if it ever will has yet to be settled).
>
>Tobie, I accept your objections to having a TestTWF fork of any of the
>repos, for all the reasons you outlined. There is another alternative,
>however, to just living with the status quo.
>
>The simplest solution here is to simply turn on write access to the CSS
>WG GitHub repo for the duration of the event (and possibly just leaving
>it on after). This way the test suite owners can simply merge PRs during
>the event directly into the GitHub repo and everything will work as
>expected. 

Yes!! I didn't realize that was an option, but if this constraint were
lifted, it will solve the bulk of the problems I'm trying to avoid. If
you're willing to proceed with this, I'd ask that we get a little lead
time of at least a few days so we can make sure it's all working properly
by the day of the event.  Also, can/should we employ a branching strategy
here? I.e., create at testtwf-shenzhen branch and merge PRs into that.  I
don't know if make things more or less complicated, but it seems like a
good use of branching.



>I can easily reverse the direction of the Git<->Hg bridge and we just
>treat the GitHub repo as canonical during the event.

What are the implications of the work to do afterward? Implicit in my
wishes to reduce the pain during the event is to also reduce mine
personally afterward. I'm willing to do some follow-up work if necessary,
but I'd like to get an idea of what it might be.

>
>After the event, we can manually re-sync the Hg repo to the GitHub repo
>if there are any issues with the automatic conversion (or if there are
>any contributions to the Hg repo during the event). I've also been
>exploring setting up the Gut<->Hg bridge as bi-directional which would
>let us leave write access on in both places.

So if a manual re-sync is only required if something goes wrong or if
someone contributes to the HG repo in the interim, can we just make HG
read-only for this period of time to control that part of it?  A public
announcement and a pointer to the Github workflow should cover that.

>
>As far as using the CSS WG GitHub issue tracker, that's already been
>turned on. There are issues with being able to set milestones due to odd
>GitHub access restrictions, but those go away when write-access is turned
>on. I'll deal with synchronizing the issues with the Shepherd data.

Labels are really what I'm after, but I think they're lumped in with
milestones in the access restrictions you're talking about. I haven't yet
decided if or how I'd want to use milestones, but please let me know if
using them in some way interferes with something you're working on. For
example, I see that most/all of the CSS specs are in there now as
milestones. Should I stay away from those? Is it ok to create new ones?


>
>This leaves the CLA issue, but I can work on either modifying the WPT CLA
>tool to work with the CSSWG repo, or replicating the functionality via
>our existing tooling prior to the event.

I believe I already took care of this. [1]

>
>The only other issue this would leave open is having contributors submit
>tests to two repos, although the process would be identical so the burden
>would be minimal. This unfortunately isn't something we can resolve until
>such time as we're ready to merge the repositories.

The alternative you're proposing here solves the majority of my concerns
so that we can live with this a while longer.

Thanks so much, Peter!

-Rebecca



[1] 
https://github.com/w3c/csswg-test/commit/4a3a6280989d62aeac6bb96b09fa9e0d45
bfedc8

Received on Friday, 4 October 2013 20:01:14 UTC