Re: CSS test GitHub repo strategy for TestTWF Shenzhen

Hey Peter,

Following up on this thread because we're just a few weeks from TestTWF
and there are a few details I'd like to get confirmed.

1. We'll be moving forward with your proposal to make the Github repo the
canonical repo, yes?

2. When is the earliest possible that this can be ready? May we get it
switched a week or two (preferably) before the event?

3. May we use the issue tracker to start logging tests we'll be suggesting
for people to write?  I'd need permission on the repo to use labels on
them. I've already got a list accumulating for 3-4 different CSS specs so
filtering by labels is highly desired.

4. I assume we'll have to grant certain people the right permissions for
merging, labeling, etc?  I currently have 4-5 people who'll be supporting
CSS test groups + some additional CSSWG members confirmed as experts.
If/when you grant me the proper permissions, will I be able to do this
myself or do you need a list of github Ids?

5. Finally, and this is probably more of a feature request: For this event
and going forward, we will be adopting the process of the WPT repo in that
merged PRs will be considered approved. When they land in Shepherd, they
go in as 'Submitted for Review' if they don't have reviewer links,
'Accepted' if reviewed by someone not on a short list, or 'Approved' if
reviewed by Certain People. Since the reviewer is identified in Github,
it's kind of make-work to have to manually add this as metadata (although
I realize Shepherd needs it there). Is there a way automate this and
insert the reviewer info base on the person who does the merge?

Also, for this workflow, I'd like to promote the TestTWF experts in
Shepherd to be on the list of Certain People whose reviewed tests are
automatically set to Approved.  It will cut down the work I have to do in
Shepherd afterward. Any objection to this?

Prior to the event or when you get these changes in place, please let me
know if there are any details about this workflow that might be impactful
during or after the event.  I'm happy to meet or do a call with you
anytime.

Thanks,
-Rebecca


On 10/4/13 1:01 PM, "Rebecca Hauck" <rhauck@adobe.com> wrote:

>
>
>On 10/4/13 11:48 AM, "Peter Linss" <peter.linss@hp.com> wrote:
>
>>
>>On Oct 3, 2013, at 4:52 PM, Tobie Langel wrote:
>>
>>> On Thursday, October 3, 2013 at 4:58 PM, Rebecca Hauck wrote:
>>>> On 10/3/13 7:25 AM, "Tobie Langel" <tobie@w3.org
>>>>(mailto:tobie@w3.org)> wrote:
>>>>> On a branding/discoverability perspective, I'm also not keen on
>>>>>seeing a
>>>>> testthewebforward org pop up on GH that wouldn't also be the
>>>>>canonical
>>>>> location of the testtwf-website repo, web-platform-tests repo, etc.
>>>>>As
>>>>> moving those repos would just be a major annoyance to everyone
>>>>>involved,
>>>>> I'm strongly opposed to this. If you absolutely must create a new org
>>>>>for
>>>>> this, please tie it to the CSS WG and not to Test the Web Forward.
>>>> 
>>>> Fair enough. FWIW, we created the testhewebforward github user in
>>>>April in
>>>> preparation for the Seattle event, where we first used GitHub. I only
>>>>just
>>>> converted it to an organization so I could add more owners. I'm aiming
>>>>for
>>>> this to be a repeatable process for other events and other organizers
>>>>who
>>>> may not be CSSWG members, so I'm not sure a CSSWG organization is the
>>>>way
>>>> to go. Would TestTheWebForward-Events be more acceptable?
>>> 
>>> No, because that again makes a special case (the CSS WG's reliance on
>>>HG as a canonical repository) the perceived norm. One of the key
>>>guidelines of this testing effort is discoverability. The existence of a
>>>TestTheWebForward branded repo that isn't the canonical location of the
>>>tests completely trumps that.
>>> 
>>> I have another proposal to make. It is probably a bit crazy, there are
>>>obvious downsides to it which might end up being deal breakers, but it
>>>elegantly solves all of the problems you've been bringing up here, and
>>>more.
>>> 
>>> What if we immediately started to use the main repository for
>>>contributions of CSS tests made during TestTWF events?
>>> 
>>> We could add directories for each CSS WG spec much like we do for the
>>>specs of other groups. When the time comes to merge the content of the
>>>CSS repo into the main directory, everything would already be setup. In
>>>the unfortunate event that the CSS WG would decide not to join the
>>>common effort, we could always merge the CSS tests back to the CSS WG's
>>>repo at relatively low engineering costs.
>>> 
>>> This solution obviously completely solves your workflow issues. It also
>>>solves the CLA problem I raised and the issues related to creating a
>>>dedicated GH org I mentioned above.
>>> 
>>> An added benefit is that it really simplifies the documentation and the
>>>explanation done at TestTWF events and on testthewebforward.org. There
>>>is one repo, one workflow, one way to do things, etc. It's also nicely
>>>forward-looking.
>>> 
>>> Now as, I mentioned before, there are obvious downsides to this (and
>>>maybe also less obvious ones), but the tradeoff might be worth it.
>>>Should we seriously consider this, collect the downsides, and have an
>>>informed conversation about it or are there immediate blockers that come
>>>to mind and make the exercise moot?
>>
>>I think the downsides of this approach make it a non-starter. Doing this
>>would leave us in one of two possible situations:
>>1) The CSS tests are split across two non-related repositories.
>>2) We move all the CSS tests from the WPT repo in to the CSS repo after
>>the event.
>>
>>#1 is a situation I wouldn't wish upon my worst enemy. Firstly, trying to
>>manage synchronization between two repos will be a nightmare, test suites
>>will get split, tests will get out of sync, and no one will be able to
>>unravel which is the proper state of the combined tests. Second, the CSS
>>test suites are not the contents of the repository, they are the product
>>of a build process, modifying our build process to handle source content
>>from multiple repos will be problematic to say the least (primarily due
>>to the synchronization issues listed previously).
>>
>>#2 breaks the relationship between the author's original PR and the test
>>source. Future modifications to the original PR will be disconnected form
>>the tests and lost (or will resurrect the test in the WPT repo,
>>re-introducing the sync issues), and to the author it will appear that
>>their submissions have simply disappeared. Not a good way to encourage
>>future contributions.
>>
>>The only way out of either of the above situations would be to move the
>>CSS tests en-masse into the WPT repo prior to the event, which is
>>something we're not willing to do at this point in time. We'd loose all
>>of our existing review information (or at the least, disconnect it from
>>the repo making the job of ever getting it back in sync a nightmare) and
>>the WPT tooling and process is not yet capable of handling the CSS WG's
>>testing needs (and the question of if it ever will has yet to be
>>settled).
>>
>>Tobie, I accept your objections to having a TestTWF fork of any of the
>>repos, for all the reasons you outlined. There is another alternative,
>>however, to just living with the status quo.
>>
>>The simplest solution here is to simply turn on write access to the CSS
>>WG GitHub repo for the duration of the event (and possibly just leaving
>>it on after). This way the test suite owners can simply merge PRs during
>>the event directly into the GitHub repo and everything will work as
>>expected. 
>
>Yes!! I didn't realize that was an option, but if this constraint were
>lifted, it will solve the bulk of the problems I'm trying to avoid. If
>you're willing to proceed with this, I'd ask that we get a little lead
>time of at least a few days so we can make sure it's all working properly
>by the day of the event.  Also, can/should we employ a branching strategy
>here? I.e., create at testtwf-shenzhen branch and merge PRs into that.  I
>don't know if make things more or less complicated, but it seems like a
>good use of branching.
>
>
>
>>I can easily reverse the direction of the Git<->Hg bridge and we just
>>treat the GitHub repo as canonical during the event.
>
>What are the implications of the work to do afterward? Implicit in my
>wishes to reduce the pain during the event is to also reduce mine
>personally afterward. I'm willing to do some follow-up work if necessary,
>but I'd like to get an idea of what it might be.
>
>>
>>After the event, we can manually re-sync the Hg repo to the GitHub repo
>>if there are any issues with the automatic conversion (or if there are
>>any contributions to the Hg repo during the event). I've also been
>>exploring setting up the Gut<->Hg bridge as bi-directional which would
>>let us leave write access on in both places.
>
>So if a manual re-sync is only required if something goes wrong or if
>someone contributes to the HG repo in the interim, can we just make HG
>read-only for this period of time to control that part of it?  A public
>announcement and a pointer to the Github workflow should cover that.
>
>>
>>As far as using the CSS WG GitHub issue tracker, that's already been
>>turned on. There are issues with being able to set milestones due to odd
>>GitHub access restrictions, but those go away when write-access is turned
>>on. I'll deal with synchronizing the issues with the Shepherd data.
>
>Labels are really what I'm after, but I think they're lumped in with
>milestones in the access restrictions you're talking about. I haven't yet
>decided if or how I'd want to use milestones, but please let me know if
>using them in some way interferes with something you're working on. For
>example, I see that most/all of the CSS specs are in there now as
>milestones. Should I stay away from those? Is it ok to create new ones?
>
>
>>
>>This leaves the CLA issue, but I can work on either modifying the WPT CLA
>>tool to work with the CSSWG repo, or replicating the functionality via
>>our existing tooling prior to the event.
>
>I believe I already took care of this. [1]
>
>>
>>The only other issue this would leave open is having contributors submit
>>tests to two repos, although the process would be identical so the burden
>>would be minimal. This unfortunately isn't something we can resolve until
>>such time as we're ready to merge the repositories.
>
>The alternative you're proposing here solves the majority of my concerns
>so that we can live with this a while longer.
>
>Thanks so much, Peter!
>
>-Rebecca
>
>
>
>[1] 
>https://github.com/w3c/csswg-test/commit/4a3a6280989d62aeac6bb96b09fa9e0d4
>5
>bfedc8
>
>

Received on Tuesday, 15 October 2013 18:38:27 UTC