CSS test GitHub repo integration plans

WAS: CSS test GitHub repo strategy for TestTWF Shenzhen

Changing the subject of the thread because I have some non-TestTWF follow-up questions.

From: "Linss, Peter" <peter.linss@hp.com<mailto:peter.linss@hp.com>>
Date: Tuesday, October 1, 2013 6:43 PM
To: Rebecca Hauck <rhauck@adobe.com<mailto:rhauck@adobe.com>>
Cc: "public-test-infra@w3.org<mailto:public-test-infra@w3.org>" <public-test-infra@w3.org<mailto:public-test-infra@w3.org>>
Subject: Re: CSS test GitHub repo strategy for TestTWF Shenzhen


On Oct 1, 2013, at 1:39 PM, Rebecca Hauck <rhauck@adobe.com<mailto:rhauck@adobe.com>> wrote:

Hey Peter,

I wanted to give you a heads up that we're going to modify our GitHub CSS test submission process for the upcoming event in Shenzhen.

The plan + workflow for Shenzhen is this:

 *   Fork the w3c/csswg-test repo into the testthewebforward account [1]
 *   Have attendees fork this repo and do PRs there
 *   Merge approved PRs directly into the testthewebforward/csswg-test forked repo during the event
 *   Send one PR from this repo back to w3c/csswg-test at the end of the event.

This solves a number of problems that caused pain during & after the Tokyo and Shanghai events:

 *   Doing hg-git merges take a fair amount longer than a straight merge into the GH repo -- very long when multiplied by many PRs. If you recall, you and I had a difficult time keeping up with them in Tokyo and I worked on them for several days afterward. The Shanghai PRs have also been time-consuming.
 *   Because of the time lag in the hg-git merge process, it took many minutes (up to 30) for people's merged files to show up in the GH repo. This was because they had to do a round trip out to Mercurial and back and often waited in a queue in the middle when the PRs stacked up.  This was confusing to people - even the experts.
 *   The experts at the event can be Collaborators on the forked repo so they can merge as they approve (asking them to do the required 3-4 hour setup for hg-git to learn the steps is unreasonable). I selfishly want to delegate here so I don't have to be a merge monkey during the event. My time will be better spent helping people write tests & reviewing them.
 *   We can use the Issue Tracker in the testthewebforward fork during the event. We've tried several ways to list what tests people can write including google docs, txt files in Dropbox, readme's in the repo, whiteboards, etc.  All were flawed for obvious reasons, but mostly because it wasn't easy to tell who was doing what and what was already done. We've gotten lots of feedback on this and people were rightfully frustrated. The GH issue tracker, labels, and assignments solve all of this for us. My colleague Mihai Balan has started a similar dialog with you about this in a broader context, and in that thread I proposed we do this mini GitHub experiment during the Shenzhen event to test it out [2].

This all sounds reasonable to me. The only downside I can see is that I expect PR messages and issues filed against the testthewebforward repo will likely never get mirrored back into the main repo,

Do you mean to Mercurial?  We've done tests with dummy github repos and no PR or commit messages got lost in this workflow. We didn't go as far as setting up a dummy Mercurial repo to test this because hg-git seems to transfer everything over from the PRs fine. Once we verified everything came in to the original github repo, there was no reason to think it would get dropped in the hg-git merge.

As far as issues logged not getting mirrored – you're right, they stay with the repo where they started. But right now, we aren't able to use the issue tracker at all in the css github repo anyway, so what are we losing exactly?  Does Mercurial even have a notion of issues? Either way, github handles cross-repo issues very well, so they'll remain linked to their original source in the commit message & PRs.  This is partly of why I asked about the work you have planned.  Is there something you're working on that relates to the issue tracker?


although this may be a good thing as TestTWF events tend to generate a lot of thrashing as people find their way.

Sure, there always is throw-away stuff from these events, but this time we're planning on only merging the PRs that are approved to be inline with the WPT repo. The 'thrash' will be filtered out by this process. On the other hand, the issues I'd very much consider valuable so we can encourage people to continue working from that list after the event.  I wouldn't consider them thrash and I'm really hoping that this will be a good start of a solution to this long running problem – both at events and beyond.


I'm also wondering if there is anything you're working on that we might be able to leverage or if any of this impacts what you have planned?

I don't see any impact on my planned work from this. I might be able to have the "desired tests" tracker working before Shenzhen but I don't want to commit to it at this point.

A broader question I have - Is the goal still to eventually have the css test repo integrated with the web-platform-tests repo? I've been under that impression all along, but it's not clear to me how they're going to eventually merge or what the timeline is.