Re: Proposal to refactor the CSS test repo

On Apr 24, 2014, at 10:56 AM, Rebecca Hauck <rhauck@adobe.com> wrote:

> 
> 
> From: Peter Linss <peter.linss@hp.com>
> Date: Wednesday, April 23, 2014 at 4:47 PM
> To: Rebecca Hauck <rhauck@adobe.com>
> Cc: "public-css-testsuite@w3.org" <public-css-testsuite@w3.org>, Elika Etemad <fantasai@inkedblade.net>
> Subject: Re: Proposal to refactor the CSS test repo
> 
>> 
>> On Apr 23, 2014, at 9:57 AM, Rebecca Hauck <rhauck@adobe.com> wrote:
>> 
>>> Hi Peter & All
>>> 
>>> Peter, I believe you and I spoke about this some time ago at TPAC and I’d like to resurrect the discussion here.
>>> 
>>> As you know, the current directory structure & naming (/approved +  /contributors/company/submitted) were part of a legacy system and don’t have much meaning or usefulness now.  In fact, if one were looking just at the names here, they’d assume that all tests in approved were actually approved and those in submitted were not. As the process, infrastructure, and the tests themselves have evolved, we now have lots of stuff on both of those directories where that’d be a false assumption.  Beyond that, it’s not that easy to browse the repo and find all the tests for a given spec because they’re split across approved/submitted and across contributors directories.  I know Shepherd provides a nice interface to see this, but since this is all code, the directories where the code is stored should be organized in a way that it’s easy to find stuff, like most/all other codebases.  I also believe we should keep a tests ‘status’ out of the directory naming entirely, especially since we’re moving to the ‘everything that’s merged is approved’ github model and since we have Shepherd for the pre-github stuff.
>>> 
>>> I propose refactor the directories to use the spec short names just as the spec repo does. 
>>> 
>>> The current root of the repo looks like this:
>>> 
>>> approved/[spec-shortname, spec-shortname…]
>>> build-test/[???]
>>> contributors/[company/testarea, company/testarea…]
>>> cvs-import/[legacy or unfilled stuff?]
>>> test-plans/[spec-shortname, spec-shortname…]
>>> tools
>>> 
>>> The proposed new structure:
>>> 
>>> tests/[spec-shortname, spec-shortname…]
>>> test-plans/[spec-shortname, spec-shortname…]
>>> tools
>> 
>> I have no issue with this plan, but do note than many tests are linked to more than one spec and I expect that to get more common as specs evolve. So while we can adopt a simple convention, like put the test in the directory for the first linked spec, people will have to be aware that the files under a single directory are not the exhaustive list of the tests for that spec. Gathering and copying all the tests for a single spec into a single package is what the build scripts do.
> 
> 
> Yes, I’d planned on going with the first spec link for the actually copy, but Tobie’s suggestion for using symlinks may work well for this scenario. AFAIK, github treats symlinks just as the file system does. Of course that means that if the original file is deleted, the onus is on the deleter to remove the symlink or they’ll be left dangling.  This is a minor issue that can be automated though.   What are the implications of this approach on the build scripts and Shepherd?

Symlinks would just appear to be additional copies of the files, both Shepherd and the build scripts already deal with that just fine (although I have to verify how mercurial's internal APIs treat them for some of Shepherd's code). For that matter, making copies of the tests rather than symlinks would also be fine with the tools, for example, we already do that with most of the support files.

Both mercurial and git handle symlinks, so long as the hg-git tool deals with them that should be ok, my main concern with symlinks is that I don't believe either mercurial or git handle them properly on Windows systems...

I'm also concerned with the general usability issues with symlinks, I'm not sure most test developers will understand, and deal with all the consequences of working with linked files.

> 
>> 
>> When you move the tests, you'll also need to be really careful of filename collisions. I don't want to see tests get lost during the move.
> 
> 
> Sure.  What do you suggest for making them unique? 

Just follow the current naming conventions. But also be really careful about updating reference links when files get renamed, this is usually one of the biggest causes of breakage.

> 
> 
>> 
>>> 
>>> I’m guessing what’s in cvs-import can be folded in and I’m not sure what build-test is for, so you’ll have to speak to that.
>> 
>> I think the cvs-import and build-test directories can be deleted, they're not used by the current build tools as far as I can tell. Elika might know more about them.
> 
>  
> Alright. I’ll ignore them for now and clean them up later.
> 
>> 
>>> 
>>> Assuming there aren’t objections to this proposal that can’t be overcome or addressed, I’d like to start the discussion the work that needs to be done for this in scripts, Shepherd, etc. I can offer some cycles to help with this and I’d invite anyone else who wants to help. 
>> 
>> The updates for Shepherd are small and in one class, the build scripts will need the most work. I'm heading out for a long weekend tomorrow but should have time to update these in the second half of next week or so.
>> 
>> When we're ready, I propose that I make a fork of the repo on our server, then you can move all the files around and push to the fork. This will give me a chance to test all of our tooling against the new repository structure before we push it to the main repo. Once it's landed I'll delete the fork.
> 
> 
> Can I just get started on my local fork in a branch? I’ll probably start this early next week.

Shepherd looks at all branches, so working in a branch wont isolate your work from Shepherd if it gets pushed to the server.

Just create a new local clone and do the re-org in that clone. When you're done let me know and I'll work with you to get a copy of that clone to test the tools against it.

> 
> 
>> 
>>> 
>>> Also in full disclosure, I’m proposing this now as a precursor to another proposal I’d like to make to get the CSS repo more integrated (as a submodule) of the WPT repo.  Since that is an entirely different different discussion, I’ll reserve that for a different thread/list, but I think getting our stuff well organized must come first.
>>> 
>>> I welcome feedback from anyone on this list and look to Peter for guidance on the best way to proceed.
>>> 
>>> Thanks!
>>> -Rebecca
>>> 
>>> 
>>