Re: Submission of transform tests

On Fri, Mar 23, 2012 at 5:23 PM, Linss, Peter <peter.linss@hp.com> wrote:
> Correct. The rules are:
> Support files in a 'support' directory.
> Reference files in a 'reference' directory (preferred), or have 'ref-' or 'notref-' filename prefix, or have '-ref' or '-notref' filename suffix (this will be expanded shortly to allow '-ref-' or '-notref-' within the filename).
> Everything else is considered a test case.

Okay, thanks.  I guess the "-ref" suffix is allowed before the
extension, so my current naming convention for refs is fine, right?
Would it be better if I changed it so the files were in a reference/
directory?

Also, with respect to test numbering: should I just rename
abspos-1a.html to abspos-001.html, abspos-1b.html to abspos-002.html,
etc.?  This looks like it matches the convention for the CSS 2.1 test
suite.  The advantage of the current naming scheme (which I copied
from Mozilla) is that it allows tests to be grouped.  E.g.,
background-1*.html all have background-1-ref.html as their ref file,
and background-2*.html all have background-2-ref.html as their ref
file.  If they were numbered consecutively, then it would be less
obvious which tests corresponded to which refs.

If I stick with the number-letter naming convention from Mozilla, I
can pad to three digits if you prefer, but it's probably never going
to be needed -- with this convention, two digits would mean ten
different *ref* files for a single part of the spec.  At least for
transforms, it's unlikely that this would ever be needed, because it's
pretty easy to reuse the same ref file for lots of tests.  (In fact,
green-rect.html is the ref file for a large fraction of my 3D
transform tests, and could be used for a majority if I wanted to tweak
them.)  The letter part of the name can be extended indefinitely
without padding, because after z would come aa, and "foo-1aa.html"
sorts after "foo-1z.html" naturally because . in ASCII comes before
all letters.

But this is pretty arbitrary and I'm happy to do whichever way makes
sense to everyone else.

> Note that we currently expect spec links in test cases to be to the version last published at http://www.w3.org/TR/, not drafts at dev.w3.org/csswg. So you'll have to update those links anyway (but wait until I update Shepherd).
>
> There's been some discussion about allowing links to draft specs and having the build system adjust links on output, but there's nothing in place to support that yet, and the current general feeling is to disallow draft links to encourage more frequent publishing of drafts.

Okay.

On Fri, Mar 23, 2012 at 9:35 PM, Peter Linss <peter.linss@hp.com> wrote:
> Done.
>
> Since the current ED has a significant number of changes to the section
> anchors and should be updated shortly anyway, I went ahead and used the
> sections from the current ED. Note however that you still need to have the
> www.w3.org/TR/ URLs in your tests, but you can use section anchors from the
> current ED version.

Thanks!

On Fri, Mar 23, 2012 at 9:51 PM, Peter Linss <peter.linss@hp.com> wrote:
> please update your Mercurial config for the test repo as well. Be sure to use the email address <ayg+csswg@aryeh.name> in the [ui] username.
>
> See: http://wiki.csswg.org/tools/hg#setting-up-mercurial-preferences
>
> You can add that setting in the .hg/hgrc file of your local clone of the test repository if you don't want to use that email address for other repositories.
>
> Doing this will allow Shepherd to correlate your Mercurial commits with the same account as used for your Author metadata (right now it's using different accounts for owner and author).

What e-mail address is it comparing against?  Would it be possible
instead to change whatever knows about ayg+csswg@aryeh.name to use
ayg@aryeh.name?  These days I prefer to use just ayg@aryeh.name for
any human-visible e-mail address, and use plus-addressing only for
automated mail reception.

On Fri, Mar 23, 2012 at 10:22 PM, Linss, Peter <peter.linss@hp.com> wrote:
> Specifically, that file shouldn't have '-ref' in the filename, it's a support file used by a reference, not a reference in itself.
>
> Also, the other .svg files really need to be in a 'support' directory as well. Shepherd does recognize them as support files for the moment, but that's only because it doesn't accept .svg tests yet. Once I turn that on it would think they're supposed to be test files.
>
> I went ahead and moved the support files and updated the URLs.
>
> I also updated all your spec links to w3.org/TR

Thanks!  Are files in support/ not allowed to have -ref in their name
either, or will shepherd figure out that they're support files based
on the directory?  When I first added those support files they were
only used by one test each, but at this point it might make more sense
to rename them to something generic anyway.

> some of them are still being reported as an error because they don't have an anchors in them. They need to link to a specific section of the spec (or multiple sections if relevant) instead of simply the spec.

Okay.  No specific section was relevant to those tests, so I just
linked them to #transform-property:

http://hg.csswg.org/test/rev/90c09ec77591


Is there a way to see a list of tests that shepherd has rejected due
to formatting errors or such?  http://test.csswg.org/shepherd/ seems
not to list such tests anywhere.

Received on Friday, 6 April 2012 09:22:13 UTC