Re: UAs passing tests if they don't implement a feature

Le Lun 25 juin 2012 8:18, Aryeh Gregor a écrit :
> On Sun, Jun 24, 2012 at 7:06 PM, Linss, Peter <peter.linss@hp.com>
> wrote:
>> Understood, note that the documentation is in a wiki… feel free to
>> update it yourself as well where you feel that improvements are
>> warranted (but it's not your responsibility to do that).
>
> I would, but I don't want to make changes that others might not agree
> with -- particularly since it seems like others tend to disagree with
> me here fairly often.


I think the whole discussion is more about consistency of source code
layout, predictability and what promotes efficient readability of source
code. A more consistent code layout promotes review, quicker reading,
text skimmability, etc. as long as all the tests writing follow the same
writing rules/guidelines.

In other areas that I know of (music writing, chess game notation, speed
reading) that have rules/standards of writing, I know you would be
invited to comply with furthermore formal rules of writing.

At Mozilla office in San Francisco on monday june 18th, I was able to
see how Elika Etemad quickly grasps what's inside a source code and
easily spot the critical ("déterminant") issue of a test; because the
code was laid out in a formal and very predictable manner, she was able
to put the finger exactly and quickly on the issue.

>> I'm thinking of some improvements to Shepherd as well here to help
>> avoid misunderstandings in the future.
>>
>> I accept that the status of "Needs Work" certainly gives the
>> impression that "this _must_ be changed". Perhaps an additional status
>> level which means "this is acceptable, but it'd be nice if…",
>> suggestions of what to call that are welcome.
>
> The Linux kernel has a directory called staging/ where drivers are put
> that are known to be useful but need code-quality work.  (They have a
> big issue where vendors will write large drivers behind closed doors,
> not to kernel coding standards, and then try dumping them so they can
> claim Linux support.)
>
> How about "Accepted For Further Work"?  "Accepted (Cleanup Requested)"?
>
> On Sun, Jun 24, 2012 at 7:15 PM, "Gérard Talbot"
> <css21testsuite@gtalbot.org> wrote:
>>
>> Le Dim 24 juin 2012 11:33, Aryeh Gregor a écrit :
>>> I appreciate the clarification, and accept that this was just a
>>> misunderstanding.  Please keep in mind that this is my first
>>> involvement with CSSWG testing; as such, things that might seem
>>> obvious to you or fantasai are not necessarily obvious to me.
>>> Up-to-date documentation is essential.
>>
>> Documentation on test format and guidelines is updated as far as I can
>> see. Always has been.
>
> Current documentation seems to suggest that, e.g., using style
> attributes is not allowed:
>
> "Do not use the style attribute (inline styles) unless specifically
> testing that attribute"
> http://wiki.csswg.org/test/format#body-content


Inline style is formally discouraged; so, it's not allowed.


> It's not clear on whether tests can be accepted if they don't obey the
> guideline.


As a test reviewer, I expect to be able to quickly and easily see the
*_whole style code_* inside one single style block; I also expect to
easily and quickly figure out where the content is and where the
structure is, where the pass/fails conditions are expressed, etc. The
more a code consistently complies with formal rules, the faster and
better I can understand a test, the less I have to think, to search,
etc.

As a test writer, I want the test reviewer (or peers or anyone examining
the source code) not to hesitate, not to search, not to guess, not to
think, not to calculate. I want to ease his/her tasks, efforts, to
optimize his/her time, etc.

>> Has there been one single test where a test was rejected or put into
>> the
>> NeedsWork status because of what you refer to as stylistic reasons?
>> Aryeh, please be specific here.
>
> Yes -- for instance, here:
>
> http://test.csswg.org/shepherd/testcase/transform-background-001#comment-776e0f90dc6c


So, this one was put in the NeedsWork status.


> See also here:
>
> http://test.csswg.org/shepherd/testcase/transform-background-001#comment-b8400648b91e
>
> Similar comments were posted on a bunch of my submitted tests, and it
> was clear they applied to all of them.


Well, then, such comments were furthermore useful and had more
importance. We want to review the first 25-50 tests from new test
authors who may not be familiar with test format and writing guidelines.


> In the first case, I saw that the feedback seemed to match the
> guidelines, and discussion on public-css-testsuite seemed to indicate
> that there was no support for changing the guidelines to allow the
> tests I submitted.  In retrospect, although I haven't reread the
> threads to be sure, perhaps it was unclear to participants in the
> discussion that I was mostly interested in not having to rewrite my
> already-written tests, rather than allowing variation for tests
> written a priori for the test suite.  In any event, it certainly
> seemed to me like my test was not going to be approved unless I
> rewrote it.

I do not know how much time and efforts retrofitting all your tests to
honor writing guidelines would imply. Some could be done with an
advanced text editor; some others - maybe - with HTML Tidy. But I know
that new tests should from now on match more closely those guidelines.

I agree with all of Dirk Schulze's comments in
http://test.csswg.org/shepherd/testcase/transform-background-001#comment-776e0f90dc6c
with one exception: I do not understand what Dirk meant with "You use
non-alphabetic chars. So please provide the encryption (necessary for
HTML5 documents)."

Regarding use double quotes for attribute values, I do not know of a
single recent HTML WYSIWYG/webpage editor which, by default, do not
quote href attribute values. Some webpage editors (eg BlueFish) will
change the color of the attribute value if double quotes do not enclose
href values.

html, head and body may be optional in HTML4 & 5 but their presence
helps identify sections of source code and readability of source code.

> In the second case, I didn't think the feedback was justified by the
> written guidelines, so after replying, I set the status back to
> Awaiting Review.


Self-descriptive tests and reftests are (and will be) preferred.


>> Several questions still remain. What are the guidelines, requirements
>> for new tests? And why people do not follow these?
>>
>> There are long-term and short-term benefits to following guidelines
>> and
>> requirements for new tests.
>>
>> Aryeh Gregor, are you and were you using (pasting) the template when
>> creating new tests?
>
> First of all, a lot of my tests were from Mozilla's internal reftest
> suite.  They weren't written to CSSWG style guidelines because they
> weren't originally written for the CSSWG at all.
>
> Second of all, at the time I wrote my tests, I looked briefly at the
> guidelines but saw that they were clearly written several years ago
> and never updated.  For instance, parts were written assuming that the
> tests were for CSS 2.1 rather than CSS 3, and XHTML was the only
> allowed input format (which made sense in 2006 but doesn't in 2012).
> I concluded that I would write the tests first and deal with style
> later.  This was apparently a mistake.


Test format
http://wiki.csswg.org/test/format

Reviewing tests
http://wiki.csswg.org/test/review

CSS Test Review Checklist (more detailed review checklist)
http://wiki.csswg.org/test/css2.1/review-checklist

Gérard
-- 
Contributions to the CSS 2.1 test suite:
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/

CSS 2.1 Test suite RC6, March 23rd 2011:
http://test.csswg.org/suites/css2.1/20110323/html4/toc.html

CSS 2.1 test suite harness:
http://test.csswg.org/harness/

Contributing to to CSS 2.1 test suite:
http://www.gtalbot.org/BrowserBugsSection/css21testsuite/web-authors-contributions-css21-testsuite.html

Received on Monday, 25 June 2012 19:21:37 UTC