Re: Proposal to introduce test suite curators

> On Mar 4, 2017, at 06:36, Gérard Talbot <www-style@gtalbot.org> wrote:
> 
> Reviewing tests that others have written and submitted
> [...] requires time, efforts, care [...]
>  I think you can *not* expect volunteers to do this.

By volunteers, I did not mean people who do not get paid. Reviewing, or writing test, as well as editing a specification, is certainly significant work, and while some people may occasionally be willing to do that for no compensation, I agree that we cannot generally depend on that.

What I meant was that a large number of people who are members of the working group are paid to do so, and that as a group, we should recognize that reviewing tests is part of the job, and we should make sure that someone is assigned to it. But since we cannot as a group designate someone and hope they'll just take care of it, some of the people whose job it is to make standardization of CSS move forward need to step up and "volunteer" for it.

Also, having explicitly vacant positions may make it easier for independents and consultants to identify the areas that need work, and seek sponsorship for doing them.

> Assessing coverage of specification by tests in a test suite is a rather very difficult task. It would imply to start by enumerating all testable statements for each section of a specification and then create sufficient tests for each testable statements (worth testing) of each section of specification.
> 
> Complete, thorough test coverage of a specification is a never ending work-in-progress, I'd say.

Agreed. And thorough assessment of test coverage in the exhaustive way you described is only justified when we try to get to PR. Earlier, it is less critical.

> Your test curator idea is, I believe, more about management, superintendant of overall test suite and communication with test authors.

Well, yes and no. As you said, reviewing tests is also very technical, but the goal is indeed one of facilitation.

But the main point is that as a group, we're responsible for getting things to REC, and that cannot happen without test suites. However, with a few exceptions, we are not handling that responsibility well, and not only do we write few tests, we do not even do a good job of taking in the tests that others submit to us. You Gérard are a notable exception to this, but we cannot just expect you to do it all while others look elsewhere. Adding Geoffrey to the team of people who take care of tests still does not cut it.

Maybe my suggestion of a test curator is not a good one, but we need to do something.

That fact that as an editor of a specification that is in late CR and should be trying to get to PR (CSS-UI-3), I do not know who to turn to review the tests I have submitted is not encouraging.

—Florian

Received on Monday, 6 March 2017 01:49:11 UTC