[csswg-drafts] Test Metadata

frivoal has just created a new issue for https://github.com/w3c/csswg-drafts:

== Test Metadata ==
The CSSWG used to require at least one`rel=help` in test metadata to associate each test with a spec (and preferably a section of a spec).

As part of the switch to wpt, `rel=help` was made optional. My understanding was that the reasoning was:
* not every test author can be bothered, and in particular browser vendor's internal tests typically do not bother. wpt doesn't want to reject tests for that, so wpt will not accept `rel=help` being mandatory
* in wpt, the path must match the spec shortname, so it carries the same information, so nothing is lost.

However, while wpt has a general practice of using paths that match spec names, it is not a strict requirement: the path is generally not expected to include the spec level (which was a strict requirement in the css rel=help checker), and tests included in a versioned directory are not expected to move if the feature is moved to a different level of the spec, as wpt has a general aversion to moving files due to some vendors have out of band data (such as reference renderings) that are tied to file names.

This means that while `rel=help` when present can be expected to point to the right place, it cannot be expected to be present, its presence should not be a requirement before accepting Pull Requests, and the path may, but does not have to, point to strictly the right place.

Questions:
1) As a WG, are we OK with this status quo?
2) If we are not, do we want to change how we store metadata?
3) What do we do (if anything) about testharness and its ability to find tests for a spec, based on non-necessarily sufficiently accurate metadata.
4) Should we create directories with versioned shortnames (in addition to the versioned ones we have) and put all new tests there?


Please view or discuss this issue at https://github.com/w3c/csswg-drafts/issues/1730 using your GitHub account

Received on Thursday, 17 August 2017 15:41:51 UTC