Re: Generating manifests for the test runner

On 23/07/14 19:08, Brad Hill wrote:
> So, what in-test-file data does it use?  I was already asking
> previously about what metadata needed to be in tests, and the
> consensus seemed to be pretty much "none".

The correct answer is "pretty much none".

> How does the manifest generator distinguish between support files,
> READMEs, and real test cases, for example?

Real testharness.js testcases are html/XML/SVG files that contain a
<script src="/resources/testharness.js">. Real reftests either have a
<link rel="[mis]match" href="/url/of/ref.html"> or a file in the same
directory with the same name and a -ref suffix. The idea is that in the
commonest case of a testharness.js test the author doesn't have to
supply any metadata at all; things Just Work.

I notice that some of the documentation around reftests in
testthewebforward.org differs from the web-platform-tests reality. That
sucks. Unfortunately the documentation setup at the moment makes it
unnecessarily difficult to keep up to date. I have a plan to change
that, but it requires some time to rework the system for generating the
site so it can pull in documentation from the various source projects.

Also there are some differences between CSS tests and
web-platform-tests, which makes for difficulties. Frustratingly, efforts
to consolidate these testsuites have stalled.

In good news, I have spent some time today rewriting the manifest
generator to work in a way that seems much more friendly to including
local changes by default, whilst still allowing for incremental updates.
It isn't tested enough to call it done yet, but hopefully I can finish
it off soon.

Received on Wednesday, 23 July 2014 18:29:55 UTC