|wpt| command has now landed

The wpt command has landed in web-platform-tests. This is a generic 
command dispatch framework that's intended as a frontend to a lot of 
useful tools for working with wpt. So, for example

./wpt run firefox dom/historical.html

will run the given test in a local copy of firefox (note: this replaces 
the |wptrun| command, although that is still present for legacy cpmpat 
reasons).

Similarly

./wpt check-stability chrome dom/historical.html

will run the test 10x in chrome and check for consistent results; much 
like the job on travis.

The "serve", "manifest" and "lint" commands are now available through 
wpt i.e.

./wpt serve

./wpt manifest

./wpt lint

In addition we also have commands for installing browser/webdriver 
binaries (wpt install), checking which files are affected by the current 
branch (wpt files-affected), checking which tests are affected by 
changes to the current branch (wpt tests-affected) and checking which 
test jobs should run for the current branch (wpt test-jobs).

The last command is notable because it is part of some work I have done 
to speed up our CI infrastructure; travis now aborts early for jobs that 
shouldn't be relevant to a given PR (e.g. unittests for tools on a test 
change); canceled jobs take about 30s (and there is probably some room 
for improvement here, although unfortunately travis doesn't yet allow us 
to avoid scheduling jobs entirely when we know they won't be necessary).

In addition we are now caching the wpt manifest on travis, and there is 
a PR in flight to cache the output of the lint. I'm hopeful this will 
cause a significant improvement in the end to end time for a typical PR. 
Of course this won't help changes that require stability checking of 
slow-running tests. It is less clear how to make improvements in that case.

Received on Thursday, 27 July 2017 17:23:27 UTC