RE: [webappsec] Help build the CSP test suite at Test the Web Forward Portland, August 3

Hi Brad,

Seems you missed this step to generate MANIFEST.json:

$ python tools/scripts/

which is specified by Test Runner section at,


From: Brad Hill []
Sent: Tuesday, June 24, 2014 7:15 AM
To: public-test-infra;
Cc: James Graham; Odin Hørthe Omdal;
Subject: Re: [webappsec] Help build the CSP test suite at Test the Web Forward Portland, August 3

OK, I'm working through the directions from web-platform-tests repo in preparation for a TTWF in which it was strongly suggested that I move all existing CSP tests from PHP to the new wptrunner format.

 I am a python novice, and I assume TTWF participants will be, too. I:

Forked the repo.
Cloned it.
$cd web-platform-tests
$git submodule update --init --recursive
edited /etc/hosts

Now I"m trying to figure out how to find/use the test runner. says:

"There is a test runner in tools/runner that is designed to provide a convenient way to run the web-platform tests in-browser. It will run testharness.js tests automatically but requires manual work for reftests and manual tests."

When I go into directories with a MANIFEST, there's no clear indication which file actually runs them.

So instead I try to go to localhost:8000/tools or /tools/runner, as suggested in README, and I get:

{"error": {"message": "", "code": 404}}

I keep digging in the directory on the command line and eventually force-browse to:


Then I get a test runner interface, but if I enter /html/syntax/parsing or any other given suggestion into the path input and click "Start", nothing happens. (Tried Safari and Chrome)

Debug output on the console shows it seems to be looking for a MANIFEST file in / and not finding it because there isn't one in the Git repo,.  It still does this even if I add "?path=/html/syntax/parsing" to the URL.

So far, this isn't very beginner friendly, or even moderate-to-expert friendly.  (because the expert knows that he is only entering the rabbit hole once he starts the debugging of a tool with out-of-date documentation, and despairs at the unknown depths it may contain vs. the time available to actually get work done)

I will provide the "beginner's mind" and send the pull request for an updated README if someone who knows how this actually works can help me walk through it successfully.


Received on Tuesday, 24 June 2014 01:55:43 UTC