W3C home > Mailing lists > Public > public-webappsec-testsuite@w3.org > June 2014

RE: [webappsec] Help build the CSP test suite at Test the Web Forward Portland, August 3

From: Zhang, Zhiqiang <zhiqiang.zhang@intel.com>
Date: Tue, 24 Jun 2014 01:54:06 +0000
To: Brad Hill <hillbrad@gmail.com>, public-test-infra <public-test-infra@w3.org>, "public-webappsec-testsuite@w3.org" <public-webappsec-testsuite@w3.org>
CC: James Graham <james@hoppipolla.co.uk>, Odin H?rthe Omdal <odinho@opera.com>, "jgraham@hoppipolla.co.uk" <jgraham@hoppipolla.co.uk>
Message-ID: <0EA8FB2070816C499E0A50AC1B0B5C16015BF6EF@SHSMSX101.ccr.corp.intel.com>
Hi Brad,

Seems you missed this step to generate MANIFEST.json:

$ python tools/scripts/manifest.py

which is specified by README.md Test Runner section at,



From: Brad Hill [mailto:hillbrad@gmail.com]
Sent: Tuesday, June 24, 2014 7:15 AM
To: public-test-infra; public-webappsec-testsuite@w3.org
Cc: James Graham; Odin Hørthe Omdal; jgraham@hoppipolla.co.uk
Subject: Re: [webappsec] Help build the CSP test suite at Test the Web Forward Portland, August 3

OK, I'm working through the directions from web-platform-tests repo README.md in preparation for a TTWF in which it was strongly suggested that I move all existing CSP tests from PHP to the new wptrunner format.

 I am a python novice, and I assume TTWF participants will be, too. I:

Forked the repo.
Cloned it.
$cd web-platform-tests
$git submodule update --init --recursive
edited /etc/hosts
$python serve.py

Now I"m trying to figure out how to find/use the test runner.

README.md says:

"There is a test runner in tools/runner that is designed to provide a convenient way to run the web-platform tests in-browser. It will run testharness.js tests automatically but requires manual work for reftests and manual tests."

When I go into directories with a MANIFEST, there's no clear indication which file actually runs them.

So instead I try to go to localhost:8000/tools or /tools/runner, as suggested in README, and I get:

{"error": {"message": "", "code": 404}}

I keep digging in the directory on the command line and eventually force-browse to:


Then I get a test runner interface, but if I enter /html/syntax/parsing or any other given suggestion into the path input and click "Start", nothing happens. (Tried Safari and Chrome)

Debug output on the console shows it seems to be looking for a MANIFEST file in / and not finding it because there isn't one in the Git repo,.  It still does this even if I add "?path=/html/syntax/parsing" to the URL.

So far, this isn't very beginner friendly, or even moderate-to-expert friendly.  (because the expert knows that he is only entering the rabbit hole once he starts the debugging of a tool with out-of-date documentation, and despairs at the unknown depths it may contain vs. the time available to actually get work done)

I will provide the "beginner's mind" and send the pull request for an updated README if someone who knows how this actually works can help me walk through it successfully.


Received on Tuesday, 24 June 2014 01:55:43 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 18:55:13 UTC