RE: [webappsec] Help build the CSP test suite at Test the Web Forward Portland, August 3

Hi Brad,

Seems you missed this step to generate MANIFEST.json:

$ python tools/scripts/manifest.py

which is specified by README.md Test Runner section at,

https://github.com/w3c/web-platform-tests#test-runner


Thanks,
Zhiqiang

From: Brad Hill [mailto:hillbrad@gmail.com]
Sent: Tuesday, June 24, 2014 7:15 AM
To: public-test-infra; public-webappsec-testsuite@w3.org
Cc: James Graham; Odin Hørthe Omdal; jgraham@hoppipolla.co.uk
Subject: Re: [webappsec] Help build the CSP test suite at Test the Web Forward Portland, August 3

OK, I'm working through the directions from web-platform-tests repo README.md in preparation for a TTWF in which it was strongly suggested that I move all existing CSP tests from PHP to the new wptrunner format.

 I am a python novice, and I assume TTWF participants will be, too. I:

Forked the repo.
Cloned it.
$cd web-platform-tests
$git submodule update --init --recursive
edited /etc/hosts
$python serve.py

Now I"m trying to figure out how to find/use the test runner.

README.md says:

"There is a test runner in tools/runner that is designed to provide a convenient way to run the web-platform tests in-browser. It will run testharness.js tests automatically but requires manual work for reftests and manual tests."

When I go into directories with a MANIFEST, there's no clear indication which file actually runs them.

So instead I try to go to localhost:8000/tools or /tools/runner, as suggested in README, and I get:


{"error": {"message": "", "code": 404}}

I keep digging in the directory on the command line and eventually force-browse to:

http://localhost:8000/tools/runner/index.html


Then I get a test runner interface, but if I enter /html/syntax/parsing or any other given suggestion into the path input and click "Start", nothing happens. (Tried Safari and Chrome)

Debug output on the console shows it seems to be looking for a MANIFEST file in / and not finding it because there isn't one in the Git repo,.  It still does this even if I add "?path=/html/syntax/parsing" to the URL.

So far, this isn't very beginner friendly, or even moderate-to-expert friendly.  (because the expert knows that he is only entering the rabbit hole once he starts the debugging of a tool with out-of-date documentation, and despairs at the unknown depths it may contain vs. the time available to actually get work done)

I will provide the "beginner's mind" and send the pull request for an updated README if someone who knows how this actually works can help me walk through it successfully.

-Brad