- From: Hill, Brad <bhill@paypal.com>
- Date: Wed, 4 Jun 2014 20:53:32 +0000
- To: Odin Hørthe Omdal <odinho@opera.com>
- CC: WebAppSec WG <public-webappsec@w3.org>
OK. I'll give it a shot. I've been too busy with other things to keep up with wptserve recently, but I need to invest some time in the suite before TTWF anyway, so I'll see what I can do. Can you point me at how users can see/test changes in real-time? That was the other issue I had with using the "canonical" repo - with only me contributing, there was nobody who really had it on their TODO list to review and approve my submissions, so merge times when I was working on CORS tests were on the order of 6-8 weeks, and only after I jumped up and down and waved a big flag. That's just not workable to have multiple people contributing and be so out of sync. -Brad On Jun 4, 2014, at 1:44 PM, Odin Hørthe Omdal <odinho@opera.com> wrote: > -CC webappsec-testsuite (public-webappsec should be a superset of the > other, no?) > > On Wed, Jun 4, 2014, at 21:41, Hill, Brad wrote: >> As I've mentioned before during the requirements and development phase of >> wptserve, CSP is a bit more complex than most specs in its test >> requirements and I'm not sure the environment has what we need. >> >> I'd love it if we could make this work, and maybe I don't understand all >> of what the new framework can do, but here's a few things we need: >> >> 1) Wildcard, or at least multiple-cname, TLS support. This is absolutely >> critical for the scenarios where CSP actually matters and will be >> deployed. > > As far as I know, this is something that is detached from the actual > framework itself. You can use the same SSL server setup and hosts with > wptserve as with PHP. > > In fact, w3c-test.org should probably be set up like that. It currently > isn't. > > Those tests that rely on that will not be as easy as other > wptserve-based tests though. They would indeed require much more > substantial setup than the common case. However, if w3c-test.org is > being set up correctly, there is currently live-syncing from pull > requests on GitHub over to that server. That way it'd be possible to > have a live server you could test directly on with very little setup. > > > jgraham also wanted to make it easy to do HTTPS-tests locally, but that > would as far as I'm aware require some bad certificate hacks. > > Anyway, there are two different things here; what can be done with one > single server or AWS instance (that's set up correctly), and what can be > done easily locally and $random_user's computer. > >> 2) The ability to not only set headers, but to access the value of the >> headers from within the tests. (e.g. to compare reports vs. the policies >> that were set) In PHP this is simple. I don't know if one can do this >> with the .headers file approach. > > That is simple, you just write that in the test directly. It's also > easier to understand and more to the point. For production code we > often Don't Repeat (Y)Ourselves, but for tests it's okay as long as it > is of basic relevancy for the test. And for included files, there is > also the substitution part of wptserve that can change parts of a file > with substitution variables. > > If you wanted to do it in a more advanced way, it's possible to write > full Python scripts that can do anything. But it's nice to keep simple > things simple. > > > I'd like trying to port a test that you find of complex enough character > to see if it's doable. If you can point one out. > >> 3) We also need to test that reports are sent correctly. Right now, We >> have a support PHP script that actually writes to temp files on disk and >> then allows the test case to call back and see what the results were. >> ... > > CORS tests do this a lot for preflight. In fact, I was very happy about > how wpt handles this. It's much better than what we have for PHP. It's > called Stash, and even has its own help page[0] :D > > 0. http://wptserve.readthedocs.org/en/latest/stash.html > > >> This is why I'm running an AWS server at my own cost so I can provide a >> turn-key environment for people who want to contribute tests, with all >> the necessary setup done, including a valid wildcard TLS certificate. >> >> If there's a better way that we can get a simpler setup working before >> TTWF, I'd love to hear it, but I'm not sure there is. > > I think your "give ssh accounts" is a very good method. But there is no > reason why that AWS server can't run wptserve. Getting w3c-test.org > to do all we want is better though, so that you won't have to maintain > that server on your own. > > An added benefit of getting these tests in the upstream is that > web-platform-tests is the canonical place for all tests of type web > platform, and thus run by the browser vendors* :) > > [*] Currently not everyone is really running everything as well as they > should. > But as far as I know they're all working on it. > > > -- > Odin Hørthe Omdal > odinho@opera.com
Received on Wednesday, 4 June 2014 20:54:02 UTC