- From: Francois Daoust <fd@w3.org>
- Date: Wed, 10 Aug 2011 18:49:18 +0200
- To: James Graham <jgraham@opera.com>
- CC: public-test-infra@w3.org
On 08/10/2011 01:51 PM, James Graham wrote: > On 08/10/2011 10:33 AM, Francois Daoust wrote: >> Hi Philippe, >> >> I've added an automatic test results reporting feature to the framework, >> see: >> http://w3c-test.org/framework/suite/nav-timing-default/ >> >> The page presents a "Report test results automatically (when possible)" >> checkbox, enabled by default. When activated, pressing "start" will run >> the test suite, jumping for test case to test case and saving the result >> reported by "testharness.js". >> >> That's a bit ugly from a pure visual point of view as the browser jumps >> from page to page as soon as it gets the result of the test, but we can >> improve that later on, I think. >> >> Note automatic results reporting gets automatically deactivated when the >> user chooses to report for a user-agent that is not the user-agent on >> which it runs the tests. > > Auto-submitting information to a third-party is an incredibly bad default. I skimmed this email, didn't quite appreciate what the feature did and was quite surprised when the harness said it had reported the results to the W3C. Happily I didn't run it on some secret internal build that happened to have new features covered by the tests, but I could have done. True. I switched off the feature by default. I still need to switch form submission to use "post" to prevent the "auto" setting from showing up in URLs. I also updated the front page to make it clearer that the user is about to enter test data that will be published online. That includes replacing the "start" button with "Submit test results", completing the checkbox message and adding a note on collected/published info. I think your comment extends to more than just automatic test result gathering in any case. The interface is unclear right now on the info that gets collected and published for manual tests as well. That needs updating as well. > When the test finishes I would expect it to a) present the results to the user and maybe b) *ask* if the user would like to submit those results to be displayed in public. Yes and yes. a) is not trivial with the current implementation (there is no real notion of "session" right now) but that's a good idea. It matches Philippe's request as well. And when we have that, we could easily add an export button to address Bryan's comment. b) needs a), of course, and should at least be done before starting a "run" in the meantime. That's what I tried to do with the latest update. I agree it's not perfect. Francois.
Received on Wednesday, 10 August 2011 16:49:41 UTC