[miniutes] HTML Testing Task Force Conf Call Agenda 3/23/2010

Below is the IRC Log from the meeting which we again focused on reviewing the submission, feedback and approval process.
Once the updates are done to the wiki, we'll be using a submitted test as an example to be approved.
One key part of this process is that we want to be very clear which tests are approved (golden).
As such they will be in the approved folder, once  test is approved changes to the test will need to have a BugZilla bug opened.
The other key part of the process is that we want the approved tests to grow overtime, unlike some other test suites where the entire test suite is given a quality label (alpha, beta, rc).

Thanks for attending and feedback to the submission, feedback and approval process.

-Kris


Here is the RAW IRC Log - sorry forgot to invite trackbot
*** krisk [836b0068@128.30.52.43] has joined #htmlt
*** krisk @gsnedders
*** Channel created on Tue Mar 23 14:57:56 2010
<gsnedders> Is it time already?
* gsnedders has got confused with different countries moving to DST at different times
<krisk> in 2 minutes...
<gsnedders> Two minutes to find a headset in Oslo. Fun.
<krisk> ready?
*** Zakim [rrs-bridgg@128.30.52.169] has joined #htmlt
<gsnedders> zakim, what's the code?
<Zakim> sorry, gsnedders, I don't know what conference this is
<gsnedders> krisk: Yes
<gsnedders> If I can find the code :)
<gsnedders> Zakim, this will be htmlt
<Zakim> ok, gsnedders, I see HTML_WG(HTMLT)11:00AM already started
<gsnedders> zakim, what's the code?
<Zakim> the conference code is 48658 (tel:+1.617.761.6200 tel:+33.4.89.06.34.99 tel:+44.117.370.6152), gsnedders
*** snewman [836b0057@128.30.52.43] has joined #htmlt
<Zakim> + +0131208aaaa
<Zakim> -[Microsoft]
<Zakim> +[Microsoft]
<gsnedders> zakim, aaaa is me
<Zakim> +gsnedders; got it
<krisk> rssagent, set logs public
<krisk> meeting: HTML Testing Task Force
<krisk> The agenda this moring is to go over the submission, feedback and approval process for test cases
<krisk> Lets first start with the submission part http://www.w3.org/html/wg/wiki/Testing/Submission/
<gsnedders> Basically all I care about is simplicity, so it seems fine
<krisk> Note the comments about not using specific logic for specific browsers
<krisk> For example the canvas tests had special logic for certain browsers
<gsnedders> We should probably have comments like taht in a separate file as all steps of the submission, from the initial submission to approval depend upon it
<gsnedders> (esp. review)
<krisk> The old jsunit test harness also had issues - since it did run in all browsers
<krisk> In the approval link it lists the 'how'
<krisk> Lets move on to the feedback part
<krisk> http://www.w3.org/html/wg/wiki/Testing/Feedback/
<krisk> Take away is that the majority of the feedback should be on the list/email(s)
<gsnedders> For submitted tests the email probably ought to go to the author of the test too
<krisk> I'll update the site
<snewman> probably should notify the author in both cases.
<snewman> approved or submitted.
<krisk> agree
<krisk> I'd like to have tests approved overtime
<krisk> And not have the situation where the suite test has a bunch of submitted and unreviewed test cases
<gsnedders> I think the real problem is getting to people to review tests
<krisk> From a project standpoint we should extreemly clear which tests are golden since they will be in the approved folder
<krisk> If we end up with alot of submitted and unrviewed tests then we can ask for more help from the html wg
<krisk> Though from a project stand point as long as we have tests reviewed overtime we should be OK
<krisk> If we get behind (nice problem to have) then we can just push to review the oldest submitted tests
<krisk> we have agreemen on the feedback section
<krisk> lets move on to the approval procedure http://www.w3.org/html/wg/wiki/Testing/Approval/
<krisk> [gsnedders] part #5 should be done in the list as well
<krisk> I'll update the wiki for this to be more clear
<gsnedders> The feedback loop could be clearer, it's not something that'll just happen once
<krisk> agree
<gsnedders> It also looks as if a whole test suite will be approved it once
<gsnedders> Having it test case by test case would be better
<krisk> agree I'll add that to the wiki
<krisk> The goal is to get tests from reviewed to approved quickly
<gsnedders> I think there should be no time at all
<gsnedders> Once reviewed, that should be it approved
<krisk> How about once the review is done and tests are updated then an email is sent the list
<gsnedders> We can probably automate that somehow, things hat are moved to automated
<gsnedders> *approved
<krisk> Well with hg its pretty easy to ove files around
<gsnedders> Then just send one once per day, to avoid flooding the list
<krisk> I can do that once we get alot of tests flowing
<krisk> I'll also update the wiki to set expectations/best practices
<krisk> If you submit a lot of tests it will take a long time for the reviews to happen
<krisk> So it's best to submit tests in smaller bunches
<krisk> any other feedback?
<krisk> I'll read up on the widget wg for the feature list
<gsnedders> http://www.w3.org/TR/test-methodology/
<gsnedders> http://dev.w3.org/2006/waf/widgets/imp-report/
<gsnedders> Those are what I'm referring to
<gsnedders> I'll send an email out to the list
<krisk> How about we plan for this in the long term
<krisk> Long term this should be and easier to do once the HTML5 spec goes farther along
<krisk> In the short term we can do this on a feature basis
<krisk> I am going use the window securty tests to test the process and have this moved to approved
<krisk> I'll respond to the list with updates to this example test to move forward
<krisk> any thing else?
<Zakim> -[Microsoft]
<Zakim> -gsnedders
<Zakim> HTML_WG(HTMLT)11:00AM has ended
<Zakim> Attendees were [Microsoft], +0131208aaaa, gsnedders
<krisk> rrsagent, make minutes
<krisk> no rssagent...
<krisk> I'll post the IRC log to the list

Received on Thursday, 25 March 2010 14:12:04 UTC