RE: HTML Testing Task Force Conf Call Agenda 2/8/2011

Meeting Notes 

* Good Number of bugs - wait to see if some bugs will be fixed before removing them from the approved folder
* Might pursue 'plan b' if we don't get a response back on the A/V test submission
* Same time/place/channel for Meetings

-Kris


IRC Log
[08:01] == krisk [qw3birc@128.30.52.28] has joined #htmlt
[08:02] <Zakim> HTML_WG(HTMLT)11:00AM has now started
[08:02] <Zakim> +[Microsoft]
[08:03] <krisk> Zakim, Microsoft is krisk
[08:03] <Zakim> +krisk; got it
[08:04] <krisk> Looks like I am the only on the conf call
[08:04]  * plh zakim, call plh-work
[08:04]  * Zakim ok, plh; the call is being made
[08:04] <Zakim> +Plh
[08:05]  * jgraham is on irc
[08:05] <krisk> We can do this on IRC
[08:06] <krisk> Let's get started
[08:06] <krisk> Agenda: http://lists.w3.org/Archives/Public/public-html-testsuite/2011Feb/0007.html
[08:06] <Ms2ger> http://ms2ger.freehostia.com/tests/results/results.html for those interested
[08:07] <krisk> So I opened a bug for to update the approved canvas tests that point to test2.w3.org
[08:07] <krisk> ...and sent an email to Philip Taylor
[08:07] <krisk> ms2ger dup'd the bug to one he had opened (fine)
[08:07] <Ms2ger> :)
[08:07] <krisk> though it's still assigned to null dave
[08:08] <jgraham> Ms2ger: Removing the % at the bottom would be good
[08:08]  * jgraham is being offtopic
[08:08] <Ms2ger> Hey, I like my irrelevant stats :)
[08:09] <krisk> I noticed that Philip Taylor <pjt47@cam.ac.uk> (pjt47@cam.ac.uk) is not in bugzilla
[08:09] <Ms2ger> I reassigned the bug
[08:09] <krisk> though MikeSmith said he uses the excor one...
[08:10] <krisk> so I'll assign 11921 to excors@gmail.com
[08:10] <krisk> rather than null dave
[08:11] <Ms2ger> Too late ;)
[08:12] <krisk> Philip normally fixes these really fast
[08:14] <krisk> So looking a bugzilla we also have bug #11917
[08:14] <Ms2ger> Just a typo in the test description
[08:15] <krisk> here is the link to all the 'bugs'
[08:15] <krisk> http://www.w3.org/Bugs/Public/buglist.cgi?bug_file_loc=&bug_file_loc_type=allwordssubstr&bug_id=&bug_id_type=anyexact&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&chfieldfrom=&chfieldto=Now&chfieldvalue=&component=testsuite&email1=&email2=&emailtype1=substring&emailtype2=substring&field0-0-0=noop&keywords=&keywords_type=allwords&long_desc=&long_desc_type=allwordssubstr&product=HTML%20WG&query_format=advanced&short_desc=&short_desc_type=allwordssubstr&status_whiteboard=&status_whiteboard_type=allwo
[08:16] <krisk> ms2ger can you assign 11917 to philip taylor and add a link to the test case in the bug?
[08:16] <Ms2ger> Will do
[08:17]  * jgraham notes that 7260 is not relevant to browsers
[08:17] <krisk> Looking at 11916 - can you point to the w3 dev spec?
[08:18] <krisk> and assign to philip taylor as well
[08:19] <krisk> Now looking at 11871
[08:20] <krisk> 11871 also should be assigned to philip taylor
[08:20] <Ms2ger> That's waiting for the WebIDL to be updated, though
[08:20] <plh> and that won't happen before April I think
[08:21] <krisk> Then we should pull the test until then
[08:21] <krisk> ...from the approved folder
[08:21] <Ms2ger> We decided to approve it until WebIDL changed, iirc
[08:24] <krisk> I find it's best to put this info in the main bug
[08:24] <krisk> so that when/if it changes all issues can be resolved
[08:25] <krisk> in this case it seems that bug #11176 should state that when/if this is fixed
[08:25] <krisk> ...http://dvcs.w3.org/hg/html/file/72c5c32c560b/tests/approved/canvas/type.delete.html
[08:25] <krisk> needs to be updated as well.
[08:25] <krisk> ...sending an email to the list and opend a new bug
[08:26] <krisk> Then we don't end up with alot of bugs that people can't take action on
[08:26] <Ms2ger> I don't think a bug overload is a big problem atm
[08:26] <krisk> Good to see a good # of bugs
[08:27] <krisk> Now looking at 11871
[08:27] <krisk> Doh!
[08:27] <krisk> Now really looking at #11825
[08:27] <Ms2ger> I marked it as fixed
[08:28] <krisk> I removed it from the approved folder
[08:28] <plh> is something going to contact David?
[08:28] <krisk> david carlisle responded to the list
[08:29] <krisk> that he agreed it could be removed from the approved folder
[08:29] <krisk> since it's testing something that is not part of the spec...
[08:30] <krisk> ...though would be correct if http://www.w3.org/Bugs/Public/show_bug.cgi?id=11204 is addressed
[08:31] <krisk> with a number of bugs on philip shall we remove them from the approved folder until he fixes?
[08:31] <krisk> as soon as he fixes we can re-approve them
[08:32] <Ms2ger> I don't care much either way
[08:32] <krisk> jgraham do you have any objections?
[08:32] <plh> philipp has been good at fixing bugs in the past
[08:32] <plh> maybe we could leave him a few days
[08:32] <jgraham> I am with Ms2ger
[08:33] <krisk> I'll send him an email and if he doesn't have time to address in a few days I'll opend a new bug to remove them until he has time to fix them
[08:34] <krisk> That is the end checking for bugs on approved tests
[08:34] <krisk> next agenda item
[08:34]  * Zakim sees nothing on the agenda
[08:34] <krisk> #2 Should we pick a date for google's audio/video tests, similar to the canvas tests?
[08:35] <Ms2ger> I'd wait until Mark has replied to feedback
[08:36] <jgraham> Right, if he doesn't respond soon we need a plan b
[08:36] <krisk> I hope he responds back...
[08:36] <plh> didn't we ping him in the past already?
[08:36] <jgraham> But I don't think we are there yes
[08:36] <jgraham> *yet
[08:36] <krisk> yes he has been pinged in the past
[08:36] <krisk> let's wait till the next time we meet
[08:37] <krisk> If he can't fix some of the issues we can propose a patch to his tests
[08:37] <krisk> and then set a date to have them approved
[08:37] <Ms2ger> Sounds good
[08:40] <krisk> jgraham do you think Simon Pieters could look at these Audio and Video tests from Opera?
[08:41] <krisk> ...and send some feedback to the list
[08:42] <krisk> If Mark fixes the bugs does anyone object to picking date of march 8th to approve these tests?
[08:42] <jgraham> krisk: I can ask
[08:43] <jgraham> March 8th sounds fine
[08:43] <krisk> of course that  would assume all feedback has been addressed
[08:43] <krisk> Ms2ger do you agree?
[08:43] <Ms2ger> Once someone at Opera has looked at them, they can go in, IMO
[08:43] <krisk> Ok
[08:44] <krisk> Moving on to agenda item #3
[08:44] <Ms2ger> I won't be able to make meetings in the next few months
[08:44] <krisk> where in the world are you based?
[08:45] <krisk> This time seems to work for folks in Europe and east and west coast?
[08:45]  * plh suspects a comet or something
[08:45] <krisk> I think this same time/schedule works
[08:46] <krisk> unless we has someone participate from Australia or Asia
[08:46]  * jgraham thinks that Ms2ger would have greater latency if he was on a comet
[08:46] <Ms2ger> plh, a comet?
[08:46]  * plh :)
[08:47] <jgraham> Ms2ger: Where you are based, if you are having radio silence for a few months, I guess :)
[08:48] <Ms2ger> Keeping the current schedule is fine for me
[08:48] <krisk> Ok
[08:49] <krisk> I'll send an email to the list that just extends the schedule, same time, same channel, etc..
[08:49] <krisk> if someone objects they can propose a new time slot
[08:49] <krisk> Agenda item #4 Harness Updates
[08:50] <krisk> jgraham did you have any time to address any of the harness feedback?
[08:50] <jgraham> I just push some changes to testharness.js
[08:50] <jgraham> About 5 minutes ago
[08:50] <jgraham> So I probably broke the world
[08:50] <jgraham> But there is a setup() function
[08:51] <jgraham> and a way to stop the harness deciding it is done until you explicitly tell it so
[08:51] <jgraham> and more documentation
[08:51] <Ms2ger> And CRs?
[08:51] <jgraham> CRs?
[08:52] <krisk> code review
[08:52] <plh> james, are you planning to reply to Anderson?
[08:52] <jgraham> Oh. I would love code review. I didn't think we had a system for that
[08:52] <Ms2ger> U+000D
[08:53] <jgraham> plh: I had forgotten about that. Yes. But it feels like we might have quite different requirements
[08:54] <plh> :(. it's a shame if they're going to fork but if we really can't accommodate...
[08:54] <jgraham> Ms2ger: Oh. Yes. That would be a stupid thing to do
[08:54] <jgraham> plh: Well afaict they fundamentally don't want something that is pass/fail
[08:54] <jgraham> Or shouldn't
[08:55] <krisk> I am quite sure anderson will respond to the list
[08:56] <krisk> i suspect it's possible to reach an agreement that doesn't result in a complete fork of the harness
[08:56] <plh> maybe we should ask Anderson to be around next time
[08:57] <plh> and see if the difference of approach could be resolved
[08:57] <plh> I'd really like them not forking
[08:57] <krisk> I'll send him an email and ask him to attend
[08:58] <krisk> Next agend item #5
[08:58] <krisk> Implementation reports
[08:58] <Ms2ger> Don't we get enough of them thrown at us?
[08:58] <krisk> with all the canvas tests that have been approved it seems like we should ask browser vendors to update their results
[08:59] <krisk> The key part is 'Their'
[08:59] <plh> I'm not doing anything with the test results nowadays
[08:59] <krisk> folks to send results to the list (good)
[08:59] <plh> http://w3c-test.org/html/tests/reporting/report.htm is in in limbo from what I can tell
[08:59] <Ms2ger> http://ms2ger.freehostia.com/tests/results/results.html is something I wrote up today
[08:59] <krisk> though they come from people that are not reps of the browser vendor
[09:01] <krisk> I know I can update IE9's data, though we would also want to get an update from Firefox
[09:01] <jgraham> I continue to think it is harmful to publish results at any official-looking location at this point
[09:01] <jgraham> If vendors want to use the data internally that is fine
[09:02] <jgraham> But we are nowhere near done
[09:02] <jgraham> (also, if there is some place encouraging people to report results, that should be changed)
[09:02] <krisk> Don't you think it would be very good feedback to the spec?
[09:03] <jgraham> Feedback on the spec?
[09:03] <jgraham> I haven't seen any feedback on the spec as a result of people running tests
[09:03] <krisk> Though we have ~800 some canvas tests
[09:03] <jgraham> Right, canvas is pretty much the only place where we have a useful amount of coverage
[09:03] <Ms2ger> jgraham, didn't you see Boris' color serialization feedback?
[09:04] <krisk> if we have a few browsers passing then it's a good signal to the world that indeed canvas can be built in an interoperable manner
[09:05] <Ms2ger> I think the world already knows that, tbh
[09:05] <jgraham> Ms2ger: Oh, I didn't realise that was a result of running the HTML testsuite
[09:05] <Ms2ger> Well, strictly as a result of me running it ;)
[09:05] <krisk> Then it should not be that big of a deal to get results and flush out any more issues with the spec
[09:05] <jgraham> I think that <canvas> is an exception to the general rule that what we have isn't useful
[09:06] <plh> I
[09:07] <jgraham> Also in W3C land <canvas> is a seperate deliverable
[09:07] <krisk> Since the spec is so large we do need to make progress
[09:07] <krisk> from a project management view...
[09:07] <plh> m ok with updating results as long as the results come from the browser vendor
[09:08] <krisk> waiting till the very end is not a good plan
[09:08] <jgraham> I don't see how uploading results or not changes whether we make progress
[09:08] <jgraham> The main thing we need to make progress is an indication of where we have good coverage
[09:08] <jgraham> and uploading results doesn't provide that
[09:08] <jgraham> Having an annotated version of the spec might
[09:08] <jgraham> (Philip has this for <canvas>)
[09:09] <jgraham> It should also be something to poitn contributers at if they want to know where they can help
[09:10] <plh> I can certainly improve the documentation for that effect
[09:10] <krisk> I don't think we will be able to resolve this today (10 minutes past meeting ending)
[09:10] <plh> I realize that it's lacking at the moment
[09:10] <jgraham> I would really like a system where we have metadata for each test indicating which sentence(s) it tests
[09:11] <jgraham> But that is a non-trivial amount of effort
[09:11] <jgraham> Even copying Philip's code
[09:11] <krisk> shall we adjourn?
[09:11] <plh> +1
[09:12] <Ms2ger> +2
[09:12] <jgraham> +3
[09:12] <krisk> Ok
[09:12] <Zakim> -Plh
[09:12] <Zakim> -krisk
[09:12] <Zakim> HTML_WG(HTMLT)11:00AM has ended
[09:12] <Zakim> Attendees were krisk, Plh

-----Original Message-----
From: public-html-testsuite-request@w3.org [mailto:public-html-testsuite-request@w3.org] On Behalf Of Kris Krueger
Sent: Monday, February 07, 2011 9:15 PM
To: 'public-html-testsuite@w3.org'
Subject: HTML Testing Task Force Conf Call Agenda 2/8/2011

Agenda

#1 Check for any bugs on approved tests
#2 Should we pick a date for google's audio/video tests, similar to the canvas tests?
#3 New meeting schedule (presume same time, same channel, etc...)
#4 Harness Update (from last week)
#5 Implementation reports - time to ask folks to repost results (ONLY browser owner's can post and it's optional)

If you have other items you would like, please email me directly.

-Thanks!

IRC #HTMLT
Time 16:00-17:00 UTC (11:00am-12:00pm Boston local) Zakim Bridge +1.617.761.6200, conference 48658

Received on Tuesday, 22 February 2011 04:06:23 UTC