W3C home > Mailing lists > Public > public-html-testsuite@w3.org > September 2010

RE: HTML Testing Task Force Conf Call Agenda 8/24/2010

From: Kris Krueger <krisk@microsoft.com>
Date: Thu, 2 Sep 2010 16:16:49 +0000
To: "'public-html-testsuite@w3.org'" <public-html-testsuite@w3.org>
Message-ID: <FA9085650D2F8B4A9D501ED9D9706E9A14AD594A@TK5EX14MBXW652.wingroup.windeploy.ntdev.microsoft.com>
Below is the IRC log from the meeting and links (though I get permission denied).

This morning I moved the Opera getElementsByClassName tests and integrated them into the test runner.
Currently these test log the result without manually clicking the Pass/Fail/Not Implemented buttons.

-Kris

http://www.w3.org/2010/08/24-htmlt-minutes.html
http://www.w3.org/2010/08/24-htmlt-irc

*** krisk [477029dc@128.30.52.43] has joined #htmlt
*** krisk Ms2ger gsnedders @jgraham
*** Channel created on Sun Aug 15 01:32:28 2010
<krisk> trackbot-ng, start telcon
<krisk> zakim, list conferences
*** Zakim [rrs-bridgg@128.30.52.169] has joined #htmlt
<krisk> zakim, this is htmlt
<Zakim> ok, krisk; that matches HTML_WG(HTMLT)11:00AM
<krisk> trackbot-ng, start telcon
*** trackbot [trackbot@128.30.52.169] has joined #htmlt
<trackbot> Sorry... I don't know anything about this channel
<trackbot> If you want to associate this channel with an existing Tracker, please say 'trackbot, associate this channel with #channel' (where #channel is the name of default channel for the group)
*** plh [plh@128.30.52.28] has joined #htmlt
* plh will join in 2 minutes
<krisk> OK
<krisk> Just getting conf call setup...
*** RRSAgent [rrs-loggee@128.30.52.169] has joined #htmlt
<RRSAgent> logging to http://www.w3.org/2010/08/24-htmlt-irc
<krisk> zakim, who is here?
<Zakim> On the phone I see krisk
<Zakim> On IRC I see RRSAgent, plh, trackbot, Zakim, krisk, Ms2ger, gsnedders, jgraham
<jgraham> I think gsnedders and I will not be around much
<krisk> Will anyone else be dialing in?
* plh zakim, who is on the phone?
* Zakim sees on the phone: krisk
* plh zakim, call plh-work
* Zakim ok, plh; the call is being made
<Zakim> +Plh
<jgraham> We have to leave at 17:25
<krisk> That is fine - should be a quick meeting
<krisk> Agenda-> http://lists.w3.org/Archives/Public/public-html-testsuite/2010Aug/0010.html
<krisk> Chair: krisk
<krisk> Scribe: krisk (note most folks are on IRC)
<krisk> Agenda Item #1 Check for any bugs on approved tests (currently zero)
<krisk> Ms2ger - posted about XHTML5 tests see -> http://lists.w3.org/Archives/Public/public-html-testsuite/2010Aug/0011.html
<krisk> I'll take a peek at this feedback and report back, if others have feedback (not on default == automated) please respond
<krisk> Now the other feedback is to have automated be the default for tests - manual tests need a reason
<Ms2ger> krisk, FWIW, I can rewrite those xhtml test as reftests if you'd like
<plh> the problem is that we have no way to run reftests
<jgraham> Ms2ger: (I have a mild preference for javascript tests since there is somewhat less that can go wrong)
<plh> ie, the tests would still be ran manually
<krisk> For simple DOM tests (e.g. getElementsbyClassName) they should be automated
<jgraham> plh: The CSSWG are using reftests exclusively for CSS3
<plh> and do they have a way to run them?
<jgraham> Not sure, I will talk to fantasai/Tab
<plh> I doubt they do
<jgraham> But browser vendors can all run reftests now
<krisk> Nope - they are device/os dependent
<plh> I guess one thing we can do: let the harness run the reftests manually, and if someone has a better way to run those tests, good for them
<jgraham> That is better
<plh> so, maybe we need to modify the harness to allow reftests to be ran manually for now
<krisk> Let's agree to that
<plh> at least, we'll be able to accept reftests
<Ms2ger> Sounds good
<jgraham> In practice being able to run reftests is necessary to automate other W3C testsuites so I don't think it is any problem for us to have the same requirement
<jgraham> Having manual tests is always a problem though
<jgraham> and is causing problems in practice with CSS 2.1
<plh> yes, but I also don't want top exclude some class of mobile user agents for example in the process of developing reftests
<plh> s/top/to/
<jgraham> Nor do we
<krisk> OK then let's state this as our plan...
<jgraham> So reftests should always be possible to run manually too
<plh> sounds great to me
<krisk> Tests that can be tested via javascript should not be manual
<plh> so Kris, how hard would it be for you to allow reftests in the harness?
<jgraham> i.e. they should always have human-readable instructions
<krisk> Tests that need some non-javascript verification need to have manual instructions in the test
<plh> for reftests, it's a simple comparison
<krisk> That should not be a problem
<plh> so you need to be able to display two files
<krisk> Let's move on to the next agenda item
<jgraham> You want to display them in a way that allows you to flip between them with the tests in the same place in the viewport
<jgraham> Like in two tabs
<jgraham> Makes spotting small differences easier
<Ms2ger> What jgraham said
<plh> and the default instruction should be "For this test to pass, the two following pages must be exactly identically."
<krisk> sounds good
<krisk> glad to see we are making progress
<plh> I guess, we need a file naming convention or something
<krisk> a simple .ref. in the file name - e.g. test.html and test.ref.html
<plh> yep
<Ms2ger> Mozilla uses -ref, btw
<plh> we're using names like a-href at the moment
<plh> using -ref might clash with that
<krisk> they only do internally - not for tests on the w3 site
<krisk> Agenda Item #2 #2 Approve 25 more of Philip Taylors Canvas Tests
* jgraham has to go now
<krisk> I looked at tests from http://test.w3.org/html/tests/submission/PhilipTaylor/canvas/size.attributes.setAttribute.trailingjunk.html to http://test.w3.org/html/tests/submission/PhilipTaylor/canvas/toDataURL.arguments.3.html
<krisk> The look fine to me - any objections?
<plh> no objection from me
<krisk> gsnedders - I assume you are OK given the past IRC chats
<plh> let's assume so
<krisk> OK
<krisk> Agenda item #3 Conditionally approve Opera and Microsofts getElementsByClassName tests - additional work is test harness integration.
<krisk> I'd like to approve these and move them into the harness so they can be good examples of automated tests
<plh> sounds good to me as well. do we have overlap between the two series of tests?
<krisk> We have discussed this in the past, I just want to formalize this work
<krisk> I don't think so - their is only like ~40 total tests
<plh> I'm fine with approving both series of tests
<krisk> Given the ways that this API can be used, it's possible to create alot more tests
<krisk> Though the value of additional tests goes down pretty fast...
<krisk> Which is why the API works well today across borwsers (except in cases like namespaces)
<krisk> OK then let's conditionally approve these tests
* Ms2ger Yay namespaces
<krisk> Any other items people want to discuss? Or shall we adjourn?
<Ms2ger> krisk, I'd like to submit some of my own tests
<Zakim> -Plh
<Zakim> -krisk
<Zakim> HTML_WG(HTMLT)11:00AM has ended
<Zakim> Attendees were krisk, Plh
<krisk> rrsagent, generate minutes
<RRSAgent> I have made the request to generate http://www.w3.org/2010/08/24-htmlt-minutes.html krisk

-----Original Message-----
From: public-html-testsuite-request@w3.org [mailto:public-html-testsuite-request@w3.org] On Behalf Of Kris Krueger
Sent: Monday, August 23, 2010 9:39 PM
To: 'public-html-testsuite@w3.org'
Subject: HTML Testing Task Force Conf Call Agenda 8/24/2010

Agenda

#1 Check for any bugs on approved tests (currently zero)
#2 Approve the next 25 more of Philip Taylors Canvas Tests
#3 Conditionally approve Opera and Microsofts getElementsByClassName tests - additional work is test harness integration.

If you have other items you would like, please email me directly.

-Thanks!

IRC #HTMLT
Time 16:00-17:00 UTC (11:00am-12:00pm Boston local) Zakim Bridge +1.617.761.6200, conference 48658
Received on Thursday, 2 September 2010 16:17:23 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Thursday, 2 September 2010 16:17:23 GMT