- From: François Daoust <fd@w3.org>
- Date: Tue, 19 Jul 2011 17:04:53 +0200
- To: <public-test-infra@w3.org>
Hi,
The minutes of yesterday's call are available at:
http://www.w3.org/2011/07/18-testing-minutes.html
... and copied as raw text below.
Thanks,
Francois.
-----
18 Jul 2011
See also: [2]IRC log
[2] http://www.w3.org/2011/07/18-testing-irc
Attendees
Present
Shadi, Plh, francois, MikeSmith, wilhelm, Michael_Cooper,
Jeanne
Regrets
Chair
Philippe
Scribe
francois
Contents
* [3]Topics
1. [4]Scope missing APIs relevant to accessibility in WG
charter
2. [5]Framework
3. [6]Informal summer meeting
4. [7]Next call
* [8]Summary of Action Items
_________________________________________________________
Scope missing APIs relevant to accessibility in WG charter
shadi: comment 2 should be the one we should focus on
... related to accessibility, UI automation
... other ones are more editorial.
... or can be handled at your discretion.
mike: UI automation is something that is quite different from what
we have in the current scope section. My understanding is that is a
vendor specific platform and an OS specific API.
... Am I wrong about that?
Michael: yes and no. There's a Microsoft and an Apple API that are
vendor-specific.
... A testing API needs to be aware of these APIs.
Mike: The WebDriver API should be at the core of the charter. The
scope of it is well precised. It's something that runs in a Web
browser.
<shadi>
[9]https://lists.w3.org/Archives/Team/w3mreq/2011Jul/0083.html
[9] https://lists.w3.org/Archives/Team/w3mreq/2011Jul/0083.html
plh: I think Wilhelm said before that they take inspiration from
accessibility APIs.
mike: right, what I do not understand here is how UI automation
relates to this. I mean at the technical level.
... If I don't understand it, I can't write in the charter. If
someone else can propose text, then I could include it in the
charter, but I can't do that on my own.
... Another thing is that it seems kind of weird to me to mention UI
automation in a charter focused on Web browsers.
... I do not have enough details to be able to write up a
description to how this work relates to what we have in the charter.
wilhelm: I'm familiar with what we do here, at Opera. The different
parts of the spec are at various maturity level. The core of the
spec is only interested at what happens at the Web page level.
... In addition to the core API, there are some experimentations
going on but no consensus yet among browser vendors.
... I'd prefer to postpone extensions until we have more feedback
experience.
plh: ok, so maybe we're not clear enough in the charter. If we say
we're not going to standardize the UI part, then we should make it
clear in the charter.
<plh> [10]http://www.w3.org/2011/08/browser-testing-charter.html
[10] http://www.w3.org/2011/08/browser-testing-charter.html
[reviewing draft charter text: "there is a specific need to simulate
user actions such as clicking links, entering text, and submitting
forms"]
plh: what did you have in mind, Mike, with "any other potential APIs
useful in performing automated testing of Web applications"?
mike: I didn't have anything specific in mind.
... Happy to remove.
plh: that's the part you're most interested in, shadi, I believe. Is
your comment still relevant now that we know the API is only
intended to drive things in the HTML page and not in the browser
chrome?
Michael: I would say yes as you can still drive the HTML content
through the accessibility APIs.
... I think we need to acknowledge that.
plh: section 1.1 is about the scope. Trying to extend this section
is a mistake, I think. It should actually be tighter.
mike: I agree we need review. But what does "dependency" mean?
Michael: It's more a point of coordination.
plh: ok, let's see where we can put it, then.
[discussion about "draft notes" section that is to go away]
Michael: I would add a new sub-section under Dependencies and
Liaisons section, such as "relevant API".
<plh> - Accessibility Interoperability Alliance (AIA)
<plh> -- <[11]http://www.atia.org/aia/>
[11] http://www.atia.org/aia/%3E
<plh> - Open Ajax Alliance
Mike: that's easy enough to do, yes.
<plh> -- <[12]http://www.openajax.org/>
[12] http://www.openajax.org/%3E
Mike: Do they have some task force dedicated to accessibility I
could point to?
Michael: there is an accessibility sub-group of the Open Ajax
Alliance, but I would not restrict the coordination to it as the
whole things is relevant to our work.
Mike: I have enough information to update the charter. I would still
ask Michael to send more precise wording. "consider any other
potential APIs" is too loose, I think.
Shadi: the idea is "consider" as looking at the appropriate
relationship with existing solutions if, of course, organizations
are willing to contribute their work anyway.
Michael: one reason to mention vendor-specific APIs is to have
everyone be conscious of the overlap.
Mike: that won't happen. It is not duplicating anything that
everyone else has done. There is no existing API that works across
browsers right now.
plh: one way could be to expose these API in JavaScript.
Mike: right, but there is no standard mechanism that does what this
API does.
... It would be good to get some direct review from people in these
companies.
plh: Mike, to summarize, you need to run through Shadi's message.
Mike: ok, I can do that by tomorrow.
<MikeSmith> francois: what's the status on the IG charter:
<MikeSmith> plh: it's stable, no changes needed
Framework
<MikeSmith> [13]http://w3c-test.org/framework/
[13] http://w3c-test.org/framework/
mike: so, Francois deployed something not working and then cowardly
left for vacation leaving everything on my plate
... I had a look, and ran into weird problems, apparently because
PHP version was not 5.3, and also Apache version was too late.
... I upgraded everything on the server.
... The thing is working.
... I also replaced it with a clone of the CSS server Mercurial
repository.
... Now we can pull and push updates directly.
... What we could do this week is to setup things for it to work
with the Web Performance WG spec.
plh: Yes, good guinea pig. That's a good test.
Mike: everybody else who is interested in getting the framework
populated now that it's set up, I'm happy to help with that.
... It's team only because of SSH access.
... I think it's bad practice to all act as root on that server.
... Better to create a user account on that machine.
plh: sounds good to me.
shadi: if you can write this up somewhere, that would help.
plh: you still need someone with root access to the machine.
shadi: just an email that explains what to request from systeam and
what to setup on one's machine would be good not to forget steps
later on.
plh: how to add a group in the framework would be a good thing to
document
mike: pretty well documented, actually.
... The process is clear but requires a lot of manual work. We could
provide a better interface later on.
... You work with text files, and have to work with a few mysql
statements.
plh: ok, I don't think we need to focus so much on making things
automated at this stage.
mike: there are two pieces: the "framework" which Peter calls
"harness". The second is "shepherd" wich is the test suites
management system.
... I'm not sure I see that as a requirement.
plh: well, that's why we picked up the framework in the first place.
... So would be good to deploy.
mike: once we get the framework setup for Web Performances WG, we'll
be clearer as to what we need.
... That should be doable this week.
plh: we may need to update the tests, so could take longer.
... Another thing is multi-WG support.
<MikeSmith> here is Peter's how-to:
plh: Actually, I wonder if we want to tie this framework tied up to
working groups.
<MikeSmith>
[14]http://hg.csswg.org/dev/harness/raw-file/tip/docs/index.html
[14] http://hg.csswg.org/dev/harness/raw-file/tip/docs/index.html
plh: better to tie it to specifications to handle cases when a
working group get splitt up for instance.
mike: here's the thing. That's more or less how it works today.
... It's per spec and test suite. We could simply add a column to
link it to a working group.
<MikeSmith> francois: the harness is already set up to work with
different specs
<MikeSmith> … and we could easily add a column for WG info if we
want to
[francois thanks mike a lot for taking up the hard task of debugging
the deployment while he was enjoying sun on vacation!]
<MikeSmith> francois, debugging wasn't so hard, and Tom helped some.
I'm glad you got some vacation man
Informal summer meeting
shadi: I'm wondering what the process and expectations are.
... which kind of organizations are to attend the meeting. What's
team attendance expectations, that sort of things.
wilhelm: the general idea is to get an overview of what browser
vendors handle tests today. We need to know what vendors need to
test. And what they'd be ready to contribute.
shadi: are we only focusing on browser vendors? Aren't we going to
miss something?
wilhelm: to answer the two questions, that's good to narrow the
scope. We will have to extend things afterwards. In one or two days,
there are only so many stuff you can cover. I hope we'll kickstart a
long discussion here.
shadi: The fix-up later approach is typically what makes
accessibility added afterwards, which always creates problems. I'm
concerned we'd be leaving a bunch of stakeholders.
plh: this is an informal meeting. Not a group coming with a set of
requirements.
shadi: we want browser vendors to be more considerate about
accessibility issues.
<Zakim> MikeSmith, you wanted to say we have the consideration
already, and this is not a case of risking "fix it later"
Mike: we are already considering accessibility. Any group
considering work in W3C already considers accessibility, even if not
explicitly mentioned. I don't like the implication that we are
ignoring accessibility here, and using a fix-it-up later approach.
... It's not a workshop, it's an informal meeting.
plh: What I heard Wilhelm say is that we have browser vendors which
have an extensive set of test cases and do not know how to leverage
those in W3C.
wilhelm: that's correct
plh: We could say that anyone who has vast amount of tests is
welcome to attend to see how these can be contributed.
... It's not a meeting about coming up with requirements.
<gsnedders> I wonder how much of the infrastructure is going to be
shared for a11y testing, because if you're testing what's passed to
the OS APIs that's inevitable going to be a separate API to test.
Michael: what's missing here is that there are all ways to tackle
testing things. For me, it's not optional to have other stakeholders
here, it's important to have them.
<MikeSmith> which people
shadi: we are missing people who are doing accessibility testing.
<jeanne> there is a outside group doing accessibility testing - I
will look up the name
<MikeSmith> bingo
shadi: The more we shortcut the deadlines, the more difficult it is
to get people on board.
<MikeSmith> there is interest from browser vendors
plh: the goal is to see if there's interest from browser vendors in
the first place.
<MikeSmith> the lack of a response from them so far does not
indicate they are not interested
shadi: I do think we do need to take that we are trying to involve
other players, and we should take care not to leave them aside
unintentionally.
plh: what we want to learn is how people do testing rather than how
we should be doing testing.
... That's why I'm more inclined to attract people that have good
amounts of tests.
<Zakim> MikeSmith, you wanted to ask Michael to explain exactly what
things and to say the idea is to keep this meeting small
plh: I need someone to take an action item to come up with a precise
list.
Mike: in some comments you're making, Shadi, you're telling us that
we may be forgetting accessibility but you're staying vary vague.
<jeanne> jeanne found the group, and it is the same group MichaelC
has been discussing. Sorry for distraction.
Mike: Another point is that it is to remain an informal meeting,
focused, with fewer people. The idea is to get these people into the
same room.
... The goal is to save time, see how we can stop reinventing the
wheel across groups. It would be a good idea to see people
representing accessibility concerns but, as mentioned before, people
involved in browser development are already addressing accessibility
concerns in their everyday work.
shadi: we're really not having diverging points at all.
... Having accessibility as part of every developer day job is a
good goal, but it's still a goal, not the situation today, even
within W3C.
... There has to be a compromise between the number of people
attending this meeting and the number of viewpoints we can have.
... I think we're all having the same vision for the future.
plh: What I'd like to have to move forward is to identify
individuals, including accessibility.
... That's what Wilhelm did, targeting specific individuals, not
companies.
... It would be good if you could come up with a list of people,
Shadi.
<MikeSmith> while I agree that making specific mention of
accessibility coordination in charters in activity statements and
other places does not imply that accessibility concerns are being
neglected or that there is not a high level of accessibility
consideration among the people who produced the activity statements
and charters
shadi: I'm asking Jeanne if she can take this up.
... We'll try to come up with a list.
wilhelm: goal is to have the meeting as soon as humanly possible.
mike: one thing about timing we should agree on is that when we have
critical mass, we still need to announce it, about a month in
advance for travel arrangements.
plh: we're going to have to announce it as well in some fashion.
... We're going to have to strike a balance between open and closed
door.
... Not a proper workshop (no position paper).
... We should have a list of questions ready for people interested
to show up to check whether we need them in the meeting.
... We need people with a large number of tests who would like to
contribute them.
<shadi> [15]http://www.w3.org/2005/10/Process-20051014/events.html
[15] http://www.w3.org/2005/10/Process-20051014/events.html
shadi: note there is no requirement for position papers to call some
event a workshop.
Next call
plh: I'm on the road next call, so propose to cancel it.
<jeanne> jeanne is on vacation
plh: Next call on August, 1st.
[call adjourned]
Summary of Action Items
[End of minutes]
Received on Tuesday, 19 July 2011 15:05:19 UTC