Request for bikeshedding

'Afternoon!

As we are making progress on the WebDriver test suite, it's about time
we figure out how other working groups can use our stuff to automate
their currently manual tests. WebDriver should be useful for many
other working groups, including CSS, HTML and WebApps.

I have thrown together an example for discussion here:

  https://github.com/wilhelm/web-platform-tests/tree/wilhelm/webdriver-examples/webdriver/examples

The suite consists of a Python script for the whole test suite, plus
one atomic HTML file for each test.

The HTML files contain human-readable steps and pass conditions, and
can be run and understood independently of the WebDriver script. This
is required both to be able to run the tests manually in browsers
without WebDriver support, and to assist browser vendor developers
fixing bugs triggered by a particular test case.

The WebDriver script loads each of the test files in sequence,
performing steps identical to those a human would, checking for the
same assertions.

This pattern would only be used for the subset of tests that require
some sort of human interaction. reftests and testharness.js tests are
unaffected.

We need to make both setup and authoring of WebDriver tests anywhere
in the web-platform-test repo as simple as possible. Ideally before we
start pushing our conventions onto other groups. Hence this request
for bikeshedding. What sources of friction can you see in the proposed
format?

On my own list of issues are:

 * We are currently not using wptserve. This is being worked on.

 * We don't pull in all the APIs we might need. I had to manually pull
in ActionChains.

 * The required files to run WebDriver tests are essentially part of
the WebDriver conformance test suite, not a shared resource between
all test suites in web-platform-test.

 * Referencing self.driver on every second line gives me writer's
cramp. We should alias this to something shorter in our helper files.

 * The same applies to the find methods. 99% of all tests in
web-platform-test will probably use find_element_by_css_selector. It
would be great to alias this to something shorter, as a poor man's
page object. b.find('#result') would be nice.

 * Writing test names like test_onclick_handler_on_button is suboptimal.

What else?

And which of the above are fixable - and desirable to fix? :)

-- 
Wilhelm Joys Andersen
http://wja.no

Received on Thursday, 27 February 2014 02:18:30 UTC