Re: Testing WebDriver in Web Platform Tests

 > Keep in mind that when a test calls a command, that command is not 
always the
 > command under test, so convenience functions are still useful.

What this particular issue comes down to for me is the use of exceptions for
expected behavior. For example, it's somewhat indirect to express "Switch to
Window should return an error," by writing "the wdclient library should 
an exception." That looks much more like a test for the library than for the
protocol (and again, it puts more extraneous code under test, raising 
like, "is the library currently authored to throw the correct error 
under the
correct conditions?").

I do agree that building assertions into the framework can help catch
unexpected errors, but there's a case to be made for making a distinction in
how we express expectations in these two contexts:

- Assertions for "mainline" behavior (e.g. the 200 from typical 
invocations of
   "Execute Script" that you might make from a test for "Get Title") 
should be
   completely implicit and exhaustive as possible. This maximizes the 
   we'll identify strange corner cases while keeping the test code concise
- Assertions for behavior under test should be as explicit and direct as
   possible (promoting test readability and ensuring that they are 
   exactly what they claim to be validating).

In other words, exception generation is another aspect differentiating the
"high level" API (used for convenience) from the "low level" API (used to
express behavior under test).

Received on Friday, 27 January 2017 00:20:44 UTC