Re: Testing WebDriver in Web Platform Tests

Hi Mike,

My immediate thought is that we want to avoid a few things:

   1. code duplication - particularly for more complex parameters, such as
   those described in the Actions section of the spec
   2. injecting errors - how easy will it be to test error conditions by
   sending malformed requests?

It's hard to say more without seeing code, so I'm looking forward to seeing
your examples :). I'd suggest having at least two cases, one with a simple
test case (e.g. navigating to a URL) and another with a complex request
object (e.g. set up an action chain and execute it).

Sam.

On Tue, Jan 24, 2017 at 10:24 AM Mike Pennisi <mike@bocoup.com> wrote:

Hello all,

I'm interested in helping out with improving coverage for the WebDriver
spec in
the Web Platform Tests project (I'm the contractor that David referred
to in a
recent posting [1]). I know that we have a fair number of old tests based on
the JSON Wire Protocol that are currently disabled. I'd like to re-use
existing
work where possible, but I'm also interested in designing a strategy
that will
encourage test readability and clarity in spec coverage. So before
writing too
many tests, I was hoping to get input from folks here.

Specifically: would anyone object to my not using the "wdclient" library
[2]?
We definitely need to take steps to lessen duplication, but I'd much
prefer to
use traditional unit testing patterns for this--"setup/teardown" routines,
helper functions, or even pytest's "fixture" feature. This will make the
tests
simpler and more direct, and that promotes readability (which is good for
"drive-by" contributors), traceability (which is good for implementers
wondering why a test fails), and specification parity (which is good for
test
suite maintainers trying to re-assess coverage of a given section of the
spec).
I think we need to prioritize these qualities over the developer ergonomics
provided by a binding library.

I'm currently working on a few examples to demonstrate what tests written
without a framework could look like; I'll share them here when I have a few
workable examples. In the mean time, does anyone here have any thoughts
about
my reasoning in general?

Mike

[1]
http://www.w3.org/mid/CAAoW2AFG0ZOzu6DyjMAPwZZ9r-He9oTy7dK7f4KhM12p=JToTQ@mail.gmail.com
[2] https://github.com/w3c/wdclient