[whatwg] Cross browser automated tests

Collin, James, and I have been working on cross-browser security
testing as well.  We've implemented a subset of the WebKit
LayoutTestController for Firefox and ran the WebKit security
LayoutTests on both Firefox 2 and Firefox 3 RC1.

http://crypto.stanford.edu/websec/cross-testing/

There are a number of interesting failures, which we are investigating.

On Tue, May 20, 2008 at 7:51 PM, Ian Hickson <ian at hixie.ch> wrote:
> On Wed, 21 May 2008, Sylvain Pasche wrote:
>> To meet these goals, such an API should be as unobtrusive as possible:
>> * One .js file to include
>
> I recommend not embedding any JS, but instead requiring that the following
> two lines be used to report results (or something like them):
>
>   if (parent.reportResults)
>     parent.reportResults(...);

The WebKit LayoutTestController might be a reasonable starting point
for a testing API.  Just calling parent.reportResults might be
limiting for security testing because a frame might not be able to
access its parent.  For example, the LayoutTestController has a
globally readable and writable boolean flag that helps cross-domain
frames signal that the test is complete.

> ...and using <iframe>s to embed the tests one after another.

This is also limiting for security tests because some behavior must be
tested in a top-level frame.

> This makes the test even easier to maintain, and also makes it a lot
> easier to reimplement the harness or to share tests between hardnesses.

One of our goals for the project is to integrate with Firefox's
testing harness.  Ideally, the Firefox source tree could check out the
WebKit LayoutTests unmodified and run them as part of their
per-checkin regression testing.

Adam

Received on Tuesday, 20 May 2008 21:17:41 UTC