RE: Testing progress for generateKey

Here is the pull request https://github.com/w3c/web-platform-tests/pull/3016

I intend to merge it into the test suite on Monday unless people object.  I think it does a good job of covering the needed cases.

Jim


-----Original Message-----
From: charles.engelke@gmail.com [mailto:charles.engelke@gmail.com] On Behalf Of Charles Engelke
Sent: Friday, May 13, 2016 2:22 PM
To: public-webcrypto@w3.org
Subject: Testing progress for generateKey

Tests are now available for all supported algorithms for generateKey.
They run as individual pages and under the test hahttps://developer.microsoft.com/en-us/microsoft-edge/platform/issues/7444732/rness,
including in workers.

Chrome passes all success tests except for those involving AES with 192-bit keys (which it does not support). Chrome fails some failure tests by reporting the wrong error in some cases where there are two bad parameters. It seems it may be checking them in a different order than in the spec. That has been reported as issue 611801, at https://bugs.chromium.org/p/chromium/issues/detail?id=611801&colspec=ID%20Pri%20M%20Stars%20ReleaseBlock%20Component%20Status%20Owner%20Summary%20OS%20Modified
.

Firefox passes all success tests. I used Firefox nightly because it has RSA-PSS support, which will apparently be in the next regular release. However, Firefox generation of RSA keys is several times slower than in Chrome, so the tests take 10-30 minutes to run.

Firefox fails most failure tests. In one case, Firefox seems to ignore bad usage parameters instead of throwing an error (bug 1270634 at https://bugzilla.mozilla.org/show_bug.cgi?id=1270634). In another case it reports the wrong error when given certain bad parameters (bug
1270599 at https://bugzilla.mozilla.org/show_bug.cgi?id=1270599).
There may be others; it takes a long time to run the tests and when there are thousands of failures for just these two root causes it takes a while to check to see if there are others.

Edge is far from complete, though it does pass many of the success tests as of now. I have only reported one bug to Microsoft: the CryptoKey object's algorithm name is set to whatever parameter is given to generateKey, instead of adjusted to match the official capitalization. That's issue 7444732 at https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/7444732/
.

I haven't tested Safari because it uses a prefixed implementation.

I've submitted the tests as a pull request and updated them based on feedback from Jim. I'm not sure of the next steps for having them accepted.

Charlie

Received on Saturday, 14 May 2016 06:17:45 UTC