W3C home > Mailing lists > Public > public-webcrypto@w3.org > May 2016

Re: Testing progress

From: Charles Engelke <w3c@engelke.com>
Date: Mon, 2 May 2016 14:10:38 -0400
Message-ID: <CAFeVzdzpY6Kbi0Yih5ZicBgkhoBQxa+6cN9cR1Uw8QEckg1P7A@mail.gmail.com>
To: Harry Halpin <hhalpin@w3.org>
Cc: "public-webcrypto@w3.org" <public-webcrypto@w3.org>
Re: key reuse. We're testing that we can generate a key, so it's going
through that slow, expensive, process every time the parameters are
good enough to get that far in the algorithm.

Now that I'm fairly far along, I want to see if there's use case
trimming that can cut that way down without reducing the actual scope
of test coverage.


On Mon, May 2, 2016 at 2:00 PM, Harry Halpin <hhalpin@w3.org> wrote:
> On 05/02/2016 01:56 PM, Charles Engelke wrote:
>> I'll be on the call today. Here's a quick update on work I've been
>> doing on testing.
>> My work in progress is at my fork of the Test the Web Forward
>> repository: https://github.com/engelke/web-platform-tests/tree/master/WebCryptoAPI.
> Thanks so much for the update! This is great work, exactly what the
> Working Group needs.
>> I'm focusing on a single method at first: generateKey. I'm first
>> testing that every possible combination of proper parameters creates a
>> valid key with the correct properties, then that lots of possible
>> combinations of invalid parameters throws an error, AND that it throws
>> the correct error per the spec.
>> Issues:
>> - No test coverage of RSA-OAEP, ECDSA, or ECDH yet. I don't see any
>> particular problems adding them, I'm just not that far along yet.
>> - Can't automate testing of PBKDF2 because it requires user input
>> outside control of the browser.
> We could test some collection of strings I imagine.
>> - The tests are slow, and timeouts have been a problem. I think I've
>> handed that, but the test pages take many minutes to run. I think the
>> root problem is just that generating RSA keys is slow, and that there
>> are a lot of combination to check (there are currently 1800 "success"
>> tests and almost 30,000 "failure" tests).
> Could we just reuse the same key?
>> I've only run these tests for Chrome and Firefox (and tried an
>> earlier, less complete version for Edge) so far. The browsers do very
>> well on the tests for successes, but have a fair number of issues on
>> tests for failures. They fail where they should, but not necessarily
>> by throwing the correct error. Even checking the correct error is
>> problematic, because the browsers don't necessarily have the necessary
>> DOMException property names defined in each case.
>> Chrome's non-successes are for 192 AES keys, which it doesn't support.
>> Firefox's non-successes are for not supporting RSA-PSS.
> As long as two browsers support 192 keys, we can keep it in the spec.
> I think they said the  added RSA-PSS into nightly build.
>> Charlie
Received on Monday, 2 May 2016 18:11:06 UTC

This archive was generated by hypermail 2.3.1 : Monday, 2 May 2016 18:11:06 UTC