Re: Testing progress

Re: key reuse. We're testing that we can generate a key, so it's going
through that slow, expensive, process every time the parameters are
good enough to get that far in the algorithm.

Now that I'm fairly far along, I want to see if there's use case
trimming that can cut that way down without reducing the actual scope
of test coverage.

Charlie

On Mon, May 2, 2016 at 2:00 PM, Harry Halpin <hhalpin@w3.org> wrote:
>
>
> On 05/02/2016 01:56 PM, Charles Engelke wrote:
>> I'll be on the call today. Here's a quick update on work I've been
>> doing on testing.
>>
>> My work in progress is at my fork of the Test the Web Forward
>> repository: https://github.com/engelke/web-platform-tests/tree/master/WebCryptoAPI.
>
> Thanks so much for the update! This is great work, exactly what the
> Working Group needs.
>>
>> I'm focusing on a single method at first: generateKey. I'm first
>> testing that every possible combination of proper parameters creates a
>> valid key with the correct properties, then that lots of possible
>> combinations of invalid parameters throws an error, AND that it throws
>> the correct error per the spec.
>>
>> Issues:
>>
>> - No test coverage of RSA-OAEP, ECDSA, or ECDH yet. I don't see any
>> particular problems adding them, I'm just not that far along yet.
>>
>> - Can't automate testing of PBKDF2 because it requires user input
>> outside control of the browser.
>
> We could test some collection of strings I imagine.
>>
>> - The tests are slow, and timeouts have been a problem. I think I've
>> handed that, but the test pages take many minutes to run. I think the
>> root problem is just that generating RSA keys is slow, and that there
>> are a lot of combination to check (there are currently 1800 "success"
>> tests and almost 30,000 "failure" tests).
>
> Could we just reuse the same key?
>>
>> I've only run these tests for Chrome and Firefox (and tried an
>> earlier, less complete version for Edge) so far. The browsers do very
>> well on the tests for successes, but have a fair number of issues on
>> tests for failures. They fail where they should, but not necessarily
>> by throwing the correct error. Even checking the correct error is
>> problematic, because the browsers don't necessarily have the necessary
>> DOMException property names defined in each case.
>>
>> Chrome's non-successes are for 192 AES keys, which it doesn't support.
>> Firefox's non-successes are for not supporting RSA-PSS.
> As long as two browsers support 192 keys, we can keep it in the spec.
>
> I think they said the  added RSA-PSS into nightly build.
>
>
>>
>> Charlie
>>
>
>

Received on Monday, 2 May 2016 18:11:06 UTC