Re: Would minimal tests for importKey/exportKey be worthwhile?

Okay, I'll work on this. As I said, all I'm really doing are minimal tests.
I'm working on a fork of the test suite at
https://github.com/engelke/web-platform-tests/tree/import_export . I'm
happy to give commit privilege to anyone who wants it, or you can just send
PRs.

Right now we have importKey being tested for valid source data for all
algorithms except RSA and EC ones. Both raw and jwk formats are tested, but
only a single, minimal jwk object is being tried.

Whenever the importKey is performed with extractable === true, the key is
then exported to the same format and compared to the original to check that
round trip works.

There is so much more that can be tested even on the success paths, to say
nothing of testing expected errors, but this is a start. I do expect to try
to add RSA and EC keys in the next few days.

Charlie

On Tue, Aug 2, 2016 at 6:10 PM, Eric Roman <ericroman@google.com> wrote:

> I am strongly in favor of importKey()/exportKey() tests.
>
> I think these are crucial, since importKey()/exportKey() is a rich area
> for implementation incompatibilities/bugs.
> Some great areas to test here are:
>
> * JWK
> * Asymmetric key validation
> * Enforcement of "usages"
>
> The sort of test structures I found to work well in Chrome were:
>
> * Import invalid keys and assert failure (largely test data driven; we
> have a corpus of bad keys you are free to use as a start).
> * Import/export roundtrip tests. This isn't perfect, but does simplify
> expressing the tests as well as generating the data. (test data involves a
> set of equivalent key serializations in different formats, or different
> expressions of the same format). Also easy to augment the data set by
> running generateKey() + exportKey() on a variety of Web Crypto
> implementations.
>
> Here is some key data that Chrome uses which we can contribute:
>
>
> https://chromium.googlesource.com/chromium/src/+/master/components/test/data/webcrypto/bad_ec_keys.json
>
> https://cs.chromium.org/chromium/src/components/test/data/webcrypto/ec_private_keys.json
>
> https://chromium.googlesource.com/chromium/src/+/master/components/test/data/webcrypto/bad_rsa_keys.json
>
> I think JWK is especially worthy of testing, given how new the format is,
> and the possibility for parsing issues or invalid comparison of "use" /
> "alg" to Web Crypto's equivalents. Selfishly, the other advantage of strong
> JWK testing is any bugs the test suite uncovers are likely to be fixed
> promptly in browsers (certainly I can address bugs in Chrome's JWK
> handling).
>
> Lastly, invalid private key import is a good area to test for
> implementation quality (although arguably less interesting for
> compatibility/compliance). We found that while mature crypto libraries were
> hardened against maliciously constructed public keys, the same level of
> scrutiny had not been given to private key import. For instance see
> CVE-2015-0209, which we uncovered when testing PKCS8 export.
>
> Cheers.
>
> On Tue, Aug 2, 2016 at 1:28 PM, Mark Watson <watsonm@netflix.com> wrote:
>
>> Yes, I think this would be worthwhile and we would be willing to help. I
>> will try and find some resource here, though I have not had much success
>> with that. If not, I can help myself, just not for a couple of weeks.
>>
>> We do have some WebCrypto tests from our old NfWebCrypto plugin here:
>> https://github.com/Netflix/NfWebCrypto/tree/master/web/jsTests
>>
>> ...Mark
>>
>> On Tue, Aug 2, 2016 at 1:15 PM, Charles Engelke <w3c@engelke.com> wrote:
>>
>>> The subject line says it all. I don't have the time (or expertise) to
>>> develop a full suite of tests for importKey and exportKey, but could test
>>> the "happy paths" that should succeed without throwing errors.
>>>
>>> Would this be worthwhile? I've already done this for everything but
>>> public key algorithms. Adding them would be more work, so before I dive in,
>>> is it even worth it? Or should we wait for someone who can build a more
>>> thorough suite?
>>>
>>> Charlie
>>>
>>
>>
>