Re: Would minimal tests for importKey/exportKey be worthwhile?

I am strongly in favor of importKey()/exportKey() tests.

I think these are crucial, since importKey()/exportKey() is a rich area for
implementation incompatibilities/bugs.
Some great areas to test here are:

* JWK
* Asymmetric key validation
* Enforcement of "usages"

The sort of test structures I found to work well in Chrome were:

* Import invalid keys and assert failure (largely test data driven; we have
a corpus of bad keys you are free to use as a start).
* Import/export roundtrip tests. This isn't perfect, but does simplify
expressing the tests as well as generating the data. (test data involves a
set of equivalent key serializations in different formats, or different
expressions of the same format). Also easy to augment the data set by
running generateKey() + exportKey() on a variety of Web Crypto
implementations.

Here is some key data that Chrome uses which we can contribute:

https://chromium.googlesource.com/chromium/src/+/master/components/test/data/webcrypto/bad_ec_keys.json
https://cs.chromium.org/chromium/src/components/test/data/webcrypto/ec_private_keys.json
https://chromium.googlesource.com/chromium/src/+/master/components/test/data/webcrypto/bad_rsa_keys.json

I think JWK is especially worthy of testing, given how new the format is,
and the possibility for parsing issues or invalid comparison of "use" /
"alg" to Web Crypto's equivalents. Selfishly, the other advantage of strong
JWK testing is any bugs the test suite uncovers are likely to be fixed
promptly in browsers (certainly I can address bugs in Chrome's JWK
handling).

Lastly, invalid private key import is a good area to test for
implementation quality (although arguably less interesting for
compatibility/compliance). We found that while mature crypto libraries were
hardened against maliciously constructed public keys, the same level of
scrutiny had not been given to private key import. For instance see
CVE-2015-0209, which we uncovered when testing PKCS8 export.

Cheers.

On Tue, Aug 2, 2016 at 1:28 PM, Mark Watson <watsonm@netflix.com> wrote:

> Yes, I think this would be worthwhile and we would be willing to help. I
> will try and find some resource here, though I have not had much success
> with that. If not, I can help myself, just not for a couple of weeks.
>
> We do have some WebCrypto tests from our old NfWebCrypto plugin here:
> https://github.com/Netflix/NfWebCrypto/tree/master/web/jsTests
>
> ...Mark
>
> On Tue, Aug 2, 2016 at 1:15 PM, Charles Engelke <w3c@engelke.com> wrote:
>
>> The subject line says it all. I don't have the time (or expertise) to
>> develop a full suite of tests for importKey and exportKey, but could test
>> the "happy paths" that should succeed without throwing errors.
>>
>> Would this be worthwhile? I've already done this for everything but
>> public key algorithms. Adding them would be more work, so before I dive in,
>> is it even worth it? Or should we wait for someone who can build a more
>> thorough suite?
>>
>> Charlie
>>
>
>