Re: [WebCryptoAPI-tests][test-infra] Import a Mozilla Crypto API test. (#1490)

@ms2ger - Thanks for putting this together. I don't want to discourage the contribution, but I do think there are a few issues. I'm talking with a couple members of the Chromium team to try to come up with a more concrete set of suggestions for how we might want to test Web Crypto, so that we can circulate with implementors. In particular, the hope is that by setting some clear goals/structure, we make it easy for anyone to translate all of our respective tests into a common framework.

A few bits of concrete feedback on this PR:
- This starts with a DH test, which is currently only Firefox. While a good test for implementation-specific tests, starting with virtually any other algorithm will help ensure that we're testing the framework/harness across other browsers
- The script uses absolute paths (such as /resources/testharness.js). This may make it difficult to run via file:// URLs. I don't know if that's a hard block, but it's something to consider.
- A massive test-vectors.js is going to cause a lot of problems, as it is only going to grow. We're talking about many thousands of lines of vectors. We should look at breaking it down - at a minimum, algorithm specific, but better yet, test specific.
- Unused code in the test vectors and util.js seems suboptimal, and if we don't nuke it now, may make it harder to nuke / detect unused in the future.
- Invalid JWK data that is unused in the test vectors (e.g. look for / in base64url encoded data for x)

As I mentioned, I've poked some of the people responsible for implementation and testing on the Chromium side (who lack github accounts) to poke about this. I hope to have more to share once everyone is back from holidays. Web Crypto will be a little bit of a special case, because

1. We can't test ``window.crypto.subtle`` behaviour without having an algorithm to test for.
2. Some tests about key families/formats will be generic for that key family, but need to be tested per-algorithm.
  * For example, testing RSA SPKI import may need to be duplicated across tests for ``RSA-OAEP``, ``RSA-PSS``, and ``RSASSA-PKCS1_v1_5``
  * For example, AES key size tests may need to be duplicated across ``AES-GCM``, ``AES-CTR``, etc
3. Implementations may not support a given algorithm, so there may be certain multiplicative effects for duplication of tests to cover the permutations.
  * For example, deriving a key with ``ECDH`` requires a destination key algorithm. We may need to test the permutations of target algorithms for what an implementation supports




View on GitHub: https://github.com/w3c/web-platform-tests/pull/1490#issuecomment-68311708

Received on Monday, 29 December 2014 22:43:39 UTC