Re: Testing progress

I also believe based on my testing it is not supported by any other UA and
as such believe it is a good candidate for removal.

Ryan

On Mon, May 2, 2016 at 12:09 PM, Tim Taubert <ttaubert@mozilla.com> wrote:

> Eric Roman wrote:
> >
> >
> > On Mon, May 2, 2016 at 11:10 AM, Charles Engelke <w3c@engelke.com
> > <mailto:w3c@engelke.com>> wrote:
> >
> >     Re: key reuse. We're testing that we can generate a key, so it's
> going
> >     through that slow, expensive, process every time the parameters are
> >     good enough to get that far in the algorithm.
> >
> >     Now that I'm fairly far along, I want to see if there's use case
> >     trimming that can cut that way down without reducing the actual scope
> >     of test coverage.
> >
> >     Charlie
> >
> >     On Mon, May 2, 2016 at 2:00 PM, Harry Halpin <hhalpin@w3.org
> >     <mailto:hhalpin@w3.org>> wrote:
> >     >
> >     >
> >     > On 05/02/2016 01:56 PM, Charles Engelke wrote:
> >     >> I'll be on the call today. Here's a quick update on work I've been
> >     >> doing on testing.
> >     >>
> >     >> My work in progress is at my fork of the Test the Web Forward
> >     >> repository:
> >
> https://github.com/engelke/web-platform-tests/tree/master/WebCryptoAPI.
> >     >
> >     > Thanks so much for the update! This is great work, exactly what the
> >     > Working Group needs.
> >     >>
> >     >> I'm focusing on a single method at first: generateKey. I'm first
> >     >> testing that every possible combination of proper parameters
> >     creates a
> >     >> valid key with the correct properties, then that lots of possible
> >     >> combinations of invalid parameters throws an error, AND that it
> >     throws
> >     >> the correct error per the spec.
> >     >>
> >     >> Issues:
> >     >>
> >     >> - No test coverage of RSA-OAEP, ECDSA, or ECDH yet. I don't see
> any
> >     >> particular problems adding them, I'm just not that far along yet.
> >     >>
> >     >> - Can't automate testing of PBKDF2 because it requires user input
> >     >> outside control of the browser.
> >
> >
> > RE: generateKey() for PBKDF2:
> >
> > Aside from the problem with automation, there is a bigger issue -- does
> > any User Agent actually implement generateKey() for PBKF2?
> >
> > Chrome certainly doesn't.
> > And as far as I can tell neither does Firefox
> > (I can only speak authoritatively for Chromium's implementation).
> >
> > I raised my concerns with PBKDF2's specification in this bug:
> > https://www.w3.org/Bugs/Public/show_bug.cgi?id=27689
> >
> > I don't believe as specified generateKey() is implementable -- or at the
> > very least would lead to a *good* implementation.
> >
> > I am happy to re-visit Chrome's implementation if we generate more
> > discussion around this issue.
> >
> > Barring that if no UA implements it and the WG isn't able to respond to
> > feedback on it, I say we remove it from the spec.
>
> +1. We have no current plans to implement it. It's heavily
> underspecified, and can't possibly work for (Service)Workers.
>
> - Tim
>
>
> >     > We could test some collection of strings I imagine.
> >     >>
> >     >> - The tests are slow, and timeouts have been a problem. I think
> I've
> >     >> handed that, but the test pages take many minutes to run. I think
> the
> >     >> root problem is just that generating RSA keys is slow, and that
> there
> >     >> are a lot of combination to check (there are currently 1800
> "success"
> >     >> tests and almost 30,000 "failure" tests).
> >
> >
> > Unfortunately this is going to be slow, and we can't really fix that :(
> >
> > RSA key generation is extremely slow.
> >
> > Your test surely wants to try a variety of key sizes, so sticking to
> > small key sizes (which would be on way to improve the performance) is
> > not really an option.
> >
> >     >
> >     > Could we just reuse the same key?
> >     >>
> >     >> I've only run these tests for Chrome and Firefox (and tried an
> >     >> earlier, less complete version for Edge) so far. The browsers do
> very
> >     >> well on the tests for successes, but have a fair number of issues
> on
> >     >> tests for failures. They fail where they should, but not
> necessarily
> >     >> by throwing the correct error. Even checking the correct error is
> >     >> problematic, because the browsers don't necessarily have the
> >     necessary
> >     >> DOMException property names defined in each case.
> >
> >
> > What property names are missing?
> >
> >
> >     >>
> >     >> Chrome's non-successes are for 192 AES keys, which it doesn't
> >     support.
> >     >> Firefox's non-successes are for not supporting RSA-PSS.
> >     > As long as two browsers support 192 keys, we can keep it in the
> spec.
> >
> >
> > Other things to be testing for generateKey() for RSA:
> >
> >   * Supported key size
> >   * Supported public exponent.
> >
> > We will see differences here -- Chrome has a minimum and maximum allowed
> > key size for RSA, and also restricts the public exponent to being 3 or
> > 65537 (more details at https://www.chromium.org/blink/webcrypto).
> >
> > Side comment: not sure if we would want this under the conformance test
> > suite, or leave it to implementation testing.. but when testing our own
> > implementation we had an early bug where generated keys were
> > unextractable (exportKey() would fail on them even when marked
> > extractable). To catch these sorts of problems you would ideally run all
> > the operation tests on each successfully generated key, and verify that
> > all key usages are respected, and that operations work as expected on
> > the key.
> >
> >     >
> >     > I think they said the  added RSA-PSS into nightly build.
> >     >
> >     >
> >     >>
> >     >> Charlie
> >     >>
> >     >
> >     >
> >
> >
>
>