Re: [W3C Web Crypto WG] about tests

It's pretty far back in my queue.  I've looked at it briefly, but every
time I try to get started, I run into the wall of useless documentation on
testthewebforward.com.

It might help to have a consolidated event to get some focused attention,
even if it's only for the testing guys to explain how they like their tests
done.

--Richard

On Tue, Dec 16, 2014 at 8:47 AM, GALINDO Virginie <
Virginie.Galindo@gemalto.com> wrote:
>
> Dear implementers,
> Any bandwidth on your side to start a consistent test suite, allowing to
> demonstrate in the end the interoperability of your implementations in
> march 2015 ?
>
> Harry,
> How can W3C help ?
> Mandating one person to work directly on test definition, one person to
> help implementers to convert their tests, money and office to organizing a
> test day ?
>
> All,
> Do we need a dedicated call in January for that ?
>
> Virginie
>
>
>
>
> -----Original Message-----
> From: Harry Halpin [mailto:hhalpin@w3.org]
> Sent: mardi 16 décembre 2014 12:03
> To: GALINDO Virginie; public-webcrypto@w3.org
> Subject: Re: [W3C Web Crypto WG] about tests
>
>
>
> On 12/16/2014 11:46 AM, GALINDO Virginie wrote:
> > Harry,
> > have tracked the references to implementer's test repository here :
> https://www.w3.org/2012/webcrypto/wiki/Main_Page#WG_Testing_activity,
> based on additional information shared recently by Eric Roman.
> > Netflix few test are supposed to be around here :
> > https://github.com/Netflix/NfWebCrypto/tree/master/
>
> The wiki-page just links to the precise links that I mention below,
> although it's good to have the Google input/out docs.
>
> Yet, the problem that the implementers test-suites are all still basically
> custom and many are limited in coverage so substantial work remains.
>
> > Virginie
> >
> > -----Original Message-----
> > From: Harry Halpin [mailto:hhalpin@w3.org]
> > Sent: mardi 16 décembre 2014 11:00
> > To: public-webcrypto@w3.org
> > Subject: Re: [W3C Web Crypto WG] about tests
> >
> > On 12/16/2014 10:15 AM, GALINDO Virginie wrote:
> >> Harry,
> >> During our F2F meeting you mentionned the possibility to get some
> supports for the testing aspects.
> >> Any news and/or concrete achievement on that side ?
> >
> > It's quite important from a developer-perspective to make sure they can
> know they have interop, so it is important.
> >
> > I've been looking at the test-suites. Unforunately, it's a large amount
> of work due to the uneven and non-portable nature of the tests:
> >
> > Mozilla[1]: Uses a custom framework. Few tests. No net cost reduction by
> porting.
> >
> > Google[2]: No tests per se, only test data.
> >
> > Google[3]: C++ code. Again not worth porting.
> >
> > Apple[4]: Custom JS framework. Very few tests. Hard to port.
> >
> > So in other words, existing tests seem to require starting from
> > scratch
> > (difficult) or help from the browser vendors in porting tests and
> covering.
> >
> > Also, are all those links rights? Is there a Netflix implementation or
> any others with tests we should be aware of?
> >
> > Would implementers be willing to help create the unified test suite?
> >
> >    cheers,
> >      harry
> >
> >
> > [1]:
> > http://dxr.mozilla.org/mozilla-central/search?tree=mozilla-central&q=p
> > ath%3Adom%2Fcrypto%2Ftest&redirect=true
> > [2]:
> > https://code.google.com/p/chromium/codesearch#chromium/src/content/tes
> > t/data/webcrypto/
> > [3]:
> > https://code.google.com/p/chromium/codesearch#chromium/src/content/chi
> > ld/webcrypto/test/
> > [4]: http://trac.webkit.org/browser/trunk/LayoutTests/crypto
> >
> > W3C is working internally on trying
> >> Virginie
> >> ________________________________
> >> This message and any attachments are intended solely for the addressees
> and may contain confidential information. Any unauthorized use or
> disclosure, either whole or partial, is prohibited.
> >> E-mails are susceptible to alteration. Our company shall not be liable
> for the message if altered, changed or falsified. If you are not the
> intended recipient of this message, please delete it and notify the sender.
> >> Although all reasonable efforts have been made to keep this
> transmission free from viruses, the sender will not be liable for damages
> caused by a transmitted virus.
> >>
> >
> > ________________________________
> >  This message and any attachments are intended solely for the addressees
> and may contain confidential information. Any unauthorized use or
> disclosure, either whole or partial, is prohibited.
> > E-mails are susceptible to alteration. Our company shall not be liable
> for the message if altered, changed or falsified. If you are not the
> intended recipient of this message, please delete it and notify the sender.
> > Although all reasonable efforts have been made to keep this transmission
> free from viruses, the sender will not be liable for damages caused by a
> transmitted virus.
> >
> ________________________________
>  This message and any attachments are intended solely for the addressees
> and may contain confidential information. Any unauthorized use or
> disclosure, either whole or partial, is prohibited.
> E-mails are susceptible to alteration. Our company shall not be liable for
> the message if altered, changed or falsified. If you are not the intended
> recipient of this message, please delete it and notify the sender.
> Although all reasonable efforts have been made to keep this transmission
> free from viruses, the sender will not be liable for damages caused by a
> transmitted virus.
>
>