Re: [W3C Web Crypto WG] about tests

On 12/16/2014 06:24 PM, Richard Barnes wrote:
> It's pretty far back in my queue.  I've looked at it briefly, but every
> time I try to get started, I run into the wall of useless documentation on
> testthewebforward.com.
> 
> It might help to have a consolidated event to get some focused attention,
> even if it's only for the testing guys to explain how they like their tests
> done.

If we can find a sponsor of around 20K or so, we can fly everyone out to
SV and do a whole test official W3C the web forward event (and hopefully
fix the docs so they are better!). Or self-organize on a smaller budget:

http://testthewebforward.org/events/#host

Otherwise, we can try to fly  a staff member out to really sit down with
you and other CA-based implementers at the earliest opportunity. That's
cheaper of course, as its less people flying around.

> 
> --Richard
> 
> On Tue, Dec 16, 2014 at 8:47 AM, GALINDO Virginie <
> Virginie.Galindo@gemalto.com> wrote:
>>
>> Dear implementers,
>> Any bandwidth on your side to start a consistent test suite, allowing to
>> demonstrate in the end the interoperability of your implementations in
>> march 2015 ?
>>
>> Harry,
>> How can W3C help ?
>> Mandating one person to work directly on test definition, one person to
>> help implementers to convert their tests, money and office to organizing a
>> test day ?
>>
>> All,
>> Do we need a dedicated call in January for that ?
>>
>> Virginie
>>
>>
>>
>>
>> -----Original Message-----
>> From: Harry Halpin [mailto:hhalpin@w3.org]
>> Sent: mardi 16 décembre 2014 12:03
>> To: GALINDO Virginie; public-webcrypto@w3.org
>> Subject: Re: [W3C Web Crypto WG] about tests
>>
>>
>>
>> On 12/16/2014 11:46 AM, GALINDO Virginie wrote:
>>> Harry,
>>> have tracked the references to implementer's test repository here :
>> https://www.w3.org/2012/webcrypto/wiki/Main_Page#WG_Testing_activity,
>> based on additional information shared recently by Eric Roman.
>>> Netflix few test are supposed to be around here :
>>> https://github.com/Netflix/NfWebCrypto/tree/master/
>>
>> The wiki-page just links to the precise links that I mention below,
>> although it's good to have the Google input/out docs.
>>
>> Yet, the problem that the implementers test-suites are all still basically
>> custom and many are limited in coverage so substantial work remains.
>>
>>> Virginie
>>>
>>> -----Original Message-----
>>> From: Harry Halpin [mailto:hhalpin@w3.org]
>>> Sent: mardi 16 décembre 2014 11:00
>>> To: public-webcrypto@w3.org
>>> Subject: Re: [W3C Web Crypto WG] about tests
>>>
>>> On 12/16/2014 10:15 AM, GALINDO Virginie wrote:
>>>> Harry,
>>>> During our F2F meeting you mentionned the possibility to get some
>> supports for the testing aspects.
>>>> Any news and/or concrete achievement on that side ?
>>>
>>> It's quite important from a developer-perspective to make sure they can
>> know they have interop, so it is important.
>>>
>>> I've been looking at the test-suites. Unforunately, it's a large amount
>> of work due to the uneven and non-portable nature of the tests:
>>>
>>> Mozilla[1]: Uses a custom framework. Few tests. No net cost reduction by
>> porting.
>>>
>>> Google[2]: No tests per se, only test data.
>>>
>>> Google[3]: C++ code. Again not worth porting.
>>>
>>> Apple[4]: Custom JS framework. Very few tests. Hard to port.
>>>
>>> So in other words, existing tests seem to require starting from
>>> scratch
>>> (difficult) or help from the browser vendors in porting tests and
>> covering.
>>>
>>> Also, are all those links rights? Is there a Netflix implementation or
>> any others with tests we should be aware of?
>>>
>>> Would implementers be willing to help create the unified test suite?
>>>
>>>    cheers,
>>>      harry
>>>
>>>
>>> [1]:
>>> http://dxr.mozilla.org/mozilla-central/search?tree=mozilla-central&q=p
>>> ath%3Adom%2Fcrypto%2Ftest&redirect=true
>>> [2]:
>>> https://code.google.com/p/chromium/codesearch#chromium/src/content/tes
>>> t/data/webcrypto/
>>> [3]:
>>> https://code.google.com/p/chromium/codesearch#chromium/src/content/chi
>>> ld/webcrypto/test/
>>> [4]: http://trac.webkit.org/browser/trunk/LayoutTests/crypto
>>>
>>> W3C is working internally on trying
>>>> Virginie
>>>> ________________________________
>>>> This message and any attachments are intended solely for the addressees
>> and may contain confidential information. Any unauthorized use or
>> disclosure, either whole or partial, is prohibited.
>>>> E-mails are susceptible to alteration. Our company shall not be liable
>> for the message if altered, changed or falsified. If you are not the
>> intended recipient of this message, please delete it and notify the sender.
>>>> Although all reasonable efforts have been made to keep this
>> transmission free from viruses, the sender will not be liable for damages
>> caused by a transmitted virus.
>>>>
>>>
>>> ________________________________
>>>  This message and any attachments are intended solely for the addressees
>> and may contain confidential information. Any unauthorized use or
>> disclosure, either whole or partial, is prohibited.
>>> E-mails are susceptible to alteration. Our company shall not be liable
>> for the message if altered, changed or falsified. If you are not the
>> intended recipient of this message, please delete it and notify the sender.
>>> Although all reasonable efforts have been made to keep this transmission
>> free from viruses, the sender will not be liable for damages caused by a
>> transmitted virus.
>>>
>> ________________________________
>>  This message and any attachments are intended solely for the addressees
>> and may contain confidential information. Any unauthorized use or
>> disclosure, either whole or partial, is prohibited.
>> E-mails are susceptible to alteration. Our company shall not be liable for
>> the message if altered, changed or falsified. If you are not the intended
>> recipient of this message, please delete it and notify the sender.
>> Although all reasonable efforts have been made to keep this transmission
>> free from viruses, the sender will not be liable for damages caused by a
>> transmitted virus.
>>
>>
>

Received on Wednesday, 17 December 2014 15:17:27 UTC