Re: [W3C Web Crypto WG] about tests

On 12/16/2014 06:27 PM, Richard Barnes wrote:
> On Tue, Dec 16, 2014 at 4:59 AM, Harry Halpin <hhalpin@w3.org> wrote:
>>
>> On 12/16/2014 10:15 AM, GALINDO Virginie wrote:
>>> Harry,
>>> During our F2F meeting you mentionned the possibility to get some
>> supports for the testing aspects.
>>> Any news and/or concrete achievement on that side ?
>>
>> It's quite important from a developer-perspective to make sure they can
>> know they have interop, so it is important.
>>
>> I've been looking at the test-suites. Unforunately, it's a large amount
>> of work due to the uneven and non-portable nature of the tests:
>>
>> Mozilla[1]: Uses a custom framework. Few tests. No net cost reduction by
>> porting.
>>
> 
> I'm a little perplexed about how having working tests doesn't save time.
> At the very least, we've looked up test vectors for a whole bunch of
> cases.  And the tests themselves are pretty self-contained.  ISTM that
> between these and the Netflix tests (below), one could make a pretty good
> first pass at a test suite.

Agreed the Mozilla framework and the Netflix ones are the closest to
porting. But to put this into perspecive, if we want ShadowDOM quality
tests, there's still lots more to do. To compare to Shadow DOM,
WebCrypto is more complex: WebIDL fragments (316 vs 38 LOC), more
normative statements (142 vs 92) and more algorithmic steps (1,462 vs
134). However, it's more formulaic so we should be in better shape and
can simplify/automate.




> 
> 
> 
>> Google[2]: No tests per se, only test data.
>>
>> Google[3]: C++ code. Again not worth porting.
>>
>> Apple[4]: Custom JS framework. Very few tests. Hard to port.
>>
>> So in other words, existing tests seem to require starting from scratch
>> (difficult) or help from the browser vendors in porting tests and covering.
>>
>> Also, are all those links rights? Is there a Netflix implementation or
>> any others with tests we should be aware of?
>>
> 
> https://netflix.github.io/NfWebCrypto/web/test_promise.html
> 
> 
> 
>> Would implementers be willing to help create the unified test suite?
>>
>>    cheers,
>>      harry
>>
>>
>> [1]:
>>
>> http://dxr.mozilla.org/mozilla-central/search?tree=mozilla-central&q=path%3Adom%2Fcrypto%2Ftest&redirect=true
>> [2]:
>>
>> https://code.google.com/p/chromium/codesearch#chromium/src/content/test/data/webcrypto/
>> [3]:
>>
>> https://code.google.com/p/chromium/codesearch#chromium/src/content/child/webcrypto/test/
>> [4]: http://trac.webkit.org/browser/trunk/LayoutTests/crypto
>>
>> W3C is working internally on trying
>>> Virginie
>>> ________________________________
>>> This message and any attachments are intended solely for the addressees
>> and may contain confidential information. Any unauthorized use or
>> disclosure, either whole or partial, is prohibited.
>>> E-mails are susceptible to alteration. Our company shall not be liable
>> for the message if altered, changed or falsified. If you are not the
>> intended recipient of this message, please delete it and notify the sender.
>>> Although all reasonable efforts have been made to keep this transmission
>> free from viruses, the sender will not be liable for damages caused by a
>> transmitted virus.
>>>
>>
>>
> 

Received on Wednesday, 17 December 2014 15:14:07 UTC