- From: Kazuyuki Ashimura <ashimura@w3.org>
- Date: Fri, 4 May 2018 14:58:13 +0900
- To: Public Web of Things IG <public-wot-ig@w3.org>, public-wot-wg@w3.org
available at: https://www.w3.org/2018/05/02-wot-test-minutes.html also as text below. Thanks a lot for taking these minutes, Daniel! Kazuyuki --- [1]W3C [1] http://www.w3.org/ - DRAFT - WoT Testing 02 May 2018 [2]Agenda [2] https://www.w3.org/WoT/IG/wiki/Test_WebConf#Agenda Attendees Present Kaz_Ashimura, Michael_McCool, Daniel_Peintner, Tomoaki_Mizushima, Toru_Kawaguchi, Matthias_Kovatsch, Michael_Lagally, Michael_Koster Regrets Chair McCool Scribe dape Contents * [3]Topics 1. [4]Agenda 2. [5]Categories of "testing" 3. [6]Reference implementations? 4. [7]Levels of "testing" 5. [8]Actions 6. [9]Testing manager? 7. [10]Time slot for the testing call? * [11]Summary of Action Items * [12]Summary of Resolutions __________________________________________________________ <kaz> scribenick: dape Agenda <kaz> Agenda: [13]https://www.w3.org/WoT/IG/wiki/Test_WebConf#Agenda [13] https://www.w3.org/WoT/IG/wiki/Test_WebConf#Agenda McCool: Any additions to the agenda? ... we have plan ... 1. step, markup spec with normative assertions <kaz> [14]plan [14] https://github.com/w3c/wot/blob/master/testing/plan.md McCool: RFC 2119 keywords ... anyone started with normative assertions ? Kaz: think Sebastian and Zoltan started with it... McCool: Section "Conformance" needs to be added ... particular words MAY; MUST, ... etc need to be used <kaz> [15]scripting api draft [15] https://rawgit.com/zolkis/wot-scripting-api/229a1bcbc0e08b3af259b18e3d1c3f2ef065877c/index.html Kaz: Zoltan added conformance section as Section 9 <kaz> [16]section 9 [16] https://rawgit.com/zolkis/wot-scripting-api/229a1bcbc0e08b3af259b18e3d1c3f2ef065877c/index.html#conformance McCool: will be protocol binding separate document till to the end? Koster: plan to provide normative statements for TD McCool: for security it is the same ... assume normative statements are a big chunk Categories of "testing" McCool: 1. specification validation (required by W3C) ... spec implementable and consistent ... 2. Implementation testing ... check implementation satisfy requirements ... a) scripting API ... b) TD validation ... general useful tool ... c) network interface described by TD ... would like to have an automated thing that tests this ... wonder which is actually necessary Koster: do they need to be reference implementation kaz: usually we don't have "reference" implementations ... just need 2 implementations for each feature McCool: 3. category is interop testing ... difficult problem Reference implementations? Koster: we can have reference implementation ... reference implementation helps to get started McCool: Could be online tool Koster: PlugFest could be venue Kaz: usual W3C specifications are related to Web browsers ... so the W3C Testing Framework mainly handles Web browser testing ... on the other hand, we need server side also for WoT ... that's why, reference impl for server side is useful Levels of "testing" Daniel: Wonder whether w3c testing only means spec validation ? Kaz: the main purpose of "testing" for Candidate Recommendation transition is spec validation McCool: we can add levels ... and we missed security testing Koster: there is our spec we test against but there is more (e.g., in security) McCool: we can just have recommendation ... provide framework how to test Kaz: maybe we might want to rename "security testing" to "advanced testing" which include security testing McCool: I believe we have statement in charter that requires us to test security ... in our tooling we can point to how to improve security Kaz: ah, that's true <kaz> [17]WG Charter, fyi In order to enhance the security of WoT systems, we will also generate and implement a security testing plan which will include both functional and adversarial testing of the proposed standards and their implementations. We will only recommend an implementation of the proposed standards for use in production once it has passed such testing. ]] [17] https://www.w3.org/2016/12/wot-wg-2016.html McCool: I think we need validation tool ... "functional testing" could be basic authentication ... ... hope we can reuse things that already exists Matthias: w.r.t. CoAP there is some penetration testing ... for CoAP we broke it down by feature ... test it pair-wise ... also test it with security enabled (difficult and maybe also impossible with all combinations) McCool: test common combination ... w.r.t. reference implementation ... we need 2 separate implementations ... for example in the case of scripting we might just have a limited number ... need to focus breaking things down by feature ... after that we can create table with common combinations ... maybe PlugFest could be used to implement test cases Koster: in physical realization: we need system under test and test tool ... can have scripts for exposed thing and consumed thing using node-wot McCool: for PlugFest (or after) I would like to have a skeleton testing framework in place ... implies that before the PlugFest we need test description (before that test assertions) Koster: need to talk about how to implement ... what new tools we need Actions McCool: Next, will create section on wiki for "Actions" ... normative assertions ... test descriptions ... each group should think about this ... we might want to have MD template Testing manager? McCool: Q: Is there any test expert who could help us/me? Matthias: mainly time issue ... talked to Ege Korkan ... he could volunteer ... will ask him Time slot for the testing call? Koster: should also re-consider time slot McCool: Kaz, can you do a doodle? Kaz: will do ... BTW, we might want to think about collaboration between PlugFest and Testing ... PlugFest call should start again ... possibly done within this testing call, but maybe need a separate call <kaz> Koster: right. plugfest has a bit different logistics cycle <kaz> [adjourned] Summary of Action Items Summary of Resolutions [End of minutes] __________________________________________________________ Minutes formatted by David Booth's [18]scribe.perl version 1.152 ([19]CVS log) $Date: 2018/05/04 05:54:46 $ [18] http://dev.w3.org/cvsweb/~checkout~/2002/scribe/scribedoc.htm [19] http://dev.w3.org/cvsweb/2002/scribe/
Received on Friday, 4 May 2018 05:59:24 UTC