[wot-test] minutes - 23 May 2018

available at:
  https://www.w3.org/2018/05/23-wot-test-minutes.html

also as text below.

Thanks a lot for taking these minutes, Michael Lagally!

Kazuyuki

---

   [1]W3C

      [1] http://www.w3.org/

                               - DRAFT -

                                WoT Test

23 May 2018

   [2]Agenda

      [2] https://www.w3.org/WoT/IG/wiki/Test_WebConf#Agenda_23_May_2018

Attendees

   Present
          Kaz_Ashimura, Dave_Raggett, Ege_Korkan, Michael_McCool,
          Michael_Lagally, Tomoaki_Mizushima, Toru_Kawaguchi

   Regrets

   Chair
          McCool, Ege

   Scribe
          mlagally_

Contents

     * [3]Topics
         1. [4]normative statement extraction
         2. [5]TD validation/TD-based thing validation
         3. [6]F-Interop
         4. [7]AOB
     * [8]Summary of Action Items
     * [9]Summary of Resolutions
     __________________________________________________________

   <ege>
   [10]https://github.com/egekorkan/wot/blob/master/testing/Presen
   tation.pdf

     [10] https://github.com/egekorkan/wot/blob/master/testing/Presentation.pdf

   <kaz> [11]https://github.com/w3c/wot/pull/444

     [11] https://github.com/w3c/wot/pull/444

   <kaz> scribenick: mlagally_

normative statement extraction

   <inserted> [12]PR 138

     [12] https://github.com/w3c/wot-thing-description/pull/138

   McCool: <presents tool to extract normative assertions>
   ... PR is available, not yet merged

   Ege: where will the assertions be shown

   McCool: "npm runassertion" produces a html result file:
   assertions.html

TD validation/TD-based thing validation

   <inserted> [13]Ege's slides on Thing Testing

     [13] https://github.com/egekorkan/wot/blob/master/testing/Presentation.pdf

   Ege: topic could be named TD based thing validation
   implementation
   ... test a thing by just using it's TD
   ... simplified V-model, it is a continuous process
   ... want to use a TD in all parts of the development cycle
   ... we test a thing via network interfaces (input and output)
   ... this is blackbox-testing
   ... main idea is to rely on JSON-schema embedded in TD
   ... JSON schema entries specify the datastructure
   ... illustrates schema validation >
   ... we use existing tools for JSON validation
   ... we do JSON value generation (JSON-fakers)
   ... feel free to play with the online-tools (see preso)
   ... testbench will be ready for next plug fest

   McCool: what does interop testing mean? A device can interact
   with another?

   Ege: a device can send fake values

   McCool: interop: you create a fake thing, similar to the things
   in the Oracle device simulator
   ... this is pretty close to what we already have in the
   simulator

   Ege: can the simulator define functions and change values
   dynamically ?

   Lagally: yes, you can define dynamic behaviour via functions

   scribenick: kaz

   McCool: suggests Ege and Michael Lagally work together
   ... would like to go back to trigger for events

   scribenick: dsr

   Koster: we may not need to be able to trigger events - we may
   just add a test event that we can trigger

   McCool: are we testing framework or a thing?

   Koster: you don't need to test hardware, etc, just the event
   generating mechanism

   McCool: we use descriptions of existing devices- cannot trigger
   events
   ... for framework we can do standardized tests

   Koster: we can add a specific trigger mechanism

   McCool: we can define a pattern to make things more testable

   Ege: domain is to test the scripting API implementation

F-Interop

   <inserted> [14]F-Interop site

     [14] https://www.f-interop.eu/

   <inserted> [15]Dave's slides

     [15] https://www.w3.org/2018/05/08-dsr-wot.pdf

   Dave: EU project, runs until September
   ... so far they are testing at protocol level
   ... W3C brought in higher level (applciation level) testing
   ... interop testing of WoT platforms
   ... e.g. node-WoT and another platform
   ... you have a suite of tests that can check WoT interop
   ... we create a test app for each platform
   ... they expose and consume things via platform API
   ... also have a backchannel using F-interop
   ... framework is very similar to what Ege was presenting
   ... we have declarative tests
   ... can be derived from specs
   ... we need machine-interpretable version of the test cases
   ... test apps need also access to the tests

   Koster: we plan to use F-interop for Wishi as well
   ... this is part VPN, part script replay

   McCool: what is the smallest piece to test a framework

   Dave: a TD with a single property, e.g. boolean
   ... simple TD, run a couple of tests both from the consumed and
   exposed side of a thing
   ... for events you have to register
   ... we are testing on a much higher level than on protocols,
   i.e. API level

   McCool: any concrete example to better understand?
   ... effort?

   Dave: 2 months work between May and September
   ... show sth. at TPAC

   McCool: we can start immediately after plugfest, can we have a
   demonstrator of the strategy at the plugfest

   Dave: will have a document by end of June
   ... need to figure out how to collaborate

   Koster: can we use that for general online plugfest?

   Dave: there are docker-based setups

   McCool: if we can have a node-WoT setup in docker it is easy to
   reproduce
   ... next time dig deeper into interop testing

   Kaz: interop testing is one of the test efforts - first
   priority is spec validation and implementation testing
   ... need to clarify relation between F-interop and plugfest
   ... which part of framework could be used and vice versa?

   Dave: this is discussion in testing TF

   McCool: validation needed to demonstrate interop between two
   different implementations
   ... interorganisational dependencies can be an issue
   ... need to discuss in next call
   ... licensing to be clarified
   ... we need official liaison with F-interop

   Ege: last time it was quite difficult to use
   ... you had to hack to make it work

   McCool: a brief demo would be helpful

   Dave: perhaps ask Frederico

   Ege: can ask him
   ... targeting next week

AOB

   scribenick: kaz

   Lagally: wonder if anybody else than Ege is interested in
   Oracle simulator

   Kaz: want to have a separate call?

   McCool: we can do that during the testing call itself

   [adjourned]

Summary of Action Items

   [NEW] ACTION: Dave/Ege/Fedelico to work on demonstration of
   F-Interop
   [NEW] ACTION: Kaz to create a Doodle poll to decide a new slot
   for the Testing call

Summary of Resolutions

   [End of minutes]
     __________________________________________________________


    Minutes formatted by David Booth's [16]scribe.perl version
    1.152 ([17]CVS log)
    $Date: 2018/06/04 06:10:40 $

     [16] http://dev.w3.org/cvsweb/~checkout~/2002/scribe/scribedoc.htm
     [17] http://dev.w3.org/cvsweb/2002/scribe/

Received on Monday, 4 June 2018 06:16:57 UTC