[PlugFest/Test] minutes - 14 November 2018

available at:
  https://www.w3.org/2018/11/14-wot-pf-minutes.html

also as text below.

Thanks,

Kazuyuki

---

   [1]W3C

      [1] http://www.w3.org/

                               - DRAFT -

                              WoT-PF/Test

14 Nov 2018

Attendees

   Present
          Kaz_Ashimura, Michael_McCool, Dave_Raggett, Ege_Korkan,
          Federico_Sismondi, Kunihiko_Toumura, Matthias_Kovatsch,
          Takeshi_Yamada, Toru_Kawaguchi, Michael_Lagally,
          Michael_Koster

   Regrets

   Chair
          McCool

   Scribe
          kaz

Contents

     * [2]Topics
         1. [3]Agenda
         2. [4]Update on TD Test
         3. [5]PlugFest reports
         4. [6]Panasonic report
         5. [7]Hitachi report
         6. [8]Assertion review call
         7. [9]Next steps
     * [10]Summary of Action Items
     * [11]Summary of Resolutions
     __________________________________________________________

   <scribe> scribenick: kaz

Agenda

   <McCool>
   [12]https://www.w3.org/WoT/IG/wiki/PlugFest_WebConf#Agenda_14.1
   1.2018

     [12] https://www.w3.org/WoT/IG/wiki/PlugFest_WebConf#Agenda_14.11.2018

   McCool: Agenda above
   ... (goes through the agenda)
   ... any comments? input?

   (none)

Update on TD Test

   <McCool>
   [13]https://github.com/mmccool/wot-thing-description/blob/updat
   ed-test-results/testing/results/template.csv

     [13] https://github.com/mmccool/wot-thing-description/blob/updated-test-results/testing/results/template.csv

   McCool: scanned the TD spec HTML
   ... and extracted assertions
   ... also unique ID
   ... for the actual report
   ... a set of implementations
   ... Pass/Fail/Not Implemented
   ... assumption of test suite

   [14]Draft Implementation Report (rendered version)

     [14] https://cdn.staticaly.com/gh/mmccool/wot-thing-description/updated-test-results/testing/plan.html

   McCool: appendices
   ... about test specification
   ... first thing is
   ... what is the implementation?
   ... I have my system myself
   ... single implementation
   ... but how to count?
   ... and created a template here (at section 6. System)
   ... simple HTML template
   ... every company provides testimonial
   ... and description on your implementation
   ... maybe my system includes 3 separate implementations
   ... will create a concrete template next week
   ... and test spec

   [15]Testspec document (rendered version)

     [15] https://cdn.staticaly.com/gh/mmccool/wot-thing-description/updated-test-results/testing/testspec.html

   McCool: CSV stuff
   ... can be easily merged
   ... implementation described
   ... updated with header information, e.g., ID, Pass, Fail, ...
   ... we can add more information
   ... test1.csv
   ... test2.csv
   ... as 2 example inputs
   ... to be merged
   ... we need this "Pass" column to be more than 1

   Ege: in this case
   ... TD generator
   ... not TD itself
   ... test implementation is done through TDs

   McCool: any behavioral assertions
   ... coordinate with TDs

   Kaz: we can provide typical example TDs
   ... as part of test suite

   McCool: providing TD is not enough
   ... TD itself should be part of implementations
   ... we can/should update leading text at "6. Systems" a bit
   ... "This section contains testimonials..."
   ... TD is provided by the server side
   ... constrained behavior of clients

   Ege: what kind of?

   McCool: to be decided
   ... we should go through the implementations
   ... and see corresponding assertions
   ... possibly have a separate table
   ... test cases against them
   ... will work on the template for the report at "6. Systems"
   ... and CSV files to include header
   ... and create appropriate data

   Ege: these assertions are of different types
   ... maybe better to have different sub sections?

   McCool: maybe correct
   ... originally tried to group them
   ... but kind of awkward
   ... would like to see categorization later

   Lagally: possible to have spec chapter numbers?

   McCool: came from not only one section
   ... can list related section numbers, though

   Lagally: interesting

   McCool: will fix the hyperlinks and make them correct
   ... HTML bashing
   ... capturing section number here
   ... also we don't really have the "sub constraint" structure

   Dave: we should have client-side tests

   McCool: we should draft what kind of exercise we need

   Matthias: assertions still missing
   ... about what the clients should do
   ... we should define the meaning
   ... how the client must behave

   McCool: agree we need client constraint
   ... but what should be the process?

   Kaz: when to start concrete review?
   ... still need help to generate the initial assertion list?

   McCool: can generate some additional assertions using a
   separate color, e.g., based on the plugfest results
   ... and see corresponding features within the TD spec

   Matthias: ok
   ... but you don't have to do that by yourself
   ... e.g., Editors should also do it

   McCool: right
   ... we can go through the plugfest reports and pick up possible
   assertions
   ... e.g., about client behavior
   ... will generate initial PR within 24H
   ... and talk with Sebastian

PlugFest reports

   McCool: some questions about the format

   Matthias: quick reviewed
   ... panasonic and hitachi started to use table

   Lagally: can do another round to update the report with new
   template if needed

   McCool: note each company may have multiple implementations
   ... let's give a unique identifier to each
   ... for interoperability test later

   Lagally: we had a lot of simulators
   ... not sure how we should handle them

   McCool: would count a simulator one implementation

   Lagally: one implementation for one simulator?

   Matthias: e.g., Oracle simulator is one implementation

   Lagally: code-base is a clear definition

   McCool: yes
   ... separate code-base

   Kaz: make sense

   McCool: panasonic, hitachi, want to walk through the reports?

Panasonic report

   [16]https://github.com/w3c/wot/blob/master/plugfest/2018-lyon/r
   esult-panasonic.md

     [16] https://github.com/w3c/wot/blob/master/plugfest/2018-lyon/result-panasonic.md

   Yamada: 3 points here
   ... summary picture
   ... tables of confirmed implementations
   ... browser-based, nodeRED-based
   ... and use cases
   ... semantic integration scenarios
   ... first
   ... the diagram
   ... nodeRED connected with devices via proxies, wot servers

   <mkovatsc> Need to clarify "NG" entry -- either "NO" or "NA"

   Yamada: some field has "NG"
   ... with browser-based implementation
   ... and nodeRED-based one
   ... issue with OCF.json which had multiple entries

   Kaz: do you expect only one TD entry within a TD file?

   Yamada: right
   ... because TD playground expected so

   McCool: maybe we need to have an exec summary?
   ... e.g., at the top of the document?

   Lagally: have a summary section at the top of my report

   Kaz: also Matthias pointed out we should clarify which NG means
   ... NO or NA

   Toru: maybe we can update the table based on the legend

Hitachi report

   [17]https://github.com/w3c/wot/blob/master/plugfest/2018-lyon/r
   esult-hitachi.md

     [17] https://github.com/w3c/wot/blob/master/plugfest/2018-lyon/result-hitachi.md

   Toumura: followed the style of Panasonic
   ... TDs linked from the table
   ... almost all the devices got connected
   ... no big problems with connection
   ... 4. Use cases
   ... use case scenario
   ... remote monitoring and multimodal alerting
   ... realtime control
   ... Rube Goldberg
   ... chain reaction
   ... intrusion detector

   McCool: CORS mentioned in the report
   ... useful metadata?

   Toumura: our implementation didn't handle CORS

   Toru: one of the Panasonic implementations was browser-based
   ... some of the browsers could handle it
   ... but some couldn't

   McCool: crate an issue?
   ... part of the security issues

   <McCool> [18]https://github.com/w3c/wot-security/issues/121

     [18] https://github.com/w3c/wot-security/issues/121

Assertion review call

   McCool: a couple of hours during the 2nd week of Dec
   ... probably two meeting?
   ... two hours at least per one meeting

   <scribe> ACTION: kaz to create a doodle for the assertion
   review call

Next steps

   McCool: other plugfest reports to be provided based on the
   updated template (with a table)
   ... will updated the draft implementation report with
   assertions

   Lagally: regarding the update for plugfest report
   ... based on the table from panasonic/hitachi reports?

   Kaz: we can include an empty table based on Panasonic table
   ... also exec summary at the top of the template

   McCool: we can ask Panasonic to copy the table to the template?

   Lagally: can quickly skim Oracle report?

   [19]https://github.com/w3c/wot/blob/master/plugfest/2018-lyon/r
   esults-oracle.md

     [19] https://github.com/w3c/wot/blob/master/plugfest/2018-lyon/results-oracle.md

   McCool: someone knows the template structure should update the
   template.md

   Toru: Panasonic can do that

   McCool: great

   Lagally: thanks!

   [adjourned]

Summary of Action Items

   [NEW] ACTION: kaz to create a doodle for the assertion review
   call

Summary of Resolutions

   [End of minutes]
     __________________________________________________________


    Minutes manually created (not a transcript), formatted by
    David Booth's [20]scribe.perl version 1.154 ([21]CVS log)
    $Date: 2018/11/15 06:36:11 $

     [20] http://dev.w3.org/cvsweb/~checkout~/2002/scribe/scribedoc.htm
     [21] http://dev.w3.org/cvsweb/2002/scribe/

Received on Thursday, 15 November 2018 06:38:03 UTC