[PlugFest/Test] minutes - 28 November 2018

available at:
  https://www.w3.org/2018/11/28-wot-pf-minutes.html

also as text below.

Thanks,

Kazuyuki

---

   [1]W3C

      [1] http://www.w3.org/

                               - DRAFT -

                           WoT-PlugFest/Test

28 Nov 2018

   [2]Agenda

      [2] https://www.w3.org/WoT/IG/wiki/PlugFest_WebConf#Agenda_28.11.2018

Attendees

   Present
          Kaz_Ashimura, Michael_McCool, Ege_Korkan,
          Matthias_Kovatsch, ege, Dave_Raggett, Taki_Kamiya,
          Tomoaki_Mizushima, Toru_Kawaguchi

   Regrets
          Michael_Lagally, Michael_Koster

   Chair
          McCool

   Scribe
          kaz

Contents

     * [3]Topics
         1. [4]Agenda
         2. [5]update on the implementation report
         3. [6]PlugFest reports
         4. [7]TestFest planning
         5. [8]Test Specification
     * [9]Summary of Action Items
     * [10]Summary of Resolutions
     __________________________________________________________

   <scribe> scribenick: kaz

Agenda

   McCool: goes through the agenda on the wiki

   [11]Agenda

     [11] https://www.w3.org/WoT/IG/wiki/PlugFest_WebConf#Agenda_28.11.2018

update on the implementation report

   McCool: updated the draft
   ... pending PR 294

   [12]PR 294

     [12] https://github.com/w3c/wot-thing-description/pull/294

   McCool: will merge it tomorrow after minor corrections
   ... including implementations from Intel, Siemens, Panasonic
   ... fair number of implementations
   ... 7.1 result table
   ... MUSTs and MAYs
   ... collect data on how many implementations implemented each
   feature
   ... marking features without enough implementations
   ... general assertions all the vocabularies are required
   ... id, vocab-base, etc.
   ... parent assertion (general ones)
   ... and textual things on syntactic context
   ... that's the dependency
   ... and then
   ... interoperability test results
   ... at 7.2
   ... Intel's OCF metadata translator, etc. vs other
   implementations
   ... description under /testing/inputs/interop/ subdirectory
   ... e.g., intel.csv

   [13]https://github.com/mmccool/wot-thing-description/tree/updat
   ed-test-results/testing/inputs/interop

     [13] https://github.com/mmccool/wot-thing-description/tree/updated-test-results/testing/inputs/interop

   McCool: for CR/PR transition, we don't need this part
   ... but would be useful for implementers to check
   interoperability

   [14]https://github.com/mmccool/wot-thing-description/tree/updat
   ed-test-results/testing/inputs/implementations

     [14] https://github.com/mmccool/wot-thing-description/tree/updated-test-results/testing/inputs/implementations

   McCool: all the implementers need to generate implementation
   reports like the above
   ... there is another file
   ... can do this table imple.csv
   ... each implementer to submit an HTML file

   [15]https://github.com/mmccool/wot-thing-description/tree/updat
   ed-test-results/testing/inputs/implementations

     [15] https://github.com/mmccool/wot-thing-description/tree/updated-test-results/testing/inputs/implementations

   McCool: and results

   [16]https://github.com/mmccool/wot-thing-description/tree/updat
   ed-test-results/testing/inputs/results

     [16] https://github.com/mmccool/wot-thing-description/tree/updated-test-results/testing/inputs/results

   Kaz: so 2 files?
   ... HTML and result.csv?

   McCool: result.csv can be done during the TestFest

   <McCool>
   [17]https://github.com/mmccool/wot-thing-description/tree/updat
   ed-test-results/testing/inputs/implementations

     [17] https://github.com/mmccool/wot-thing-description/tree/updated-test-results/testing/inputs/implementations

   McCool: can work with the table rendered by GitHub
   ... need description on the description about each
   implementer's implementation
   ... as HTML

   [18]https://github.com/mmccool/wot-thing-description/tree/updat
   ed-test-results/testing/inputs/implementations

     [18] https://github.com/mmccool/wot-thing-description/tree/updated-test-results/testing/inputs/implementations

   Kaz: that description will go to the pinkish area within the
   plan.html

   McCool: right
   ... need to merge the PR
   ... tomorrow
   ... meanwhile please submit your implementation reports to my
   repo

   [19]https://github.com/mmccool/wot-thing-description/tree/updat
   ed-test-results/testing/inputs/implementations

     [19] https://github.com/mmccool/wot-thing-description/tree/updated-test-results/testing/inputs/implementations

   McCool: any questions?

   dsr: wondering about adding a link to the WG page?

   McCool: this is still draft

   Kaz: after McCool's merging this to the wot-thing-description
   repo, we can add some description and links to the
   wot-thing-description repo
   ... all the PlugFest participants are encouraged to add their
   information
   ... to your repo today
   ... or to wot-thing-description repo after tomorrow

   McCool: will merge the PR within 12 hours
   ... or you can send your report to me by email
   ... or via PR as usual

PlugFest reports

   <McCool>
   [20]https://github.com/w3c/wot/tree/master/plugfest/2018-lyon

     [20] https://github.com/w3c/wot/tree/master/plugfest/2018-lyon

   <kaz> [21]Fujitsu's PlugFest report summarizing the results

     [21] https://github.com/w3c/wot/blob/master/plugfest/2018-lyon/result-fujitsu.md

   McCool: (refering to Fujitsu's PlugFest report)
   ... not sure who was connected to Intel
   ... we have these checking points
   ... validation, for example
   ... ideally we can talk about validation suite
   ... what was validated

   Ege: errors? or just what was validated?

   McCool: we can generate result CSV files as a side effect of
   validation

   Ege: some sort of JSON schema?
   ... to identify which assertion

   McCool: generate CSV files
   ... we can generate this as a side effect of playground

   Ege: ok

   McCool: another discussion here
   ... 4 statuses: pass, not-imple, failed, not mention
   ... I count "not-imple"
   ... means TD has no actions, possibly

   Ege: could mean if you can't have 2 things at once
   ... either property or action

   McCool: the way I'm counting the implementations
   ... chunks of codes
   ... if someone has more than one implementation
   ... need some kind of logical merge

   Ege: sometimes not all the TDs can't be supported at once

   McCool: let me try some result merger

   Ege: will generate CSV per TD

   McCool: ok
   ... go back to Fujitsu's plugfest report
   ... 3.1.3 Connect with Remote/Local Proxy
   ... Thing Directory consumes the TD and reproduces it
   ... manipulate the TD
   ... still counts as an implementation experience
   ... here Term definition within criteria.md

   <kaz>
   [22]https://github.com/w3c/wot/blob/master/testing/criteria.md

     [22] https://github.com/w3c/wot/blob/master/testing/criteria.md

   McCool: section on what to be included
   ... and what not to be included

   Ege: what test bench?

   McCool: node-wot could be a component

   <kaz> [23]Fujitsu's PlugFest report summarizing the results

     [23] https://github.com/w3c/wot/blob/master/plugfest/2018-lyon/result-fujitsu.md

   McCool: go back to the plugfest report again
   ... 3.1.2 register with Thing Directory
   ... need to clarify the relationship
   ... between the plugfest report and implementation report

TestFest planning

   McCool: week of Dec 10
   ... 2 hours on Monday (Scripting/Security)
   ... 3 hours on Wednesday (Editor/Main/Testing)

   toru: what time?

   McCool: 9pm on Monday
   ... and 9pm on Wednesday
   ... and then 4pm on Friday
   ... all JST
   ... create a wiki page

   <scribe> ACTION: kaz to create a wiki page for the TestFest

   McCool: 2 weeks from now
   ... feasible?
   ... in the worst case, we can collect TDs
   ... and fill out the result files
   ... validator to create the result files?

   Ege: can work on it

   McCool: we can collect TDs
   ... generate subset of assertion result CSV files
   ... if we could have an automated tool (based on the
   playground) that would be great
   ... if not, we can do that manually
   ... also we can collect interop.csv as well
   ... and we can update the draft implementation report
   ... note that we still need implementation.html description
   from implementers
   ... also still useful to have human-readable
   preparation.md/results.md
   ... btw
   ... still working on VPN server setting
   ... based on an OpenVPN server
   ... anything else?
   ... wondering about how to allocate a 3-hour call

   Kaz: we can simply extend the first hour, e.g., the Editor's
   slot

   McCool: would like kaz to include a template including the
   links for the calls

Test Specification

   McCool: within the plan.html
   ... test spec setting in appendix
   ... smaller team to work on that?

   Kaz: if people are interested they can join. right?

   McCool: me, Ege and Dave are the initial candidates

   taki: interested
   ... if additional assertions to be added

   Kaz: as well :)

   <McCool>
   [24]https://www.w3.org/WoT/IG/wiki/WG_Binding_Templates_WebConf

     [24] https://www.w3.org/WoT/IG/wiki/WG_Binding_Templates_WebConf

   McCool: binding template call on Dec. 4?

   Kaz: have a dedicated discussion with Ralph on testing
   requirements at that time
   ... so how about having the discussion tomorrow on Nov 29 at
   11pm JST (6am US Pacific, 9am US Eastern, 2pm UK, 3pm EU)?

   all: sounds good

   Toru: when to fix the TD spec for the TestFest?

   taki: i18n aspect, etc.
   ... a couple of items to be done
   ... all properties at once, etc.
   ... to be discussed on 30th

   McCool: backward compatibility with Lyon version?
   ... thing template, etc., are optional
   ... don't remember anything breaks backward compatibility
   ... would focus on Lyon TDs
   ... but also would generate assertions based on the latest
   draft

   Kaz: ok with that direction
   ... the results from the TestFest don't have to be perfect
   ... Taki, please note that we'll have a W3C Japanese member
   meeting on Nov. 30 and many of the JP participants including
   myself can't make the TD call on Nov. 30

   [adjourned]

Summary of Action Items

   [NEW] ACTION: kaz to create a wiki page for the TestFest

Summary of Resolutions

   [End of minutes]
     __________________________________________________________


    Minutes manually created (not a transcript), formatted by
    David Booth's [25]scribe.perl version 1.154 ([26]CVS log)
    $Date: 2018/11/29 02:20:40 $

     [25] http://dev.w3.org/cvsweb/~checkout~/2002/scribe/scribedoc.htm
     [26] http://dev.w3.org/cvsweb/2002/scribe/

Received on Thursday, 29 November 2018 02:22:54 UTC