[PlugFest/Testing] minutes - 22 January 2020

available at:
  https://www.w3.org/2020/01/22-wot-pf-minutes.html

also as text below.

Thanks,

Kazuyuki

---
   [1]W3C

      [1] http://www.w3.org/

                               - DRAFT -

                          WoT-PlugFest/Testing

22 Jan 2020

Attendees

   Present
          Kaz_Ashimura, Michael_McCool, Sebastian_Kaebisch,
          Kunihiko_Toumura, Michael_Koster, Takahisa_Suzuki,
          Tomoaki_Mizushima

   Regrets

   Chair
          McCool

   Scribe
          kaz

Contents

     * [2]Topics
         1. [3]Plan and expectation
         2. [4]Next calls
     * [5]Summary of Action Items
     * [6]Summary of Resolutions
     __________________________________________________________

Plan and expectation

   McCool: we should talk about our plan/expectation first
   ... kaz previously mentioned test framework
   ... myself think about interoperability testing
   ... more focused consumer viewpoint
   ... conformance, etc.
   ... generally need to think about features
   ... what are the features and related assertions
   ... for the Helsinki meeting, we should have test framework
   ... should use this time for sorting out it
   ... would like to state the issues
   ... what kind of test would be appropriate, etc.

   Sebastian: what do you mean by "framework"?
   ... set of tools?

   McCool: tools and procedure
   ... various tools, things, etc.
   ... data to be used for multiple implementations
   ... interoperability testing
   ... formal output like CSV
   ... we're half of that
   ... also should think about behavioral testing

   Kaz: right
   ... we should provide resources for non-WoT people to
   understand WoT standards and generate implementations

   McCool: yes
   ... how we communicate with implementation reports, etc., as
   well

   Sebastian: our assertion table should be a good starting point

   McCool: yes
   ... don't think we have to have automated test
   ... but might need some additional APIs for testing
   ... how do we test everything?
   ... should clarify our plan

   Kunihiko: many specs are in incubation phase
   ... we should investigate the gaps
   ... not only test/interoperability but also we can use plugfest
   for feasibility check

   McCool: meaning we need some prototyping?

   Kunihiko: right

   McCool: should split into testing part and prototyping part
   ... if we have a test suite, people can test implementations
   ... and we can focus on gaps
   ... ambiguity, etc.
   ... generally speaking, we should hold plugfest/test calls
   regularly
   ... would have a separate repository
   ... also should clarify we use the same webex as the main call
   ... regarding conformance, we don't have any description about
   conformant implementations
   ... kind of marketing topic as well

   Koster: some kind of online tools people can do something
   ... we always need to look at files
   ... but also have some idea of online things
   ... like eclipse foundation's tools

   McCool: virtual devices with online urls
   ... so that we can play with them
   ... there is testing and kind of like examples and
   documentation
   ... but not well-organized yet
   ... we care about test coverage
   ... someone is doing multi-language system
   ... if we put WoT testing repo in one place
   ... might be cleaner to navigate people

   Koster: developer journey will start
   ... how do I get involved?
   ... download tools to check conformance

   McCool: good point
   ... what to be supported in each step of the procedure
   ... two steps, examples and how to see if it's correct
   ... different stages

   Koster: a lot of hints for developers
   ... engage the hands-on side

   McCool: one is prototyping to check new ideas
   ... and evangelism for new comers
   ... then testing for conformance check
   ... one event handler or multiple handlers?

   Koster: those are questions I have as well

   McCool: would be good if we could have a hackathon
   ... should have experts there
   ... probably should consider both training and prototyping

   Koster: during OCF ones, they work together to validate test
   cases
   ... should be online?

   McCool: for testing the test cases
   ... do we have same conception as the spec?
   ... ambiguity about the meaning
   ... f2f time is good for validation
   ... pragmatically what would be our actions?

   Kaz: should we have multiple tasks including prototyping,
   conformance test and documentation?

   McCool: would create a dedicated GitHub repo for the work

   Kaz: what should be the title for the repo?

   McCool: "wot-testing"
   ... what do you think?

   Sebastian: ok

   McCool: specifically testing, planning for test cases, etc.
   ... landing point for the work

   Koster: test content should be self-contain
   ... a lot of different resources there
   ... before we create a new repo, we should clean them up

   McCool: a lot of old contents
   ... would suggest we create a new repo and move necessary
   contents to there
   ... and then make update

   Sebastian: makes sense
   ... but who to handle what?

   McCool: can volunteer
   ... and work with Ege

   Kaz: what about JP participants?

   McCool: can issue a call-for-participants
   ... also we can have discussion during the marketing call as
   well
   ... should have a resolution for "wot-testing" repo?

   proposal: we'll create the "wot-testing" GitHub repo for
   further testing/plugfest work

   (no objections)

   RESOLUTION: we'll create the "wot-testing" GitHub repo for
   further testing/plugfest work

   <scribe> ACTION: kaz to create "wot-testing" repo

Next calls

   Kaz: we'll have a marketing call tomorrow at 14UTC
   ... and the next test/plugest call will be right after the main
   call next Wednesday

   [adjourned]

Summary of Action Items

   [NEW] ACTION: kaz to create "wot-testing" repo

Summary of Resolutions

    1. [7]we'll create the "wot-testing" GitHub repo for further
       testing/plugfest work

   [End of minutes]
     __________________________________________________________


    Minutes manually created (not a transcript), formatted by
    David Booth's [8]scribe.perl version 1.154 ([9]CVS log)
    $Date: 2020/02/03 02:04:56 $

      [8] http://dev.w3.org/cvsweb/~checkout~/2002/scribe/scribedoc.htm
      [9] http://dev.w3.org/cvsweb/2002/scribe/

Received on Monday, 3 February 2020 02:07:03 UTC