W3C home > Mailing lists > Public > www-voice@w3.org > January to March 2013

Re: ISSUE-825: comments from Darmstadt [SCXML-LC-Comments]

From: Stefan Radomski <radomski@tk.informatik.tu-darmstadt.de>
Date: Tue, 22 Jan 2013 16:35:35 +0000
To: "<jbeard4@cs.mcgill.ca>" <jbeard4@cs.mcgill.ca>
CC: Voice Browser Working Group <w3c-voice-wg@w3.org>, "www-voice@w3.org" <www-voice@w3.org>
Message-ID: <6A775895400B8040A331C2963964820C195199A2@exchange01.tk.informatik.tu-darmstadt.de>
Hey Jake,

we will have a look at your test harness, but reading[1] I can't help but wonder whether we will have a SCION- or a SCXML-interpreter once we pass all tests in the harness. Is there still an ongoing effort to reunite both semantics again? In any case, could you identify the SCION semantics related tests more explicitly until this issue is resolved?


[1] https://github.com/jbeard4/SCION/wiki/SCION-vs.-SCXML-Comparison

On Jan 22, 2013, at 4:26 PM, Jacob Beard <jbeard4@cs.mcgill.ca<mailto:jbeard4@cs.mcgill.ca>>


I also think the issue of an SCXML test suite is very important, and I would like to contribute two comments.

First, I think that many existing open source SCXML implementations have written their own test suites. Would it be useful to issue an "RFTC" (Request for Test Cases?) so that these existing implementations can contribute their tests to the W3C for inclusion as part of an official, standard test suite? This would also help to illustrate differences in semantics between existing implementations.

Second, regarding testing infrastructure, an important part of the work I did on SCION [0] was to create a language-independent test harness [1]. This test harness could potentially allow SCXML implementations to be tested in a uniform way, thus making it easier to ensure compliance.

To provide some background: the SCION engine is written in portable ECMAScript, and so it was possible to write bindings to it for various high-level languages and environments, including the JVM, .NET CLR, CPython, as well as various existing JavaScript shell environments such as Rhino and Node.js. There was a need to test each of these language bindings, and so I created an HTTP-based client-server protocol for testing SCXML implementations. This meant that in order to test each language implementation, one simply had to write an HTTP server in the language in which the SCXML engine API is exposed, and implement the server side of the protocol. This made it possible to test each binding uniformly, independent of language or environment, with minimal effort.

This testing harness is relevant to the W3C SCXML standardization effort, because it would allow the W3C to publish a standard test suite, which could then be easily run against any language implementation. It would save specification implementers the trouble of determining how to run W3C's tests and of rolling their own custom test harness for that test suite. I think providing or recommending a testing harness would be an improvement over providing test cases only.

I look forward to your comments. Thanks,


[0] https://github.com/jbeard4/SCION
[1] https://github.com/jbeard4/scxml-test-framework

On Tue, Jan 22, 2013 at 9:21 AM, Jim Barnett <Jim.Barnett@genesyslab.com<mailto:Jim.Barnett@genesyslab.com>> wrote:
  On the issue of the conformance test, a comprehensive set of tests will be provided as part of the Implementation Report Test Plan when we publish the Candidate Recommendation of SCXML.  Strictly speaking, these are not "conformance tests" since their purpose is not to validate a specific implementation but to prove that the specification can be implemented.  However, as a practical matter, they're a pretty good imitation of a conformance test.  We would be  delighted if you would  run them and submit a report.

I'm not sure when the tests will be published.  If we receive comments that lead us to make substantive changes to the specification, we will have to publish another Last Call working draft.  Once all comments have been dealt with and we are confident that no further changes are necessary, we will publish the Candidate Recommendation, along with the tests.

We will respond to your other comments once we have had a chance to study them.  If you will included "ISSUE-825" in the subject line of any subsequent correspondence, it will help us keep track of thread automatically.


-----Original Message-----
From: Voice Browser Working Group Issue Tracker [mailto:sysbot+tracker@w3.org<mailto:sysbot%2Btracker@w3.org>]
Sent: Tuesday, January 22, 2013 9:13 AM
To: w3c-voice-wg@w3.org<mailto:w3c-voice-wg@w3.org>; www-voice@w3.org<mailto:www-voice@w3.org>
Subject: ISSUE-825: comments from Darmstadt [SCXML-LC-Comments]

ISSUE-825: comments from Darmstadt [SCXML-LC-Comments]


Raised by: James Barnett
On product: SCXML-LC-Comments

from Stefan Radomski [radomski@tk.informatik.tu-darmstadt.de<mailto:radomski@tk.informatik.tu-darmstadt.de>]

Hey there,


So if you are using or have used SCXML, please take the time to send us your comments.  The more comments we get, the more confident we can be that we are producing a sound and useful standard.

we implemented the SCXML draft (or large parts of it) in C++[1]. We do feature an ecmascript (v8) and an experimental prolog (SWI) data model. The overall state of the implementation is still rather crude and very much undocumented but it builds fine on MacOSX, Linux and every now and then on Windows.

I guess what we missed the most is a set of conformance tests as some ..scxml files with sequences of configurations the interpreter is supposed to go through and internal events to raise. Especially the logic around "filterPreempted" is still somewhat fuzzy to us.

Apart from that, there are some areas in the draft where we could imagine some improvements. Just off the top of my head:
- When you use the "Null" datamodel, you cannot use the <param> element to pass even literal strings to e.g. an invoker as both "expr" and "location" are subject to evaluation by the datamodel. Maybe the obvious "value" attribute with a literal string could be included.
- Sending events via the basichttp ioprocessor should allow for the other party to send events in the response.
- Is there a reason scopes in the datamodel were disallowed so explicitly?
- When evaluating foreach with "item" already defined in the datamodel, is it to be reset after foreach ends?
- When embedding an interpreter, #_parent could be specified to send events to the embedding application.

I am sure we will run into many more problems once interoperability with other interpreters becomes an issue.

Best regards

[1] https://github.com/tklab-tud/uscxml

FB20 Telecooperation | Darmstadt University of Technology Hochschulstr. 10 | D-64289 Darmstadt Germany | Room S2|02 / A108 Tel +49 (6151) 16-6670<tel:%2B49%20%286151%29%2016-6670> | Fax +49 (6151) 16-3052<tel:%2B49%20%286151%29%2016-3052>
Received on Tuesday, 22 January 2013 16:36:09 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 21:07:43 UTC