Re: An IDL for SCXML interpreters

Hi Stefan,

Please excuse the delay in my reply.

> On Jul 3, 2015, at 10:01 AM, Stefan Radomski <radomski@tk.informatik.tu-darmstadt.de> wrote:
> 
> Hey there,
> 
> [explicitly bcc'ing David, Jacob and Zjnue for I know that they maintain SCXML implementations] - this post is pertaining to action point 1 from my previous mail:
> 
> 1. An Interface Description Language (IDL) for SCXML interpreters.
> 1.1 For simple life-cycle and interpretation of state-charts.
> 1.2 A set of hooks for (on-line) visualisation.
> 1.3 A set of hooks for (on-line) modelling / debugging.
> 
> For now, I'd like to focus on point 1.1, but we ought to keep the others in mind and maybe eventually extend our ambitions. At least 1.2 is very useful as it would allow or a common visualisation. I hereby propose a first sketch for an Interface Description Language (IDL) for SCXML (see below). It will only allow to instantiate interpreters, run them and deliver events. A typical session would look like this:
> 
> import namespace scxml;
> try {
> Interpreter session = Implementation.fromURI("http://www <http://www/>.example.com/foo.scxml");
> for(;;) {
> int state = session.step();
> if (state == FINISHED)
> break;
> if (state == AFTER_MACROSTEP && session.configuration.in('foo'))
> session.receive('bar.baz');
> }
> } catch (Exception e) {
> print e;
> }
> 

Here are my comments on this.

In SCION, there’s a notion of compiling the SCXML into a “model” object, which is a native object used to create the session (scxml.urlToModel <https://github.com/jbeard4/SCION#scxmlurltomodelurlfunctionerr-model>). This gives the client a way to create multiple sessions without reparsing/recompiling the SCXML into its object representation, which is a relatively expensive operation. 

Additionally, the model can be can be parsed from a DOM object (scxml.documentToModel <https://github.com/jbeard4/SCION#scxmldocumenttomodelscxmldocumentfunctionerr-model>), which provides opportunities for model transformation using XML DOM API. Finally, in SCION there are convenience methods for parsing SCXML from filesystem (scxml.pathToModel <https://github.com/jbeard4/SCION#scxmlpathtomodelpathfunctionerr-model>), and from a string representation (scxml.documentStringToModel <https://github.com/jbeard4/SCION#scxmldocumentstringtomodelscxmldocstringfunctionerr-model>).

After that, can you instantiate the session using the model object: scxml.scion.Statechart(model, options)

It’s useful to have a mechanism to specify the SCXML _sessionid. This is passed in on the options argument.

session.step() seems equivalent to session.start() in SCION. This causes the state machine to enter its initial state.

Currently in SCION, the return value of session.start() is a basic configuration (set of basic states), encoded as a JavaScript array containing state id strings. In SCION, the basic configuration is kept as an internal data structure, so this cheap to return. 

In terms of querying the session for current state, you can use session.getConfiguration() to get the basic configuration, and use regular JavaScript semantics to query the array (e.g. "configuration.indexOf(’stateId') > -1”). You can a can also use session.isIn(’stateId') to query the session for basic and composite states.

In SCION, session.gen() is equivalent to session.receive(). I chose the name “gen" from Harel’s paper The Rhapsody Semantics of Statecharts <http://research.microsoft.com/apps/pubs/default.aspx?id=148785>. In SCION, the return value is the same as session.start(): a basic configuration.

Finally, you can use session.isFinal() to check if the session is in a final state.

Altogether, here is an example of the SCION API:

    scxml.urlToModel(url,function(err, model){

        if(err) throw err;

        //you can inspect the generated code if you like using JavaScript's Function.prototype.toString
        console.log(model.toString());       

        //instantiate the interpreter
        var statechart1 = new scxml.scion.Statechart(model);

        //you can instantiate a second interpreter using the same model
        var statechart2 = new scxml.scion.Statechart(model);

        //start the interpreter
        var initialConfiguration = statechart1.start();

        //send events
        statechart1.gen({name : 'foo', data : 'bar'});
    });


> A couple of notes and possible variations:
> 
> == 1. Threads
> I want to avoid any assumptions about the availability of threads in the embedding platform. As such, there is e.g. no non-blocking interpret(). It would be the responsibility of the target platform to emulate something like this by embedding the block above in a thread.
> 

I agree with this. A blocking API for interpret/receive/gen works well with the notion of “Fast Computation” for big-step modeling languages described in section 2.3 of Big-Step Semantics <https://cs.uwaterloo.ca/~sesmaeil/publications/2009/CS-2009-05.pdf> by Nancy Day, et al. gen() should be a fast, synchronous, blocking operation.

> == 2. Exceptions
> Similarly I want to avoid any assumption about the availability of exceptions, therefore step() might return FAULT and set the lastError attribute:
> 
> int state;
> for(;;) {
> state - session.step();
> switch(state) {
> case FAULT:
> print session.lastError;
> break;
> case FINISHED:
> return;
> }
> }
> 

I’ve been considering this. The SCXML spec says that scripting errors should be put on the internal queue as error.execution events. However, from a developer perspective, I often find this behavior surprising. The developer needs to remember to add a top-level <transition event=“error.*”> to catch execution errors exposed as events. So by default, in SCXML, JavaScript errors are suppressed. I therefore think it could be useful if the interpreter API could provide better visibility so that execution errors are by default not suppressed. 


> == 3. Data
> 0. If nothing in a Data object is set, its value is undefined
> 1. If an atom is given, the respective field is to be interpreted as the evaluation of that atom, e.g. "3" is a string, 3 is an integer.
> 2. If a node is given and 1. is not the case, the field is to be represented as a DOM Node
> 3. If a key in a compound is set and 1. and 2. are not the case, the respective field is a map
> 4. If an index is set and none of the above applies, the respective field is an array. Setting a an item at an index N will cause the array to behave as if it has N fields for the largest N. With all unset fields undefined.
> 

In SCION, the datamodel is kept fully private, and not exposed externally, except via the session.getSnapshot() method, which returns a copy of the full datamodel. This is a relatively cheap operation.

> == 4. Event
> Having most fields visible will allow for interpreters where all i/o processors are external by simply delivering respective Event entities via receive. The distinction between internal and external events ought to happen in receive(Event) itself depending on the event's type field. I am not sure if we need the PLATFORM class though.

I agree that receive() should be able to accommodate both internal and external events. It should be possible to inspect the _event.origin field and compare it to _ioprocessors to determine whether an event originating internal or external to the session.

> 
> == 5. Hibernating
> We might want to include a way to serialise a complete interpreter and instantiate it again from its serialised representation. I do know that Jacob was working on a respective feature. We dropped our ambitions for now as most of the datamodels we employed via 3rd party libraries would not support something like that.
> 

Right now, in SCION, datamodel elements that are “transient” (not serializable) are initialized in the top-level <script> tag, which is executed every time a session is instantiated. Rather than put them in the datamodel, they are declared in the <script> tag using JavaScript “var”. I think this is a bit ugly, and could be improved. To improve this, two things would be needed:

1. An explicit notion of transient attributes on the datamodel. This is similar to the “transient" attribute modifier in Java. These datamodel elements would not be serialized on hibernate.
2. Explicitly recognizing that top-level script tag is a code block that will executed on resume. This allows transient datamodel elements to be re-initialized. 

State machine hibernation is useful for developing scalable cloud applications, as it allows your SCXML session data to be persisted to secondary storage, rather than kept in memory. This is needed for horizontal scalability.

> == 6. Naming
> AFTER_MACROSTEP might just as well be called STABLE. I choose the former for its more explicit reference to the runtime semantics of SCXML, which might play a role if we were to specify a bunch of hooks for 1.2 and 1.3 (see debugger paper[1], table 1)
> 

I think it would be useful to explicitly capture these interpreter states as a meta-state machine.

It may be interesting to look at the states in the docker container lifecycle, as docker provides a generic container API for linux applications. These concepts could carry over to modeling the SCXML interpreter lifecycle. I created an SCXML to model this here: http://examples.scxml.io/e/docker-manager/ <http://examples.scxml.io/e/docker-manager/>
> == 7. Events *from* the interpreter
> While point 4 allows an embedding platform to send arbitrary events *into* the interpreter, maybe we ought to have something similar for everything <send> from the interpreter?
> 

I think we should consider an API for <send>. I’m currently working on improving the SCION <send> implementation, including support for custom send types, and will follow up on this item shortly.

> 
> Maybe anyone who is interested in pursuing the idea of this basic IDL could speak up and comment about the proposal? What is missing, what is superfluous for a basic IDL?

Thank you for getting this process started. Please let me know your feedback, and I can take a pass at iterating on the proposed IDL.

Regards,

Jacob Beard

> 
> Regards
> Stefan
> 
> [1] http://scxmlworkshop.de/eics2014/submissions/A%20Debugger%20for%20SCXML%20Documents.pdf
> 
> 
> 
> ============
> 
> module scxml {
> 
> exception Exception {
> const unsigned short PARSE_ERR = 1; // Given string could not be parsed as an xml document
> const unsigned short SCHEMA_ERR = 2; // XML DOM not a valid SCXML document
> const unsigned short IMPL_ERR = 4; // Unknown datamodel, ioproc, executable content or invoker used
> 
> unsigned short code;
> String cause;
> };
> 
> 
> // central place to get interpreter instances
> interface Implementation {
> Interpreter fromURI(in String uri) raises(Exception);
> Interpreter fromXML(in String xmlString) raises(Exception);
> Interpreter fromDOM(in Node scxmlRoot) raises(Exception);
> }
> 
> interface Event {
> const unsigned short PLATFORM_EVENT  = 1;
> const unsigned short INTERNAL_EVENT  = 2;
> const unsigned short EXTERNAL_EVENT  = 3;
> 
> String name;
> unsigned short type;
> String origin;
> String originType;
> String sendid;
> Data data;
> };
> 
> interface Data {
> set(in String value); // atom
> String get();
> 
> setNode(in Node node); // XML node
> Node getNode();
> 
> setKey(in String key, in Data value); // compound
> Data getKey(in String key);
> 
> setItemAt(in String index, in Data value); // array
> Data getItemAt(in String index);
> };
> 
> interface Configuration {
> boolean in(in String state);
> String item(in unsigned long index);
> readonly attribute unsigned long length;
> };
> 
> interface Interpreter {
> const unsigned int FAULT = 0; // something bad happened, look into lastError
> const unsigned int AFTER_MICROSTEP = 1; // step finished with a single microstep
> const unsigned int AFTER_MACROSTEP = 2; // step finished with a single macrostep and interpreter is stable
> const unsigned int SCXML_IDLE = 3; // interpreter is stable with no pending events on queues
> const unsigned int SCXML_FINISHED = 4; // interpreter reached top-level final state
> 
> const unsigned int step() raises(Exception) raises(Exception); // blocking per default
> const unsigned int step(in int timeOutMs) raises(Exception); // block at most for given duration, 0 for non-blocking
> 
> receive(in Event event);
> receive(in String event); // convenience
> 
> readonly attribute Exception lastError;
> readonly attribute Configuration configuration;
> readonly attribute Configuration basicConfiguration;
> }
> }
> 
> ============