Re: Big SCXML Interpreter Benchmark

Hello Stefan. I'm afraid that JSSC is specifically meant to run in a browser and it relies on browser features like the DOM parser. However, your test methodology should be simple enough to implement in a browser. You can initialize a SC with a string of XML or the URL of a file (just set up a tiny nginx server with your generated scxml files and suitable CORS headers) and measure time accurately with the Performance API (https://developer.mozilla.org/en-US/docs/Web/API/Performance). Being in a browser should have no impact on performance.

Anyway, the current version of JSSC was never meant for high performance. I designed it to handle runtime mutations of the State Chart and so it has to sacrifice lots of compile-time optimizations (plus it's JavaScript).

David

> Le 5 juil. 2017 à 14:38, Stefan Radomski <radomski@tk.tu-darmstadt.de> a écrit :
> 
> Hey there,
> 
> I did an evaluation of the performance of six different SCXML implementations, namely LXSC, Qt SCXML, SCION, scxmlcc, Commons SCXML and uSCXML [1]. I do display those results on the front-page of our interpreter and would like feedback from the other authors as to whether they feel that I misrepresent their implementation's performance.
> 
> With regard to methodology, I do create SCXML documents in increasing complexity [2] with a NULL datamodel and trigger an infinite sequence of microsteps via spontaneous transitions. These will, ever again, cause a state called 'mark' to be entered and I measure the entries per second for 25s (minus time required for setup) and average the results [3].
> 
> It is most definitely debatable whether the selection of benchmarks (currently LCCA heavy and transition preemption) are representative for any real-world workload, but I still feel that they give a good idea. If you have any other suggestions for (platform agnostic) benchmarks or want me to run your interpreter implementation differently from [3], please drop me a note.
> 
> @David: I could not get JSSCXML to run outside the environment of an HTTP browser. If you could document on how to set it up e.g. wihin Node.js, I will include it.
> 
> Regards
> Stefan
> 
> [1] https://github.com/tklab-tud/uscxml#benchmarks
> [2] https://github.com/tklab-tud/uscxml/tree/master/test/benchmarks
> [3] https://github.com/tklab-tud/uscxml/tree/master/contrib/benchmarks