- From: Jim Barnett <1jhbarnett@gmail.com>
- Date: Sat, 28 Jun 2014 22:00:17 -0400
- To: Stefan Radomski <radomski@tk.informatik.tu-darmstadt.de>
- CC: Gavin Kistner <phrogz@me.com>, Jim Barnett <jim.barnett@genesys.com>, "www-voice@w3.org (www-voice@w3.org)" <www-voice@w3.org>
- Message-ID: <53AF7331.1070707@gmail.com>
I have converted the rest of the tests in Stefan's original email to use conf:delay. I may have screwed something up so check carefully the first time you run them. - Jim On 6/28/2014 5:05 PM, Stefan Radomski wrote: > Works great! For ECMAScript the following substitutions in > confECMA.xsl will reduce all delay in test175 to 1/100th: > > <!-- delayexpr takes the value of the specified variable --> > <xsl:template match="//@conf:delayFromVar"> > <xsl:attribute name="delayexpr">(Var<xsl:value-of select="." > />.slice(0, - 1)) * 100 + 'ms'</xsl:attribute> > </xsl:template> > > <!-- computes a delayexpr based on the value passed in. this lets > platforms determine how long to delay timeout > events which cause the test to fail. The default value provided here > is pretty long --> > <xsl:template match="//@conf:delay"> > <xsl:attribute name="delayexpr">'<xsl:value-of select=". * > 100"/>ms'</xsl:attribute> > </xsl:template> > > > On Jun 28, 2014, at 18:43, Jim Barnett <1jhbarnett@gmail.com > <mailto:1jhbarnett@gmail.com>> wrote: > >> I have added conf:delay that takes a relative value. The default >> implementation in confECMA just appends 's' to it (so >> conf:delay=".5" turns into delayexpr="'.5s'". I have converted >> test175 to use this element. >> >> However, I haven't been able to test the results because the pyscxml >> console (which I normally use for testing) is offline. So I have >> just checked in these changes until I can be sure that I haven't >> screwed things up. Once it's clear that conf:delay is working ok, >> I'll convert the other tests. >> >> test175 is an odd test case, because it tests that delayexpr can be >> taken from a variable, so there is still a 1s delay hard-coded into >> the test. There should be more of a speed-up in other tests (or I >> can reduce the hardcoded delay to .5s and then set conf:delay=".25". >> >> Let me know what you think, >> >> - Jim >> >> On 6/28/2014 11:38 AM, Gavin Kistner wrote: >>> Whoops, sent that too fast. I was going to add: >>> >>> However, I’d prefer to remove this hack and have the tests generated >>> with reasonable values. I support Stefan’s suggestion of having the >>> value of the delay be relative, e.g. conf:delay=“1” vs conf:delay=“2”. >>> >>> On Jun 28, 2014, at 9:35 AM, Gavin Kistner <phrogz@me.com >>> <mailto:phrogz@me.com>> wrote: >>> >>>> FWIW, my “solution” is that my test runner waits until there are no >>>> more events to process peeks at the next delayed event off the >>>> queue, and manually advances the time for the state machine by the >>>> necessary amount so that it will be dequeued. My insanely slow, >>>> unoptimized runtime is currently processing the auto tests in about >>>> 2 seconds. >>>> >>>> On Jun 28, 2014, at 9:09 AM, Stefan Radomski >>>> <radomski@tk.informatik.tu-darmstadt.de >>>> <mailto:radomski@tk.informatik.tu-darmstadt.de>> wrote: >>>> >>>>> Can’t you keep the relative delay values and introduce a factor by >>>>> which implementations can multiply these? This way, the performant >>>>> implementations can set the factor small and others keep it at 1 >>>>> or even bigger. >>>>> >>>>> If it’s too much of a hassle, just keep it as it is and I will >>>>> live with a 1 minute delay every time the tests run. >>>>> Stefan >>>>> >>>>> On Jun 28, 2014, at 16:10, Jim Barnett <jim.barnett@genesys.com >>>>> <mailto:jim.barnett@genesys.com>> wrote: >>>>> >>>>>> One problem that occurs to me is that we may want different >>>>>> timeout values for different tests (some tests involve <invoke>, >>>>>> which is going to be slower than other operations). So setting a >>>>>> single value forconf:delaymay not work well. We could just say >>>>>> that platforms can adjust the timeout values to suit their >>>>>> conditions, but that requires a lot of editing for each >>>>>> platform. Would it be acceptable for each platform to >>>>>> setconf:delayto the largest timeout value it would ever need? >>>>>> -Jim >>>>>> *From:*Jim Barnett [mailto:1jhbarnett@gmail.com] >>>>>> *Sent:*Saturday, June 28, 2014 8:35 AM >>>>>> *To:*www-voice@w3.org <mailto:www-voice@w3.org> >>>>>> *Subject:*Re: Make SCXML IRP tests more CI-able >>>>>> I wanted to make sure that the timeout didn't fire too soon since >>>>>> that would cause the test to fail when it should succeed. Maybe I >>>>>> should addconf:timeOutand let each platform set the value as it >>>>>> sees fit. >>>>>> >>>>>> - Jim >>>>>> On 6/28/2014 8:09 AM, Stefan Radomski wrote: >>>>>> >>>>>> Hey there, >>>>>> we’d like to run the non-manual tests as part of continuous >>>>>> integration. As such, we’d be happy if we could reduce their >>>>>> runtime somewhat. Worst offenders are: >>>>>> ecma/test175.scxml = 3.07 sec >>>>>> ecma/test185.scxml = 2.07 sec >>>>>> ecma/test186.scxml = 2.07 sec >>>>>> ecma/test187.scxml = 10.07 sec >>>>>> ecma/test207.scxml = 3.10 sec >>>>>> ecma/test208.scxml = 5.07 sec >>>>>> ecma/test210.scxml = 5.07 sec >>>>>> ecma/test236.scxml = 2.07 sec >>>>>> ecma/test237.scxml = 3.07 sec >>>>>> ecma/test252.scxml = 2.07 sec >>>>>> ecma/test409.scxml = 1.07 sec >>>>>> ecma/test422.scxml = 5.09 sec >>>>>> ecma/test423.scxml = 1.07 sec >>>>>> ecma/test553.scxml = 3.07 sec >>>>>> ecma/test554.scxml = 2.08 sec >>>>>> ecma/test579.scxml = 2.07 sec >>>>>> Total Test time (real) = 65.79 sec >>>>>> Can we reduce the various delay expressions in these tests? >>>>>> I’d vote to cut them all to one tenth of their current >>>>>> values. That is 1s becomes 100ms - I can’t imagine that there >>>>>> is an implementation out there that takes more than a >>>>>> millisecond to process a benign macrostep and it cuts down >>>>>> total time for all automated tests to appr. 1/4th. >>>>>> Total Test time (real) = 18.83 sec >>>>>> It’s not exactly critical to do, but considering how often I >>>>>> run the ECMA tests it more than justified this mail. >>>>>> Regards >>>>>> Stefan >>>>>> >>>>>> -- >>>>>> Jim Barnett >>>>>> Genesys >>>>> >>>> >>> >> >> -- >> Jim Barnett >> Genesys > -- Jim Barnett Genesys
Received on Sunday, 29 June 2014 02:01:03 UTC