RE: IPR tests, part 1

David,
  My comments in line, set off by '>>'

-----Original Message-----
From: David Junger [mailto:tffy@free.fr] 
Sent: Monday, April 08, 2013 4:18 PM
To: www-voice@w3.org (www-voice@w3.org)
Subject: IPR tests, part 1

Yesterday I used Saxon 9.4 to apply the ECMAScript XSL on all tests.

I have added them to my test page (also made some modifications, like allowing a few manual tests to run automatically fixing some errors and incorrect behavior, some of which I reported here), up to test495, and here are some comments.
I have activated a total of 164 tests including 3 that aren't from the W3C. 161 of them pass.

150 and 151 are duplicates; moreover they produce a <data name="Var3">[1, 2, 3]</data>.
According to C.2.1, that should be interpreted as the string "[1, 2, 3]" rather than an actual array. Which is obviously not what the test intends. It should transform into <data name="Var3" expr="[1, 2, 3]" /> instead.
>> Yes, the tests are the same, but they are testing two different assertions.  My document structure doesn't give me an easy way of indicating that if you pass one test, you satisfy two assertions, so I repeat the test.
>> It's certainly would be easier to use an expr to create the array, but the opposite is true in XPath.  Is there any notation we can use that would create an array in-line?  Could we use JSON?  
test 307 could be automated with what we said recently about the ECMAScript datamodel:
First it tries to access a declared but unbound variable (because binding=late). According to the spec we must get 'undefined' and no error.
Then it binds the number 1 to that variable and tries to access an undeclared property of that variable. Which returns 'undefined' as well, and no error.
>> This test can be automated for ECMAScript, as you say.  But it is intended to apply to _any_ datamodel, and we can't be sure if it can be automated in all of them.  The assertion says that the behavior in the two cases must be the same, and I doubt that we can generate a .txml file such that for _any_ datamodel we can generate an xslt file that tests automatically that the results are the same.  

test 344's comments could be improved. It suggets that a non-boolean expression in a 'cond' should raise an error, which is of course absurd in ECMAScript where any value can be converted to boolean. But the test happens to put a value with a syntax error in there so of course there will be an error in that case.

>> I'll update the comment

test 351 expects the first sent event to have a sendid, just because the <send> element had an 'id' attribute. My understanding of 3.14 is that an id must be generated for each <send> that doesn't have one, but according to 5.10.1 that id must be included in the event when and only when it is an error event resulting from a failed <send>.
>> Hmm, you're right and this is a bug.  The SCXML Event I/O processor specifies that this should be filled in, but section 5.10.1 says that it's only filled in error cases. I think that 5.10.1 should be changed to say it's filled in if there's an error or if the sender specifies it  (with a reference to the Event I/O Processors section for details.)

Speaking of which, I do not see the point of having a new sendid or invokeid generated each time a send or invoke element is executed. I can imagine one case where that may be useful for send, to cancel a specific delayed send if you start several with a <foreach>, but then why allow some of them to have fixed IDs with the id attribute? And each <invoke> can only have one simultaneous session running.
>> The usecases for this were 1) <send> invoked inside <foreach>, as you indicate 2) entering/exiting and then quickly re-entering a state with an <invoke>.  When you get events back from the invoke the second time you enter the state, you might want to know whether they were from the previous or current invocation.  (We don't know how quickly the invoke will be cancelled.)  The developer can specify a fixed id in both these cases if he knows that he won't care about the different instances.  A single fixed id can be convenient if you have a bunch of delayed <send>s and want to cancel all of them at once.  

test 353 tests event.language, which is completely new to me, and, from what I can see in the test, is supposed to reflect the originating datamodel of the event. I am all for some kind of MIME type and encoding information when an event is send to a remote system, but that's transport stuff and not strictly related to the datamodel.

>> I've removed this test from the suite.  It was left over from an earlier draft of the spec which did define event.language.

test 456 seems to expect that undefined+1 == 1. Well, it's not the case. Maybe if you convert undefined to a number first, or initialize the variable to 0 so it's not undefined.

>> This has also been fixed.  Stefan ran into the same problem.  

test 457 works, but the comments say more than the test. First, an array can be instanceof some other global's Array object, in which case, annoyingly, it will tell you that it is not instanceof (your own global's) Array.
Second, there is no good reason why any iterable object couldn't be iterable with <foreach>. ECMAScript's for...in construct can iterate over Arrays items, other objects's properties (we should say to iterate only over the object's own properties, since that is what you want to do 99% of the time), and the characters in strings.
In the case of strings, it is unnecessary to make a shallow copy because strings are immutable.
>> What should the assertion/comment be, then?  The current text was suggested by someone a long time ago.  It's fine with me if we want to generalize it or modify it.  

test 488, the transformation left a conf:eventDataVal="" attribute in the first transition in s1.

>> Stefan hit this same bug.  I've changed the test to use conf:emptyEventData="", instead, since that is more precise.  

			David

Received on Tuesday, 9 April 2013 15:54:50 UTC