<Further LC Followup from IE> RE: Potential bugs identified in XHR LC Test Suite

2nd email to the new alias from me!
Dev, test and I ran a few more tests and had some results to share. A few of these should probably be clarified in the LC draft or the test cases should change.
Details below...

http://tc.labs.opera.com/apis/XMLHttpRequest/responseXML/009.htm

When Parsing Error happens, IE would still retain responseXML and put error information on the object.   Isnt this better than null as there’s more relevant information for the web developer?

http://tc.labs.opera.com/apis/XMLHttpRequest/responseXML/001.htm

The test is expecting us to return NULL in case open() has not been called.  We throw an exception in IE.   I’d pre fer if the spec says “MUST return null OR an exception” otherwise I fear sites today will be broken.

http://tc.labs.opera.com/apis/XMLHttpRequest/responseXML/012.htm
http://tc.labs.opera.com/apis/XMLHttpRequest/responseXML/013.htm

This test really doesn’t test XHR here. It seems to be focused on manipulating the XML DOM. (I also don’t think Microsoft.XMLDOM supports getElementById for an XML document FYI). Also, if I'm barking up the wrong tree here please let me know!

http://tc.labs.opera.com/apis/XMLHttpRequest/open/032.htm
What's the purpose of this test case and which part of spec is it testing? It’s difficult to understand that.

http://tc.labs.opera.com/apis/XMLHttpRequest/abort/003.htm
The abort() method resets event listeners. I’m looking at the  4/15 spec (on W3C site) and was wondering where this is specified?

http://tc.labs.opera.com/apis/XMLHttpRequest/onreadystatechange/004.htm
We don't raise onreadystatechange events from within the onreadystatechange event handler as there's danger of recursion. FYI I can't find any guidance here in the spec.

http://tc.labs.opera.com/apis/XMLHttpRequest/send/009.htm
http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/017.htm
Another dependency on e.code (As mentioned in the previous round of feedback on these tests. We can’t actually run the test until this is removed.

http://tc.labs.opera.com/apis/XMLHttpRequest/send/011.htm
In this test send(1) doesn’t work. The reason being we don’t cast an argument to a string.   This is also not defined in the spec.

Thanks!

________________________________
From: Sunava Dutta
Sent: Thursday, June 05, 2008 7:47 PM
To: Web API public; IE8 Core AJAX SWAT Team
Subject: Potential bugs identified in XHR LC Test Suite

Thanks for writing these cases by LC exit. It really makes the process of providing feedback prior to CR a lot easier. I ran these (the tests below fail on Safari3 ,Firefox 3 and IE8) with my team and had a few questions. If these issues can be addressed we can give further feedback and recommendations on the results/implementations. (Let me know if I’m wrong on our test analysis!) The rest of the tests that fail (without issues in the test itself) are being investigated further by my team so expect more over the next few days as we dig in.


http://tc.labs.opera.com/apis/XMLHttpRequest/open/033.htm
This test seems to have a bug even though it passes. Line 24 uses top.opener.rr. The framework reports FAIL although the test passes.

http://tc.labs.opera.com/apis/XMLHttpRequest/getAllResponseHeaders/001.htm

http://tc.labs.opera.com/apis/XMLHttpRequest/getAllResponseHeaders/004.php
http://tc.labs.opera.com/apis/XMLHttpRequest/getResponseHeader/001.htm

http://tc.labs.opera.com/apis/XMLHttpRequest/getResponseHeader/007.htm

http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/013.htm

http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/014.htm

http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/015.htm

http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/016.htm

http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/001.htm
http://tc.labs.opera.com/apis/XMLHttpRequest/readyState/002.htm
These tests seem to be failing when the test autorun is used in IE. Running the tests individually causes them to pass on IE8. Looks like a bug in the test framework.


http://tc.labs.opera.com/apis/XMLHttpRequest/getAllResponseHeaders/005.php
This test fails because the network timeout is too short and passes sometimes (Unpredictable).

http://tc.labs.opera.com/apis/XMLHttpRequest/send/005.htm

http://tc.labs.opera.com/apis/XMLHttpRequest/send/007.htm

http://tc.labs.opera.com/apis/XMLHttpRequest/send/008.htm

This seems to be a minor test bug. The file we are receiving does not contain the string “PASS”, which is necessary for the test to pass.


http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/010.htm
http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/020.htm
The exception object here is not directly supported by IE, causing us to fail here. Can the test be tweaked so we can test the XHR compliance here? Thanks!

http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/021.htm
The PHP page doesn’t seem to be producing valid content

http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/034.php
http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/035.php
Server-side PHP seems to be causing the test to pass or fail and we can’t determine the PASS criteria. May we get a pointer to the source here? (I think this was given a long time before but I can’t seem to dig it up!)

 Meanwhile, I’d like to re-iterate a point I had raised up awhile back. Are the tests going to be ‘complete’ /comprehensive at CR in relation to the spec? MSFT obviously wants this test suite to be official ensuring that third parties do not write individual test cases undermining the credibility of the suite and demonstrating increased/decreased compliance post CR (when it’s much harder to make changes).

Thanks!




--
Sunava Dutta
Program Manager (AJAX) - Developer Experience Team, Internet Explorer
One Microsoft Way, Redmond WA 98052
TEL# (425) 705-1418
FAX# (425) 936-7329