Re: XHR tests

On Sat, 09 Feb 2008 11:46:02 +0100, Charles McCathieNevile  
<chaals@opera.com> wrote:

> According to MS' testing 4 major browsers (I am guessing they mean  
> Opera, Safari, Mozilla/FF and IE although I am not actually sure) all  
> fail the following tests.

> http://tc.labs.opera.com/apis/XMLHttpRequest/responseXML/009.htm

That test is half-fixed, well the TC bug is fixed. Neither Opera 9.5,  
Firefox 2, nor IE 7 pass though.

Opera 9.5 - returns a document but it's empty (document.documentElement is  
null)
IE7 - like Opera
Firefox 2 - returns a <parsererror> document
Safari - passes test

There is a possible web content compatibility issue here if functions that  
expect documents may throw on null input. For example, does  
xsltprocessor.importStylesheet(null) throw? If it does the implementation  
we test for might break content that expects a broken document error to be  
handled differently.

> http://tc.labs.opera.com/apis/XMLHttpRequest/open/011.htm

Tests the value of .responseText for HEAD requests (Anne, slightly more  
verbose TITLEs please!). Expects null.

Opera 9.5 - buggy, ignore this
IE7 - empty string
Firefox 2 - empty string
Safari - empty string

I say fix spec and test and go with majority vote!

> http://tc.labs.opera.com/apis/XMLHttpRequest/open/012.htm

Tests calling open() twice in a row on an async request (leaving out 3rd  
argument to open() though, I assume a separate test tests if it defaults  
to true)

Opera 9.5 - fails because of a missing readyState 3 event
IE7 - fails because onreadystatechange property is cleared by second  
open() call
Firefox 2 - fails because second open() call throws
Safari - fails because it sends an extra readyState 0 event on second  
open() call and omits readyState 3 event

Wel, have fun discussing that..

> http://tc.labs.opera.com/apis/XMLHttpRequest/open/013.htm

Fails cross-browser because of 4 different implementations of the order  
events should happen in on open() send() open()..

> http://tc.labs.opera.com/apis/XMLHttpRequest/open/016.htm

Tests creating an XMLHttpRequest instance, changing the URL of associated  
document, and loading a relative URL. Assumes that URL should be resolved  
according to location of original document in said window.

Opera 9.5 - fails because it throws INVALID_STATE_ERROR on send()
IE7 - fails because it throws on open()
Firefox 2 - fails because it throws on open()
Safari (version 3 on Windows btw) - fails because it fails to load the new  
document when location is set (!??)

> http://tc.labs.opera.com/apis/XMLHttpRequest/open/024.htm

Same results as 016.htm

> http://tc.labs.opera.com/apis/XMLHttpRequest/open/026.htm

Much like 012.htm. Tests first async open(), sync open() then send()

Opera 9.5 - fails because it leaves out two readyState "1" and one  
readyState "3" events
Firefox 2 - fails because it throws on second open
IE7 - fails because not all expected events are sent (even fewer than  
Opera)
Safari - fails because it throws on send()

> http://tc.labs.opera.com/apis/XMLHttpRequest/open/027.htm

Reverse of 026: tests sync open(), async open(), send()

Opera 9.5 - doesn't send all expected events
Firefox 2 - doesn't send all expected events (unlikt 026 does not throw!)
IE7 - doesn't send all expected events
Safari - throws on send()

> http://tc.labs.opera.com/apis/XMLHttpRequest/open/030.htm

This is more a demo than a test. It tests creating an XHR instance from  
the XMLHttpRequest object of an about:blank window. It doesn't actually  
return a pass/fail condition, but Opera, IE and Safari agree on throwing  
an error. Firefox doesn't.

> http://tc.labs.opera.com/apis/XMLHttpRequest/open/031.htm

Tests creating an XHR instance from the XMLHttpRequest object of an  
IFRAME, removing the IFRAME from the DOM, adding a BASE href element with  
DOM methods to the document in the removed IFRAME, and using the XHR  
instance. (Anne getting rather creative/evil here :-) )

IE7 - my IE7 actually says pass here!
Opera / Safari / Firefox: all three kill the script environment of the  
window object when the IFRAME is removed from the DOM. Hence trying to add  
BASE href throws and the test says failed.

When to kill and garbage collect the script environment inside the IFRAME  
when it's removed from DOM is obviously not the XHR spec's business. I  
suggest this test is relaxed to accept several implementation choices,  
either shutting down the script environment and resolving URL by original  
IFRAME src or doing whatever IE does. We should have a corresponding evil  
security test or two checking that removing the IFRAME won't confuse the  
browser into allowing x-domain requests it shouldn't.

> http://tc.labs.opera.com/apis/XMLHttpRequest/responseText/002.htm

Tests an infinite redirect loop and .responseText. Like open/011.htm it  
flags an empty string as a failure and I think it would be better to  
specify returning an empty string for compatibility with existing  
implementations and possibly content.

> http://tc.labs.opera.com/apis/XMLHttpRequest/responseText/003.htm

This is more like a demo than a test case, it doesn't return a pass/fail  
to the framework and it doesn't really test what TITLE claims. (What it  
*does* test is that readyState is 1 in the first event sent when you call  
send() on a synchronous request..). Anne, please fix this test and figure  
out what you meant to test with this script.

> http://tc.labs.opera.com/apis/XMLHttpRequest/getAllResponseHeaders/006.htm

Tests getAllResponseHeaders() output if the request ends up in an endless  
redirect loop. Assumes that it should return null.

Opera 9.5 - fails because it returns actually returned headers
Safari - fails because it returns an empty string
IE 7 - fails because it returns an empty string
Firefox 2 - fails because send() throws at some point (?!?)

> http://tc.labs.opera.com/apis/XMLHttpRequest/abort/008.htm

Tests a specific sequence of method calls - open() async, send(), abort(),  
open()
Fails in all browsers because not all expected events are sent (or they  
are sent in different order than expected)

> http://tc.labs.opera.com/apis/XMLHttpRequest/send/003.htm



> http://tc.labs.opera.com/apis/XMLHttpRequest/send/005.htm
> http://tc.labs.opera.com/apis/XMLHttpRequest/send/006.htm
> http://tc.labs.opera.com/apis/XMLHttpRequest/send/007.htm
> http://tc.labs.opera.com/apis/XMLHttpRequest/send/008.htm

Think all these tests are broken because  
http://tc.labs.opera.com/apis/XMLHttpRequest/send/support/content-type.script  
doesn't do whatever it's supposed to. Anne?

> http://tc.labs.opera.com/apis/XMLHttpRequest/send/010.htm

This is not a test, it's a sort of demo or experiment, not entirely sure  
for what.

> http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/007.htm
> http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/003.htm


This support script is broken:
http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/support/print-header.script

- all browsers tested actually pass these test.


> http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/010.htm

Tests whether setRequestHeader throws if first argument is an empty string

Opera - doesn't throw, fails
Safari - doesn't throw, fails
IE - throws an exception but e.code is not what the script expects (it's  
undefined)
Firefox - like IE

It seems OK to require that this is a DOMException and thus has .code  
defined, the risk of breaking existing content is pretty close to 0. The  
expected code is 12 for a SYNTAX_ERR.
BTW, the visual fail message when browser doesn't throw should be  
clarified. It's confusing to say "script did not run".

> http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/011.htm

Same as 010 except it tests for first argument being undefined and IE  
doesn't actually throw here.

> http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/013.htm

Tests calling setRequestHeader twice for Content-type. Expects a  
comma-concatenated string to be sent ('Content-type: first value, second  
value').

Test is broken because  
http://tc.labs/XMLHttpRequest/setRequestHeader/support/print-header.script  
is. If the support script had worked as expected, this would have been the  
result:

Firefox - fail
IE - pass
Opera - fail
Safari - pass


> http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/014.htm
> http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/015.htm
> http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/016.htm

Fails because print-header.script is broken

> http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/020.htm

Tests both arguments to setRequestHeader being null. Same results as  
010.htm

> http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/021.htm

Test is entirely broken because it is actually a PHP script which is being  
sent in its entirety. (Hey, it's a feature! At last Sunava gets to see the  
server-side part!)

> http://tc.labs.opera.com/apis/XMLHttpRequest/setRequestHeader/001.htm

Hm.. This passes in some browsers for me. It fails in Firefox (and  
correctly outputs failed). It passes in Opera and IE7. I can't even figure  
out why it fails in Safari - it doesn't send the request..

> http://tc.labs.opera.com/apis/XMLHttpRequest/complex/001.htm

This tests stuff nobody AFAIK has implemented yet like constants:

  if(this.readyState == this.DONE && this.readyState == XMLHttpRequest.DONE)

and DOM2Events support:

client.addEventListener("readystatechange", function() {  
if(this.readyState == this.DONE) log.push(3) }, false)

This test is much too big and should be split into manageable chunks.

-- 
Hallvord R. M. Steen
Core QA JavaScript tester, Opera Software
http://www.opera.com/
Opera - simply the best Internet experience

Received on Tuesday, 12 February 2008 01:37:00 UTC