Re: Follow up on Test Jam

Gopal wrote:
> I just ran the tests under Firefox 12.0
> CORS 1.0 test results:
> Score: 100.00%
> Pass: 5
> Fail: 0
>
> CORS Opera test results:
> Score: 9.28%
> Pass: 18
> Fail: 176
> (I am sure Odin can get much better score under Opera browser)

OK, Opera test suite (I added some more tests after we did our  
implementation, so it's my fault we don't have 100% any more, have to fix  
that some time :P):

Opera    | 97.94%, 190 pass,  4 fail
IE       | 70.62%, 137 pass, 57 fail
Chromium | 59.79%, 116 pass, 78 fail
Firefox  | 56.19%, 109 pass, 85 fail


Internet Explorer
-----------------

IE /could/ maybe have +9 passes and -9 fails because I couldn't get it to  
accept my signature, made it too hard to do - I don't love https. Some  
preflight cache tests are off. Redirect preflight tests all doesn't work  
(hmm, have to look at why). Some bad tests in response, it doesn't expose  
some headers it should, and it exposes others it shouldn't (at least  
according to test, I should double check that it's correct). It also has  
some character encoding errors etc. It's preflighting on some stuff it  
shouldn't (in simple-requests tests). It also gives totally wrong statuses  
(often 0) where it should rather give us the real status (204, 400, 401,  
404, 500). And one place where I'm expecting 0 but got 204. And finally,  
the status codes given on preflight is totally off, where the preflight  
says "Yes, you're 200 OK!" and then the REAL request says "Yes, you're  
like totally 400 dude". This is ofc for accessing API's that use HTTP  
response codes in a meaningful way via javascript. Lots of testfailures  
there.

So all in all, the Trident implementation is quite solid, should at lest  
fix status issues.


Firefox
-------

Firefox doesn't handle cookies correctly, seems to delete(!?) the cookie  
when doing a withCredentials=false request, can't be right. Sync XHR  
doesn't throw when you do withCredentials=true, and server responds  
a-c-a-credentials: false. LOTS of failures because Firefox is not throwing  
correct exceptions (code is not == 19). Has charset problems on the same  
test as IE, but where IE gave almost correctly "…" instead of "…",  
Firefox gives "..." instead. Also two more different errors in the making  
of response headers. Status codes returned are not correct even for the  
simple case where IE does it right, gives status==0 instead of 204 (mind,  
Opera actually had this bug too, but fixed it ;) ). Of course,  
status-preflight doesn't get any better, always returning the very  
unhelpful "0".

All in all, the Gecko implementation is almost as solid as IE. I think the  
error codes might mask some more "real" bugs though, so can't say for  
sure. Should fix status issues and cookie issues.


Chromium
--------

Chromium has also a big problem with error codes. NETWORK_ERR is not code  
101, it's 19. Lots of stupid failures because of that (just like Fx). It's  
caching preflight when age == -1, even though it shouldn't. It has *huge  
problems redirecting*. This is a real problem. And it has a *REALLY HUGE*  
problem parsing origins, it allows MUCH more through than it should. It's  
implementation is horribly broken here. Chromium also returns "..." in the  
charset test. Hmm. Might be a test we should take a look at. Has some  
status errors, but more or less the exact opposite set of tests than IE  
and Fx. It actually does the correct thing in the status-preflight tests!

The WebKit implementation has two very serious and bad bugs. In the origin  
parsing when it gets A-C-A-Origin, and in redirecting. I've already  
notified about the first webkit bug, but can't find a bug for it now.


-- 
Odin Hørthe Omdal (Velmont/odinho) · Core, Opera Software, http://opera.com

Received on Friday, 1 June 2012 08:40:42 UTC