Subject: RE: Jetty Bug?
From: Kris Krueger <[email-masked]>
To: Joakim Erdfelt <[email-masked]>
Cc: Greg Wilkins <[email-masked]>, Philippe Le Hegaret <[email-masked]>, "Michael[tm] Smith" <[email-masked]>, Arthur Barstow <[email-masked]>
Date: Thu, 18 Oct 2012 21:39:06 +0000

Indeed ED A0 87 is correct

I made and error when I manually did the utf-8 encoding

\u D807        ==0110 1100 0000 00111

CORRECT == 1110 1101 10 100000 10 000111

WRONG    == 1110 0110 10 110000 10 000111

From: Joakim Erdfelt [mailto:[email-masked]]
Sent: Thursday, October 18, 2012 1:12 PM
To: Kris Krueger
Cc: Greg Wilkins; Philippe Le Hegaret; Michael[tm] Smith; Arthur Barstow
Subject: Re: Jetty Bug?

Other sources for "\ud807"

Online UTF8 test:

http://www.ltg.ed.ac.uk/~richard/utf-8.cgi?input=d807&mode=hex

Python:

>>> u"\ud807".encode("utf8")

'\xed\xa0\x87'

Those 2 result in the 3 byte "Hex UTF-8 Bytes" of ED AO 87

I'm still trying to understand why I can't get the same results out of Java.

--

Joakim Erdfelt <[email-masked]<mailto:[email-masked]>>

www.webtide.com<http://www.webtide.com/>

Developer advice, services and support
from the Jetty & CometD experts.

On Thu, Oct 18, 2012 at 10:32 AM, Joakim Erdfelt <[email-masked]<mailto:[email-masked]>> wrote:

The unicode "\ud807" (or byte array [0xd8, 0x07]) (or binary [01101100 000000111]) do not decode/encode/replace as E6B087 in java.

Here's some results of the experiments in pure java ...

        byte utf[] = new byte[] { (byte) 0xd8, 0x07 };

        String c = new String(utf, Charset.forName("UTF-8"));

        byte buf[] = c.getBytes();

        System.out.println("hex = " + asHex(buf));

hex = EF BF BD 07

        byte buf[] = "\ud807".getBytes(Charset.forName("UTF-8"));

        System.out.println("hex = " + asHex(buf));

hex = 3F

--

Joakim Erdfelt <[email-masked]<mailto:[email-masked]>>

www.webtide.com<http://www.webtide.com/>

Developer advice, services and support
from the Jetty & CometD experts.

On Wed, Oct 17, 2012 at 3:16 PM, Kris Krueger <[email-masked]<mailto:[email-masked]>> wrote:

Thanks for your response it looks like that test the Unicode replacment char in utf-8 form (e.g. \uFFFD).

Maybe you can try to see what occurs when you send \uD807 - which in UTF-8 would be E6B087?

\uD807 -> 3 bytes in UTF-8

\u D807  ==0110 1100 0000 00111

1110 0110 10 110000 10 000111

-Thx

From: Joakim Erdfelt [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Wednesday, October 17, 2012 1:56 PM
To: Greg Wilkins
Cc: Kris Krueger; Philippe Le Hegaret; Michael[tm] Smith; Arthur Barstow
Subject: Re: Jetty Bug?

Also, the test webapp we use for testing can be found at https://github.com/jetty-project/jetty-autobahn-websocket-webapp

To use ...

Clone the repo
$ git clone git://github.com/jetty-project/jetty-autobahn-websocket-webapp.git<http://github.com/jetty-project/jetty-autobahn-websocket-webapp.git>

Build it
Note: you'll need Java JDK 7
$ cd jetty-autobahn-websocket-webapp
$ mvn clean install

Run it
$ mvn exec:exec

Test against it (sorry, no web UI)
ws://localhost:9001/

--
Joakim Erdfelt <[email-masked]<mailto:[email-masked]>>
www.webtide.com<http://www.webtide.com>
Developer advice, services and support
from the Jetty & CometD experts.

On Wed, Oct 17, 2012 at 1:47 PM, Joakim Erdfelt <[email-masked]<mailto:[email-masked]>> wrote:
There's not much information in that test report about what's going on.

Internally, we use the autobahn testsuite for testing our websocket support.
Here's an example of the testcase for this scenario with Jetty 9.0.0-SNAPSHOT.

http://joakim.erdfelt.com/jetty/autobahn/server-9.0.0-SNAPSHOT/jetty_9_0_0_snapshot_case_6_23_1.html

And the same one for Jetty 7.6.1 (the version of jetty you are testing against)
http://joakim.erdfelt.com/jetty/autobahn/server-7.6.1/jetty_7_6_1_case_6_23_1.html

Btw, what is the license for the source files at http://w3c-test.org/webapps/WebSockets/tests/jetty/ ?
Can I take them, update them for Jetty 9.0.0 and put the updates on github?

--
Joakim Erdfelt <[email-masked]<mailto:[email-masked]>>
www.webtide.com<http://www.webtide.com>
Developer advice, services and support

from the Jetty & CometD experts.

On Wed, Oct 17, 2012 at 1:37 PM, Greg Wilkins <[email-masked]<mailto:[email-masked]>> wrote:
Hi Kris,

quick answer (without looking at the specifics of the issue),

I'm can help with the jetty-7 implementation, but I've also CC'd
Joakim Erdfelt, who has rewritten our websocket implementation for
jetty-9, so he can check we've not repeated any mistakes.

I'll look at the details of this issue later today (AU time).

cheers

On 18 October 2012 06:42, Kris Krueger <[email-masked]<mailto:[email-masked]>> wrote:
> Hello Greg,
>
> My name is Kris Krueger and I am the W3C WebSocket API Test Suite owner.  I CC'd a few folks from the W3C and Art the Web Apps Co-Chair for the Web Apps Working group so you don't think this is some random email :)
>
> The W3C WebApps Working Group has created a set of test to test support for the W3C WebSocket API specification (http://www.w3.org/TR/websockets/) to help browser interoperability.  One of the more recent changes to this specification is how to handle an unpaired surrogate.  Browsers will no longer throw an exception and instead just use the unicode replacement character when this situation occurs.
>
> We are using Jetty (7.6.1.v20120215) on the W3C for testing and have come across a bug in the Jetty server, which I'd like you to be aware of and maybe possible fix.
>
> Here is a pointer to the test case, the issue is that on line 29 the evt.reason comes back as '?' and not the replacement character.
>     http://www.w3c-test.org/webapps/WebSockets/tests/submissions/Microsoft/Close-reason-unpaired-surrogates.htm
>
> Would you be the right person to investigate this bug in Jetty?
>
> -Thanks!

--
Greg Wilkins <[email-masked]<mailto:[email-masked]>>
http://www.webtide.com
Developer advice and support from the Jetty & CometD experts.