From: Kris Krueger To: Joakim Erdfelt CC: Greg Wilkins , Philippe Le Hegaret , "Michael[tm] Smith" , Arthur Barstow Subject: RE: Jetty Bug? Thread-Topic: Jetty Bug? Thread-Index: Ac2snbJo8cfgtwyhRBGuaWKi9WLESgARC9sAAABXeIAAAE8gAAAMi+XAAB6gmAAABZjlAAAMFHVQ Date: Thu, 18 Oct 2012 21:39:06 +0000 References: <0A605FCDA3A4DC45A98B0DDD1C5351A70383DB8D@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> <0A605FCDA3A4DC45A98B0DDD1C5351A70383DEAD@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> In-Reply-To: Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: multipart/alternative; boundary="_000_6971787274726580737178666671697374706979657474717267777_" MIME-Version: 1.0 --_000_6971787274726580737178666671697374706979657474717267777_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Indeed ED A0 87 is correct I made and error when I manually did the utf-8 encoding \u D807 =3D=3D0110 1100 0000 00111 CORRECT =3D=3D 1110 1101 10 100000 10 000111 WRONG =3D=3D 1110 0110 10 110000 10 000111 From: Joakim Erdfelt [mailto:joakim@intalio.com] Sent: Thursday, October 18, 2012 1:12 PM To: Kris Krueger Cc: Greg Wilkins; Philippe Le Hegaret; Michael[tm] Smith; Arthur Barstow Subject: Re: Jetty Bug? Other sources for "\ud807" Online UTF8 test: http://www.ltg.ed.ac.uk/~richard/utf-8.cgi?input=3Dd807&mode=3Dhex Python: >>> u"\ud807".encode("utf8") '\xed\xa0\x87' Those 2 result in the 3 byte "Hex UTF-8 Bytes" of ED AO 87 I'm still trying to understand why I can't get the same results out of Java= . -- Joakim Erdfelt > www.webtide.com Developer advice, services and support from the Jetty & CometD experts. On Thu, Oct 18, 2012 at 10:32 AM, Joakim Erdfelt > wrote: The unicode "\ud807" (or byte array [0xd8, 0x07]) (or binary [01101100 0000= 00111]) do not decode/encode/replace as E6B087 in java. Here's some results of the experiments in pure java ... byte utf[] =3D new byte[] { (byte) 0xd8, 0x07 }; String c =3D new String(utf, Charset.forName("UTF-8")); byte buf[] =3D c.getBytes(); System.out.println("hex =3D " + asHex(buf)); hex =3D EF BF BD 07 byte buf[] =3D "\ud807".getBytes(Charset.forName("UTF-8")); System.out.println("hex =3D " + asHex(buf)); hex =3D 3F -- Joakim Erdfelt > www.webtide.com Developer advice, services and support from the Jetty & CometD experts. On Wed, Oct 17, 2012 at 3:16 PM, Kris Krueger > wrote: Thanks for your response it looks like that test the Unicode replacment cha= r in utf-8 form (e.g. \uFFFD). Maybe you can try to see what occurs when you send \uD807 - which in UTF-8 = would be E6B087? \uD807 -> 3 bytes in UTF-8 \u D807 =3D=3D0110 1100 0000 00111 1110 0110 10 110000 10 000111 -Thx From: Joakim Erdfelt [mailto:joakim@intalio.com] Sent: Wednesday, October 17, 2012 1:56 PM To: Greg Wilkins Cc: Kris Krueger; Philippe Le Hegaret; Michael[tm] Smith; Arthur Barstow Subject: Re: Jetty Bug? Also, the test webapp we use for testing can be found at https://github.com= /jetty-project/jetty-autobahn-websocket-webapp To use ... Clone the repo $ git clone git://github.com/jetty-project/jetty-autobahn-websocket-webapp.= git Build it Note: you'll need Java JDK 7 $ cd jetty-autobahn-websocket-webapp $ mvn clean install Run it $ mvn exec:exec Test against it (sorry, no web UI) ws://localhost:9001/ -- Joakim Erdfelt > www.webtide.com Developer advice, services and support from the Jetty & CometD experts. On Wed, Oct 17, 2012 at 1:47 PM, Joakim Erdfelt > wrote: There's not much information in that test report about what's going on. Internally, we use the autobahn testsuite for testing our websocket support= . Here's an example of the testcase for this scenario with Jetty 9.0.0-SNAPSH= OT. http://joakim.erdfelt.com/jetty/autobahn/server-9.0.0-SNAPSHOT/jetty_9_0_0_= snapshot_case_6_23_1.html And the same one for Jetty 7.6.1 (the version of jetty you are testing agai= nst) http://joakim.erdfelt.com/jetty/autobahn/server-7.6.1/jetty_7_6_1_case_6_23= _1.html Btw, what is the license for the source files at http://w3c-test.org/webapp= s/WebSockets/tests/jetty/ ? Can I take them, update them for Jetty 9.0.0 and put the updates on github? -- Joakim Erdfelt > www.webtide.com Developer advice, services and support from the Jetty & CometD experts. On Wed, Oct 17, 2012 at 1:37 PM, Greg Wilkins > wrote: Hi Kris, quick answer (without looking at the specifics of the issue), I'm can help with the jetty-7 implementation, but I've also CC'd Joakim Erdfelt, who has rewritten our websocket implementation for jetty-9, so he can check we've not repeated any mistakes. I'll look at the details of this issue later today (AU time). cheers On 18 October 2012 06:42, Kris Krueger > wrote: > Hello Greg, > > My name is Kris Krueger and I am the W3C WebSocket API Test Suite owner. = I CC'd a few folks from the W3C and Art the Web Apps Co-Chair for the Web = Apps Working group so you don't think this is some random email :) > > The W3C WebApps Working Group has created a set of test to test support f= or the W3C WebSocket API specification (http://www.w3.org/TR/websockets/) t= o help browser interoperability. One of the more recent changes to this sp= ecification is how to handle an unpaired surrogate. Browsers will no longe= r throw an exception and instead just use the unicode replacement character= when this situation occurs. > > We are using Jetty (7.6.1.v20120215) on the W3C for testing and have come= across a bug in the Jetty server, which I'd like you to be aware of and ma= ybe possible fix. > > Here is a pointer to the test case, the issue is that on line 29 the evt.= reason comes back as '?' and not the replacement character. > http://www.w3c-test.org/webapps/WebSockets/tests/submissions/Microsof= t/Close-reason-unpaired-surrogates.htm > > Would you be the right person to investigate this bug in Jetty? > > -Thanks! -- Greg Wilkins > http://www.webtide.com Developer advice and support from the Jetty & CometD experts. --_000_6971787274726580737178666671697374706979657474717267777_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Indeed ED A0 87 is correct

I made and error when I man= ually did the utf-8 encoding

 <= /p>

\u D807        = =3D=3D0110 1100 0000 00111


CORRECT =3D=3D 1110 1101 10 100000 10 000111

WRONG    =3D=3D 1110 0110 10 110000 1= 0 000111

 <= /p>

 <= /p>

From: Joakim E= rdfelt [mailto:joakim@intalio.com]
Sent: Thursday, October 18, 2012 1:12 PM
To: Kris Krueger
Cc: Greg Wilkins; Philippe Le Hegaret; Michael[tm] Smith; Arthur Bar= stow
Subject: Re: Jetty Bug?

 

Other sources for "\ud807"

 

Online UTF8 test:

http://www.ltg.ed.ac.uk/~richard/utf-8.cgi?i= nput=3Dd807&mode=3Dhex

 

Python:

>>> u"\ud807".encode("utf8&q= uot;)

'\xed\xa0\x87'

 

Those 2 result in the 3 byte "Hex UTF-8 Bytes&q= uot; of ED AO 87

 

I'm still trying to understand why I can't get the s= ame results out of Java.


--

Joakim Erdfelt <joakim@intalio.com>

Developer advice, services and support
from the Jetty & CometD experts.



On Thu, Oct 18, 2012 at 10:32 AM, Joakim Erdfelt <= ;joakim@intalio.com= > wrote:

The unicode "\ud807" (or byte array [0xd8,= 0x07]) (or binary [01101100 000000111]) do not decode/encode/replace as&nb= sp;E6B087 in java.

 

Here's some results of the experiments in pure java = ...

 

        byte utf[] =3D new byte[= ] { (byte) 0xd8, 0x07 };

        String c =3D new String(= utf, Charset.forName("UTF-8"));

        byte buf[] =3D c.getByte= s();

        System.out.println("= ;hex =3D " + asHex(buf));

 

hex =3D EF BF BD 07

 

        byte buf[] =3D "\ud= 807".getBytes(Charset.forName("UTF-8"));

        System.out.println("= ;hex =3D " + asHex(buf));

 

hex =3D 3F

 

 

--

Joakim Erdfelt <joakim@intalio.com>

Developer advice, services and support
from the Jetty & CometD experts.



On Wed, Oct 17, 2012 at 3:16 PM, Kris Krueger <krisk@microsoft.com<= /a>> wrote:

Thanks for your response it looks like that test the= Unicode replacment char in utf-8 form (e.g. \uFFFD).

Maybe you can try to see what occurs when you send \uD807 - which in UTF-8 = would be E6B087?

\uD807 -> 3 bytes in UTF-8

\u D807  =3D=3D0110 1100 0000 00111

1110 0110 10 110000 10 000111

-Thx


From: Joakim Erdfelt [mailto:joakim@intalio.com]
Sent: Wednesday, October 17, 2012 1:56 PM
To: Greg Wilkins
Cc: Kris Krueger; Philippe Le Hegaret; Michael[tm] Smith; Arthur Barstow Subject: Re: Jetty Bug?


Also, the test webapp we use for testing can be found at https://github.com/jetty-project/jetty-autobahn-websocket-webapp=

To use ...

Clone the repo
$ git clone git://github.com/jetty-project/jett= y-autobahn-websocket-webapp.git

Build it
Note: you'll need Java JDK 7
$ cd jetty-autobahn-websocket-webapp
$ mvn clean install

Run it
$ mvn exec:exec

Test against it (sorry, no web UI)
ws://localhost:9001/


--
Joakim Erdfelt <= joakim@intalio.com>
www.webtide.com Developer advice, services and support
from the Jetty & CometD experts.


On Wed, Oct 17, 2012 at 1:47 PM, Joakim Erdfelt <joakim@intalio.com> wrote:
There's not much information in that test report about what's going on.

Internally, we use the autobahn testsuite for testing our websocket support= .
Here's an example of the testcase for this scenario with Jetty 9.0.0-SNAPSH= OT.

http://joakim.erdfe= lt.com/jetty/autobahn/server-9.0.0-SNAPSHOT/jetty_9_0_0_snapshot_case_6_23_= 1.html

And the same one for Jetty 7.6.1 (the version of jetty you are testing agai= nst)
http://joakim.erdfelt.com/jetty/autob= ahn/server-7.6.1/jetty_7_6_1_case_6_23_1.html

Btw, what is the license for the source files at http://w3c-tes= t.org/webapps/WebSockets/tests/jetty/ ?
Can I take them, update them for Jetty 9.0.0 and put the updates on github?=


--
Joakim Erdfelt <= joakim@intalio.com>
www.webtide.com Developer advice, services and support

from the Jetty & CometD experts.


On Wed, Oct 17, 2012 at 1:37 PM, Greg Wilkins <gregw@intalio.com> wrote:
Hi Kris,

quick answer (without looking at the specifics of the issue),

I'm can help with the jetty-7 implementation, but I've also CC'd
Joakim Erdfelt, who has rewritten our websocket implementation for
jetty-9, so he can check we've not repeated any mistakes.

I'll look at the details of this issue later today (AU time).

cheers


On 18 October 2012 06:42, Kris Krueger <krisk@microsoft.com> wrote:
> Hello Greg,
>
> My name is Kris Krueger and I am the W3C WebSocket API Test Suite owne= r.  I CC'd a few folks from the W3C and Art the Web Apps Co-Chair for = the Web Apps Working group so you don't think this is some random email :)<= br> >
> The W3C WebApps Working Group has created a set of test to test suppor= t for the W3C WebSocket API specification (http://www.w3.org/TR/websockets/) to hel= p browser interoperability.  One of the more recent changes to this specification is how to handle an unpaired sur= rogate.  Browsers will no longer throw an exception and instead just u= se the unicode replacement character when this situation occurs.
>
> We are using Jetty (7.6.1.v20120215) on the W3C for testing and have c= ome across a bug in the Jetty server, which I'd like you to be aware of and= maybe possible fix.
>
> Here is a pointer to the test case, the issue is that on line 29 the e= vt.reason comes back as '?' and not the replacement character.
>     http://www.w3c-test.org/webapps/WebSockets/tests/submissions/Microsoft/Clos= e-reason-unpaired-surrogates.htm
>
> Would you be the right person to investigate this bug in Jetty?
>
> -Thanks!



--
Greg Wilkins <gre= gw@intalio.com>
http://www.webtide.com=
Developer advice and support from the Jetty & CometD experts.

 

 

--_000_6971787274726580737178666671697374706979657474717267777_--