test code for IDN support in validator

Hello Martin, All.

As promised, I had a look at how to support IDNs in the validator. I 
have set up and played with an instance at 
http://qa-dev.w3.org/wmvs/ot/ and it validates e.g 
http://納豆.w3.mag.keio.ac.jp without trouble.

The code is far from perfect, e.g I have not yet managed to make it 
display the actual IDN in the validation results. The following code 
(presented as a diff) will give the result attached as result-1.html, 
i.e warning about a redirection to the xn-- host.


===================================================================
RCS file: /sources/public/validator/httpd/cgi-bin/check,v
retrieving revision 1.305.2.147
diff -u -r1.305.2.147 check
--- check       24 Jun 2004 19:33:30 -0000      1.305.2.147
+++ check       16 Jul 2004 03:15:04 -0000
@@ -1352,6 +1352,14 @@

    my $uri = new URI (ref $q ? $q->param('uri') : $q);

+####### TEST AREA IRI - IDN @ olivier #######
+  use Net::LibIDN ':all';
+  my $iri_host = $uri->host;
+  my $ascii_host = Net::LibIDN::idn_to_ascii($iri_host, 'UTF-8');
+  $uri->host($ascii_host);
+#############################################
+
+
    my $ua = new LWP::UserAgent;
    $ua->agent("W3C_Validator/$VERSION " . $ua->agent);
    $ua->parse_head(0);  # Parse the http-equiv stuff ourselves. @@ Why?
===================================================================


Adding the following helps a bit (i.e no redirection warning) but still 
no pretty display of the IDN. Cf attachment result-2.html


===================================================================
@@ -1422,7 +1430,12 @@
    $File->{Modified}        = $lastmod;
    $File->{Server}          = &ent(scalar $res->server);
    $File->{Size}            = scalar $res->content_length;
-  $File->{URI}             = scalar $res->request->url;
+####### TEST AREA IRI - IDN @ olivier #######
+  my $final_url = $res->request->url;
+  my $final_host = Net::LibIDN::idn_to_unicode($final_url->host(), 
'UTF-8');
+  $final_url->host($final_host);
+  $File->{URI}             = scalar $final_url;
+#############################################
    $File->{'Is Upload'}     = FALSE;

    return $File;
===================================================================

Tell me if that will be enough for your demo, and which you'd rather 
show.
-- 
olivier