RE: https over proxy patch

Isn't it HTERR_SYSTEM error? It seems your program is failing
on connect() call. 

Here is the line in HTTCP.c

  HTRequest_addSystemError(request, ERR_FATAL, socerrno, NO, "connect");

Are you sure you can at least connect (with let's say telnet)
to this port on your proxy? You could use a sniffer like ethereal
to see what is actually going on.

Are you trying to use https port of your proxy?
Actually this should work with another port (regular http proxy
which supports CONNECT). Your proxy doesn't have to know
about SSL. After connect it works as a tunnel.
Try to use your HTTP port of your proxy instead of https.

I've managed to make my program work with both POST and GET
with https via proxy. I also used purify and found huge memory leaks
in this patch, and some other memory violations. 

Here is the list:

1. HTChannl.c free_channel() - freeing unallocated memory
htssl is deleted in isa->close() functions before

        /* https-proxy */
        if (ch->htssl) {
            /* deleted in isa->close methods  */
            /* HTSSL_free(ch->htssl); */ /*MMMMM*/
            ch->htssl = NULL;
        }

2. HTHost.c

This is a really huge memory leak ~32K per request.

I've added 

/*MMMMM*/
struct _HTOutputStream {
     const HTOutputStreamClass *  isa;
};

after similar lines for _HTInputStream in HTHost.c

and

modified one TODO section 

         HTChannel * ch = HTHost_channel(me);
         if (ch)  {
             /* TODO: release input && output rather then setting to NULL */
             HTTRACE(PROT_TRACE, "HTHost  deleting old input & output");

             /*MMMMM*/
             HTChannel_deleteInput(ch, HT_INTERRUPTED);
             HTChannel_deleteOutput(ch, HT_INTERRUPTED);

             HTChannel_input(ch)->isa->close(HTChannel_input(ch));
             HTChannel_output(ch)->isa->close(HTChannel_output(ch));

             HTChannel_setInput(ch, NULL);
             HTChannel_setOutput(ch, NULL);
         }

This releases input/output streams and fixes leaks.

3. HTTPReq.c

I also added HT_FREE(me->url); in the following piece in HTTPReq.c
This fixed another leak. me->url initially was allocated by HTParse and
afterwards overwritten by second HTParse in 'if METHOD_CONNECT'.
So the memory allocated by first HTParse was lost. This was also
per / request leak.

         else {
             me->url = HTParse(addr, "", PARSE_PATH | PARSE_PUNCTUATION);
             if (method == METHOD_OPTIONS) {
                 /*
                 ** We don't preserve the final slash or lack of same
through
                 ** out the code. This is mainly for optimization reasons
                 ** but it gives a problem OPTIONS. We can either send a "*"
                 ** or a "/" but not both. For now we send a "*".
                 */
                 if (!strcmp(me->url, "/")) *me->url = '*';
             }
             else if (method == METHOD_CONNECT) {
               HT_FREE(me->url); /*MMMMM*/
                 me->url = HTParse(addr, "", PARSE_HOST );
                 if (!strchr(me->url, ':'))
                   StrAllocCat(me->url, ":443");
             }

Another problem I had was an infinite loop on READ / WRITE without
invoking select(). I've found a fix which works for me, but I'm not so sure
whether it is absolutely correct.

In HTHost.c HostEvent() function there is:

if (targetNet) {
  HTTRACE(CORE_TRACE, "Host Event.. READ passed to `%s\'\n" _
      HTAnchor_physical(HTRequest_anchor(HTNet_request(targetNet))));
      if ((ret = (*targetNet->event.cbf)(HTChannel_socket(host->channel),
                    targetNet->event.param, type)) != HT_OK)   return ret;
}
 
I've changed it to:

if (targetNet) {
  HTTRACE(CORE_TRACE, "Host Event.. READ passed to `%s\'\n" _
      HTAnchor_physical(HTRequest_anchor(HTNet_request(targetNet))));
      if ((ret = (*targetNet->event.cbf)(HTChannel_socket(host->channel),
                    targetNet->event.param, type)) != HT_OK)
            HTTRACE(CORE_TRACE, "Host Event.. READ passed Before RETURN\n");
/*MMMMM*/
      return ret;
}

so  it always returns "ret" in that place. This fixed endless loop in
SSLReader/SSLWriter for me.

To make POST work I had to restore entityAnchor after CONNECT and before
POST.
The entity anchor probably should be saved in a way similar to
HTRequest_saveOrigMethod/
HTRequest_origMethod. I did it in my program code and it works, however
generic
change should be probably made in libwww. 

I hope this will help somebody...

--Mikhail

> -----Original Message-----
> From:	Putz Doug [SMTP:PutzDoug@JohnDeere.com]
> Sent:	Wednesday, July 25, 2001 10:13 AM
> To:	'www-lib@w3.org'
> Subject:	Re: https over proxy patch
> 
> Mikhail,
> 
> I've also tried this patch(with all the suggested changes incorpurated) to
> get https to work through proxy when doing a POST, however, I'm running
> into
> a different problem. It seems that the https request is not connecting
> throught the proxy do to an error. Any thoughts...my logging shows:
> 
>  
> 
>  library - :HTTPEvent..state..HTTP_BEGIN
>  library - :HTHost 40039d58 going to state TCP_ERROR.
>  library - :Error....... Add  73        Severity: 1     Parameter:
> `Invalid
> argument'   Where: `connect'
>  library - :HTHost 40039d58 going to state TCP_BEGIN.
>  library - :Host connect Unlocking Host 40039d58
>  library - :HTTPEvent....HTTP_ERROR
>  library - :HTTP Clean.. Called with status -1, net 4003a008
> 
> The proxy server I'm trying to go through has one port assigned for HTTP
> requests and another for HTTPS(secure) requests. The HTTP request works
> just
> fine yet the HTTPS is getting an 'Invalid argument'. I'm thinking that the
> HTSSLWriter object is getting created to soon and interfearing with the
> initial CONNECT request/response handshake with the PROXY server. Your
> thoughts?
> 
>

Received on Wednesday, 25 July 2001 12:21:15 UTC