Re: multiple requests

> Sacha writes:
>
> > How can I send off multiple requests to various hosts?
> > 
> > Will these be handled 'intelligently'? I know next to nothing
> > about how all the network stuff works, but I should think
> > processing all the requests to one host in one go would be
> > more efficient than reconnecting N times.
> > 
> > Does the library do this for me or do I have to program it
> > explicitly?
> 
> This is handled in such a way that it is completely transparent to the 
> application. The application operates in "requests" which is creating a 
> request object and passing it to libwww. A request may be to the same host 
> or to multiple hosts - it doesn't matter. The net manager (HTNet.c) handles 
> all connections including if the remote server is capable of using 
> persistent connections.
> 
> Therefore, as an application you can safely issue new requests at any point 
> in time. If the number of requests is bigger than the number of open sockets 
> then the requests are queued up in the HTNet manager and started when a 
> socket is available.



Ok... now the question is, how can I send off a batch of requests at the
same time? I mean, how can I tell the library:

	o  here is a bunch of URLs (requests)
	o  go get them
	o  let me know when you're done

?

I looked at the robot, which seems to be the example most likely to fetch
multiple documents. Broadly speaking, it seems to:

	o  start at one URL (HTLoadAnchor in main)
	o  go into the event loop
	o  parse incoming data (is this what HText functions do?)
	o  send off a new request when it sees a url in the incoming data
	o  on termination of a request check whether any other requests are 
	   pending (terminate_handler)	


Now let's assume I have a list of requests to send off in one batch. Does this
mean that I would have to:

	1. provide a function for the event loop to select whenever it is
	   idle (how would I do this?), in which a request is taken off the
	   list and sent (let's call the function 'dispatch')
		
	2. pick one request, make it pre-emptive, and send it

	3. go into the event loop

	4. when the list of requests to send is empty, un-register the
	   'dispatch' function so that the event loop no longer selects it


Would I have to do step 2 at all?

Is this the right/only/best way to do it?

Or is there an HTLoad_X function somewhere which takes a list of requests?



Confused,

Sacha.

Received on Thursday, 16 May 1996 06:13:11 UTC