Re: Making Linkchecker working parallel

On 5 Sep 2005, at 20:04, Dominique Hazaël-Massieux wrote:
>
> I got it to work by adding                 $res->request($request);
> after line 465 of RobotUA.pm;

I have contacted Marc Langheinrich about this, proposing him that  
patch. However, it seems we should not expect an answer before the  
end of the month (September 2005).

> that said, I don't know if this is a bug
> of Parallel::RobotUA or a too strong assumption on our side that a
> response necessarily has an attached request OR a previous request
> attached. Maybe we have to, I don't know.

It's an assumption on our side... because there's too little  
documentation anyway.

I think it's a fair assumption, and I can imagine no-one noticed the  
crippled response object because the typical spider/crawler built  
upon that library probably has no post-processing to be done on a  
403'd resource...

-- 
olivier