Re: Page & validation statistics available

On Sat, Apr 12, 2008 at 12:11 AM, Bojan Tesanovic <btesanovic@gmail.com> wrote:
>
> Hey Philip, can you please stop your spider I though that it will crawl 270
> pages not all my site
> I got this message after I submited a form
>
> "There are at least 7997 more URLs to visit. This will take at least 11
> hours and 6 minutes."
> this was after 5 minutes and it will kill your and my server and suck up the
> bandwith.
>
> You shoul set top limit to 1000 pages as there is no way for me to stop your
> spider.

Hi Bojan,
Sorry you're not getting what you want with Nikita, but this is not
the place to report that. This list is for the W3C validator. As it
says on Nikita's "about" page, "Nikita is a private service unrelated
to the W3C."

As to page limits and bandwidth...you asked for a free crawl, and
Nikita crawls 125 pages for free so that's where she stopped on your
site. The status page that said, "There are at least 7997 more URLs to
visit" was telling you what Nikita had seen but not yet visited on
your site. If your site had 100 pages and 8000 images, Nikita might
still be working on it.

You are also free to set your own page limit. It's the second option
on the page where you started your crawl.

Last but not least, as Frank Ellerman pointed out, the visits and link
checks pause 5 seconds between each so as not to overload Nikita or
your server.

I've told Nikita to skip the remaining link checks for your site so
your crawl is now be done.

If you have any questions or comments, please send them directly to me
and not to the W3C list.

Thanks
Philip

>
> On Apr 7, 2008, at 3:47 PM, Nikita The Spider The Spider wrote:
>
>
> Hi all,
> As a result of our conversation on validation statistics last month, I
> was inspired to collect some statistics based on the data my validator
> Nikita sees. If you're interested in the topic, you can read about it
> here:
> http://NikitaTheSpider.com/articles/ByTheNumbers/
>
>
> Cheers
>
>
> --
> Philip
> http://NikitaTheSpider.com/
> Whole-site HTML validation, link checking and more
>
>
>
> Bojan Tesanovic
> http://www.carster.us/
>
>
>
>
>



-- 
Philip
http://NikitaTheSpider.com/
Whole-site HTML validation, link checking and more

Received on Saturday, 12 April 2008 12:33:06 UTC