- From: Henri Sivonen <hsivonen@iki.fi>
- Date: Wed, 26 Nov 2008 13:26:38 +0200
On Nov 26, 2008, at 13:19, Julian Reschke wrote: > So wouldn't it make sense to address the common use case so that it > doesn't require the "bot" (the non-HTML UA) to parse the response > body? That seems like a bad optimization. Adding an off-the-shelf HTML parser to a bot is much easier than tuning the general crawling functionality and task-specific functionality of a bot. -- Henri Sivonen hsivonen at iki.fi http://hsivonen.iki.fi/
Received on Wednesday, 26 November 2008 03:26:38 UTC