Re: Think before you write Semantic Web crawlers

Am 22.06.2011 um 23:01 schrieb Lin Clark:

> On Wed, Jun 22, 2011 at 9:33 PM, Sebastian Schaffert <sebastian.schaffert@salzburgresearch.at> wrote:
> 
> Your complaint sounds to me a bit like "help, too many clients access my data".
> 
> I'm sure that Martin is really tired of saying this, so I will reiterate for him: It wasn't his data, they weren't his servers. He's speaking on behalf of people who aren't part of our insular community... people who don't have a compelling reason to subsidize a PhD student's Best Paper award with their own dollars and bandwidth.

And what about those companies subsidizing PhD students who write crawlers for the normal Web? Like Larry Page in 1998?

> 
> Agents can use Linked Data just fine without firing 150 requests per second at a server. There are TONs of use cases that do not require that kind of server load.

And what if in the future 100.000 software agents will access servers? We will have the scalability issue eventually even without crawlers, so let's try to solve it. In the eyeball web, there are also crawlers without too much of a problem, and if Linked Data is to be successful we need to do the same.

Greetings,

Sebastian
-- 
| Dr. Sebastian Schaffert          sebastian.schaffert@salzburgresearch.at
| Salzburg Research Forschungsgesellschaft  http://www.salzburgresearch.at
| Head of Knowledge and Media Technologies Group          +43 662 2288 423
| Jakob-Haringer Strasse 5/II
| A-5020 Salzburg

Received on Wednesday, 22 June 2011 23:08:32 UTC