W3C home > Mailing lists > Public > semantic-web@w3.org > June 2011

Re: Think before you write Semantic Web crawlers

From: Martin Hepp <martin.hepp@ebusiness-unibw.org>
Date: Wed, 22 Jun 2011 21:05:57 +0200
Cc: Yves Raimond <yves.raimond@gmail.com>, Christopher Gutteridge <cjg@ecs.soton.ac.uk>, Daniel Herzig <herzig@kit.edu>, semantic-web@w3.org, public-lod@w3.org
Message-Id: <405D9463-4E85-438E-BF48-1E371CA76B8A@ebusiness-unibw.org>
To: glenn mcdonald <glenn@furia.com>

> If there isn't, why not? We're the Semantic Web, dammit. If we aren't the masters of data interoperability, what are we?
The main question is: Is the Semantic Web an evolutionary improvement of the Web, the Web understood as an ecosystem comprising protocols, data models, people, and economics - or is it a tiny special interest branch.

As said: I bet a bottle of champagne that the academic Semantic Web community's technical proposals will never gain more than 10 % market share among "real" site-owners, because of
- unnecessary complexity (think of the simplicity of publishing an HTML page vs. following LOD publishing principles),
- bad design decisions (e.g explicit datatyping of data instances in RDFa),
- poor documentation for non-geeks, and
- a lack of understanding of the economics of technology diffusion.

Never ever.



On Jun 22, 2011, at 3:18 PM, glenn mcdonald wrote:

> >From my perspective as the designer of a system that both consumes and publishes data, the load/burden issue here is not at all particular to the semantic web. Needle obeys robots.txt rules, but that's a small deal compared to the difficulty of extracting whole data from sites set up to deliver it only in tiny pieces. I'd say about 98% of the time I can describe the data I want from a site with a single conceptual query. Indeed, once I've got the data into Needle I can almost always actually produce that query. But on the source site, I usually can't, and thus we are forced to waste everybody's time navigating the machines through superfluous presentation rendering designed for people. 10-at-a-time results lists, interminable AJAX refreshes, animated DIV reveals, grafting back together the splintered bits of tree-traversals, etc. This is all absurdly unnecessary. Why is anybody having to "crawl" an open semantic-web dataset? Isn't there a "download" link, and/or a SPARQL endpoint? If there isn't, why not? We're the Semantic Web, dammit. If we aren't the masters of data interoperability, what are we?
> glenn
> (www.needlebase.com)
Received on Wednesday, 22 June 2011 19:06:44 UTC

This archive was generated by hypermail 2.4.0 : Tuesday, 5 July 2022 08:45:25 UTC