Re: rubyrdf

* Aredridel <aredridel@nbtsc.org> [2003-09-11 16:47-0600]
> On Thu, 2003-09-11 at 06:02, Dmitry Borodaenko wrote:
> > Ah, finally some traffic on this list! :)

:)

welcome everyone! 

> Yes yes!  I'll be posting more in a couple days after I figure out a
> technical issue with the GUI side of my project, so I can focus again on
> the metadatabase parts.

cool, look fwd to it

> > On Wed, Sep 10, 2003 at 11:59:04AM -0600, Aredridel wrote:
> > > > My stuff was always a bit rough,
> > 
> > Yeah, that was one of the reasons I've decided to write my own RDF
> > storage for Samizdat instead of hacking RubyRdf for my needs: I just
> > couldn't force myself to lay hands on that code ;-)
> 
> Not so bad, but then, I'm comparing to what I have to write in PHP at
> times. Now /that/ is an inelegant language.
> 
> > > > but I think there is now the basics of an interesting system falling
> > > > into place: we have query engines, SQL backends, a half-decent RDF
> > > > parser, etc.  but plenty of room for improvements.
> > 
> > Agreed. One thing in particular I would like to see is decoupling of all
> > these components and development of a common RDF API for Ruby. And then
> > have different projects such as RubyRdf, Redland, Samizdat, map to
> > different parts of this API.

Yes, hopefully we can figure out some lightweight commonalities that
allow different RDF-based tools to communicate. When I integrated the 
RDF4R parser in a big hurry I hacked the two libraries together in a 
pretty rough way. That experience was one of the reasons for setting up
this list...

> I, too, would like to see this. 
> 
> The OO Graph/Statement/Node approach is a good start.  Being able to say
> 	RedlandStore.new("file.rdf").store(Samizdat.ask(nil, nil, nil)) 

Good example. 


> or equivalent would be killer -- common APIs for Node and Statement
> enough that perhaps only Graph is implementation specific, and even that
> would have enough of a common API that pretending they're the same would
> work.

Something like that should work. There are a few subtleties around Node,
eg. whether to allow a graph/datasource to be attached, how to handle
interning etc. if there is a graph, etc.
 
> > > Yeah -- the code is functioning but ugly. Functioning is what other
> > > RDF projects often lack in my experience, though.

RubyRdf was my 'teach myself Ruby' project. My first RDF work was in
Java, ~1998 and what survives of that is now in Libby Miller's Inkling
package. I then made various things in Perl, mainly playing around with
Node-centric APIs to see if RDF could be made super-simple looking in 
languages that allowed methods to be 'faked' at runtime. Then while dispairing
of Perl I stumbled across Ruby... When I move RubyRdf into dev.w3.org
I'll rejig things a little, as it has grown rather awkwardly during
occasional snatched hacking sessions.

> > > I'm not sure as to the state of the SQL support -- I've found
> > > basicrdf's Graph#toSQL, but dumping the entire graph from memory to
> > > SQL as fresh insert statements seems counterproductive to me.  Am I
> > > missing a greater part of the interface?
> > > 
> > > I've not made heads nor tails of RDF4R yet -- just the basicrdf
> > > library.  I'm planning to run the whole thing through rbbr and rdoc
> > > and browse a little more intelligently than my survey so far.  All I
> > > know is that my sixth sense for what project is most promising says
> > > "RubyRdf". <wry grin goes here>
> > 
> > Hm, why don't you take a look into Samizdat then? It doesn't need rbbr
> > to grasp: the whole storage module is less than 500 lines :)

(I'm interested especially in Samizdat's extensions to Squish, and will try to get a copy
installed here to play with. Also the IndyMedia connection is
interesting, but that's another story :)
 
> Yeah, but it's postgres-only, and there's a lot of plpgsql code that I
> don't quite grok yet in there.

FWIW RubyRdf's SQL backend works with MySQL and with PostgreSQL. Haven't
had anything else to try it on. http://www.w3.org/2001/12/rubyrdf/db/
has the respective init scripts (I couldn't find a way to initialize
databasese through DBI in a generic manner, btw).
> 
> > And it is functioning (being the basis for a real project), and its SQL
> > support is fairly advanced (being the core of the RDF storage).
> 
> Good deal.  I'll at least watch it for ideas.

+1

> > > > I'd be very happy to collaborate with you on this, anyway. Re
> > > > alternate backends, I've had MySQL working OK btw., but am generally
> > > > more focussed on Postgres. SQLLite would be interesting certainly.
> > > Alright.  For Ruby, using the DBI layer might make the most sense,
> > > though I'm using SQLite directly at the moment.

I've heard good things about SQLLite...
 
> > In Samizdat, I also use DBI, although, since I develop on Postgres, I'm
> > using some Postgres-specific constructs. I know that it won't be
> > possible to port it to MySQL (triggers, transactions, and subqueries),
> > but I don't know enough about SQLite to check what may be missing there.
> 
> SQLite doesn't do any stored procedures, though it does have limited
> triggers.  It's amazingly complete, actually.  I was surprised.
> 
> > > > Dave Beckett's redland is now quite nicely packaged with Ruby
> > > > wrappers, so there is work we could do there too. So many things to
> > > > do, so little time... ;)
> 
> > > There's plenty to do there.  I compiled redland and the ruby module
> > > the other day, and was less than impressed by the API.  It's about as
> > > un-ruby as one can get -- it's basically the C API wrapped into Ruby,
> > > hardly object-oriented.  

Yes, and we can't really complain to Dave -- he has 7 language
interfaces to maintain, even as Redland itself evolves. It is worth
noting that Redland's Python API was recently improved to be more
Python-friendly, and I've talked a bit to Dave about doing the same
thing w/ the Ruby version if/when consensus emerges about an appropriate
API.

> > >				It might be okay to /use/, but extending it
> > > in Ruby isn't going to fly without some major wrapping of the core
> > > API. I'd love to see the API reworked into a proper Ruby module, with
> > > all the namespaces sorted out and planned support for inheritance
> > > added.
> > 
> > I think first we need to agree on what common RDF API for Ruby should
> > look like?

Yes!

> Or just evolve it until some consensus is reached.  Each do what makes
> life easy for them, and eventually, one or two Right Ways will show up. 
> Then, in usual ruby fashion, do both.

Heh, I had an idea in that fashion: a 'pastiche' module which includes 
methods named after the other major / well known RDF APIs... Jena,
Redland, RdfLib, Cwm, ...

I started this w/ Cwm to learn TimBL's APIs (which have subsequently 
changed), see
http://www.w3.org/2001/12/rubyrdf/pack/lib/RDF4R/pastiche.rb

In general we would do well to look at the Python RDF APIs, since 
the languages are reasonably close. I suggest Redland-python, Rdflib,
Cwm and TRAMP would be a good start...

There are also a few reports from our SWAD-Europe project which might be
useful background reading on APIs, query language convergence, and on
RDBMS storage:

http://www.w3.org/2001/sw/Europe/reports/rdf_api_comparison_report/Overview.html
http://www.w3.org/2001/sw/Europe/reports/rdf_ql_comparison_report/Overview.html
http://www.w3.org/2001/sw/Europe/reports/scalable_rdbms_mapping_report/Overview.html

hope this helps,

Dan

Received on Thursday, 11 September 2003 19:20:22 UTC