Re: Bringing fast triples to Node.js with HDT

Hi Luca,

> As you say JavaScript
> typed arrays provide lots of functionality for reading files but
> you'll eventually hit a hard limit when it comes to memory space.
> However I think it would still be an interesting solution because even
> only 30MB in memory could amount to 10 million triples which is a lot
> of data.

I heard that the original HDT authors have plans for a pure JavaScript implementation. It shouldn't be too hard, given the existing code bases in C++ and Java, but it would take some effort. The specification should give the necessary clues.

At the moment, most use cases we see for HDT are at the server, so the Node.js solution is sufficient. But with clients becoming more and more powerful, that might change quickly :-)

> (no indexing required if it's in HDT already)

Just a small addition here: the library will generate an implementation-specific .hdt.index file the first time, to allow faster searches. This is an additional index, which will be reused from the second time onwards. In later versions of the library, I might offer the option to load the file without creating an index, but then operations would be slower.

> This makes me want to write a Pubby version for
> Node.js that works using HDT's triple matching capabilities.

Keep us updated ;-)

Ruben

Received on Wednesday, 1 October 2014 07:48:53 UTC