Transforming JSON APIs to JSON-LD

Hi all,

I just had a conversation with Vikash on IRC and showed him how a response
from a API exposing JSON can be transformed to JSON-LD using Antonio
Garrote's json-ld-macros [1]. I think this is interesting for other people
as well so here a short description of the necessary steps:

 1) define a macro to transform from JSON to JSON-LD
 2) retrieve the resource
 3) run the macro to transform to JSON-LD
 4) optionally, expand the JSON-LD to inspect it

Here's a jsfiddle illustrating it: http://jsfiddle.net/9ZU8q/4/

Please note that this is a minimal example. I've just set @vocab in the
context. So all URLs are still interpreted as strings. Properties such as
"followers_url" would have to be type-coerced in the context so that a
JSON-LD processor would recognize them as URLs.


Unfortunately jQuery's promises don't assimilate DOMFutures (or I missed
something) so there's some nesting in line 36.


I've also included the full IRC transcript below which include a couple of
more details.


[1] https://github.com/antoniogarrote/json-ld-macros


Cheers,
Markus


-------- Full transcript ----------

[13:39]	mlnt	hi vikash
[13:43]	vikash	mlnt, hey!
[13:43]	vikash	thanks mlnt for coming :-)
[13:44]	vikash	First of all, apologies.. json-ld-macros had slipped my mind
entirely :/
[13:45]	vikash	but after the email, I have cloned the repo and its working
locally as well and I see there are issues too :)
[13:45]	vikash	before I ask further questions mlnt can you first elaborate
on your points...
[13:46]	mlnt	well, what didn't you understand in my mail?
[13:47]	vikash	mlnt, ok! From the first line, did you mean that I can reuse
the code base for the creator. Though I see it uses context of socialrdf..
which is in a way vague
[13:47]	mlnt	also, it is typically better to discuss things on the
mailing list since that allows other people to chime in or learn by just
lurking on the list.. don't be affraid if you don't understand something,
just ask :-)
[13:48]	vikash	thanks :-)
[13:48]	mlnt	sure, it's open source
[13:48]	mlnt	so you can reuse it
[13:48]	vikash	so from the json-ld-macros, I can understand the algo for
conversion
[13:48]	vikash	and use that in my html :-)
[13:49]	mlnt	right
[13:49]	mlnt	in your HTML you would provide some UI to generate the
transformation rules for json-ld-macros
[13:49]	vikash	can you elaborate the term transformation rules.
[13:49]	mlnt	json-ld-macros is not restricted to socialrdf
[13:50]	mlnt	you can use anything with it
[13:50]	mlnt	it is completely generic
[13:50]	vikash	ok! Please correct me if I am wrong here.
[13:50]	vikash	Say
[13:51]	vikash	I take the input for transformation rules, if use inputs it
then its cool, else I will use schema.org as default
[13:51]	vikash	*user inputs
[13:51]	vikash	?
[13:51]	vikash	Also … "If you are talking about the schema.org tool, that's
slightly different because you don't have a JSON document to start with."
???
[13:53]	mlnt	the demo uses GitHub's API
[13:54]	mlnt	so it's similar to your task transforming LinkedIN's API
[13:54]	vikash	yup.
[13:54]	vikash	And...
[13:55]	mlnt	let's keep using GitHub's API for the moment because it
doesn't need authentication
[13:55]	mlnt	do you have curl installed?
[13:55]	vikash	yup!
[13:55]	mlnt	ok
[13:56]	mlnt	run
[13:56]	mlnt	curl -k https://api.github.com/users/lanthaler
[13:56]	mlnt	you will get back a JSON object
[13:56]	vikash	did that. also by testing ?callbacks :0
[13:56]	vikash	:)
[13:59]	mlnt	no, we don't need that for the moment
[13:59]	mlnt	ok, so the goal now is to translate that into valid JSON-LD
[13:59]	vikash	yes...
[14:00]	mlnt	to do so you have to write some transformation rules
[14:00]	mlnt	let me set-up a jsfiddle
[14:00]	vikash	thanks'
[14:01]	vikash	all this will be very useful in docs and tutorials :)
[14:07]	mlnt	ok
[14:07]	mlnt	here's the JSfiddle: http://jsfiddle.net/9ZU8q/
[14:07]	mlnt	it just renders the GitHub data as is at the moment
[14:08]	vikash	yes! JSON.stringify() :-)
[14:08]	mlnt	no we can start describing the transformation
[14:08]	mlnt	let's start by adding a context
[14:09]	vikash	Yo \o/
[14:09]	vikash	...
[14:10]	mlnt	http://jsfiddle.net/9ZU8q/1/
[14:11]	mlnt	ok, so this adds a @context which just sets @vocab
[14:13]	mlnt	that's already valid JSON-LD
[14:14]	mlnt	you have two tasks
[14:14]	mlnt	a) write the transformation rules for the LinkedIN API
[14:14]	mlnt	b) create a UI which makes it simple to create such
transformation rules
[14:16]	vikash	Yes, so thats pretty evident now. But say I am planning to
work on the first part! just the creator app. For that I can ask the user
his @vocab?
[14:17]	vikash	so the creator tool will just be
http://jsfiddle.net/9ZU8q/1/ with a @vocab either provided as user input or
schema.org which becomes pretty evident?
[14:17]	vikash	which could be one way
[14:19]	vikash	I will ask few more questions but before that can you please
throw some light on "nt. It would be nice if you could describe the forms
using JSON-LD as well instead of hard-coding them into HTML."
[14:21]	mlnt	no, for the creator app you would predefine a couple of
things for certain schema.org types
[14:21]	vikash	yes..
[14:22]	vikash	intact lets do one thing.
[14:22]	mlnt	so you don't ask the user for @vocab but for the type.. so
Person, Event, etc.
[14:22]	vikash	Let me implement something small
[14:22]	vikash	depending upon which we shall improve.
[14:23]	vikash	I think that ways I will understand better.
[14:23]	vikash	mlnt, can you explain the last point.. the hard-coding one..
[14:24]	vikash	and mlnt sorry if I woke you up early morning :D
[14:25]	mlnt	it's 2pm here
[14:25]	mlnt	I'm in Europe
[14:25]	mlnt	so all right :-)
[14:26]	vikash	hehe! Oh! I thought you are also staying in US or MIT Boston
:-P
[14:27]	mlnt	no
[14:27]	mlnt	Austria at the moment
[14:27]	mlnt	but I'm from Italy originally
[14:27]	mlnt	btw. here's the final jsfiddle: http://jsfiddle.net/9ZU8q/2/
[14:27]	mlnt	transforms the data to json-ld and then expands it
[14:28]	vikash	mlnt, and .. just small bits of info on "create a UI which
makes it simple to create such transformation rules"
[14:28]	vikash	mlnt, Did you complete the github App for JSON to JSON-LD :P
[14:28]	vikash	?
[14:29]	vikash	mlnt, And thanks! This is very helpful :)
[14:29]	mlnt	sorry, I don't understand your question
[14:31]	vikash	mlnt, Does this last fiddle the entire JSON to JSON-LD
conversion tool for github api?
[14:33]	mlnt	no, it just converts GitHub user representations
[14:33]	vikash	mlnt, because thats one of my brownie tasks :-) *wink*
[14:33]	mlnt	so just for urls starting with https://api.github.com/users/
[14:33]	vikash	Cool! That was very insightful!
[14:33]	mlnt	you have to look up the API documentation and do something
similar for the other representations
[14:34]	vikash	expanded representations are always required?
[14:34]	vikash	mlnt, ^
[14:34]	mlnt	perhaps it is possible to create a single context that you
can then simple reference in your transformation rules
[14:34]	vikash	yes.. I saw that example in spec :)
[14:34]	mlnt	no, expanded representation just shows you nicely how it is
interpreted in JSON-LD
[14:34]	mlnt	it shows you how all the properties etc. are expanded to
URLs
[14:35]	vikash	Yes!
[14:35]	mlnt	in the example you'll see that name is expanded to
http://example.com/name because I used @vocab: http://example.com/
[14:35]	vikash	yes! I could figure that out :-)
[14:36]	mlnt	It also shows you that all URLs are still just interpreted
as values (strings) instead of URLs, e.g., followers_url
[14:36]	mlnt	so you would have to adjust your context or transformation
rules accordingly
[14:40]	vikash	mlnt, and where shall be the code mirror used?
[14:41]	vikash	will read the macro js, jsonld.js, future.js and other
related files and will implement something now!
[14:43]	mlnt	ok
[14:44]	mlnt	doesn't matter where the code will be used
[14:44]	mlnt	we will find a place
[14:44]	vikash	cool! so I am keeping that aside for time being
[14:44]	vikash	:D
[14:47]	vikash	mlnt, Thanks a tonne :D
[14:50]	mlnt	your are welcome

Received on Saturday, 4 May 2013 13:13:15 UTC