Fwd: Transforming JSON APIs to JSON-LD

FYI

---------- Forwarded message ----------
From: Markus Lanthaler <markus.lanthaler@gmx.net>
Date: 4 May 2013 15:12
Subject: Transforming JSON APIs to JSON-LD
To: public-linked-json@w3.org


Hi all,

I just had a conversation with Vikash on IRC and showed him how a response
from a API exposing JSON can be transformed to JSON-LD using Antonio
Garrote's json-ld-macros [1]. I think this is interesting for other people
as well so here a short description of the necessary steps:

 1) define a macro to transform from JSON to JSON-LD
 2) retrieve the resource
 3) run the macro to transform to JSON-LD
 4) optionally, expand the JSON-LD to inspect it

Here's a jsfiddle illustrating it: http://jsfiddle.net/9ZU8q/4/

Please note that this is a minimal example. I've just set @vocab in the
context. So all URLs are still interpreted as strings. Properties such as
"followers_url" would have to be type-coerced in the context so that a
JSON-LD processor would recognize them as URLs.


Unfortunately jQuery's promises don't assimilate DOMFutures (or I missed
something) so there's some nesting in line 36.


I've also included the full IRC transcript below which include a couple of
more details.


[1] https://github.com/antoniogarrote/json-ld-macros


Cheers,
Markus


-------- Full transcript ----------

[13:39] mlnt    hi vikash
[13:43] vikash  mlnt, hey!
[13:43] vikash  thanks mlnt for coming :-)
[13:44] vikash  First of all, apologies.. json-ld-macros had slipped my mind
entirely :/
[13:45] vikash  but after the email, I have cloned the repo and its working
locally as well and I see there are issues too :)
[13:45] vikash  before I ask further questions mlnt can you first elaborate
on your points...
[13:46] mlnt    well, what didn't you understand in my mail?
[13:47] vikash  mlnt, ok! From the first line, did you mean that I can reuse
the code base for the creator. Though I see it uses context of socialrdf..
which is in a way vague
[13:47] mlnt    also, it is typically better to discuss things on the
mailing list since that allows other people to chime in or learn by just
lurking on the list.. don't be affraid if you don't understand something,
just ask :-)
[13:48] vikash  thanks :-)
[13:48] mlnt    sure, it's open source
[13:48] mlnt    so you can reuse it
[13:48] vikash  so from the json-ld-macros, I can understand the algo for
conversion
[13:48] vikash  and use that in my html :-)
[13:49] mlnt    right
[13:49] mlnt    in your HTML you would provide some UI to generate the
transformation rules for json-ld-macros
[13:49] vikash  can you elaborate the term transformation rules.
[13:49] mlnt    json-ld-macros is not restricted to socialrdf
[13:50] mlnt    you can use anything with it
[13:50] mlnt    it is completely generic
[13:50] vikash  ok! Please correct me if I am wrong here.
[13:50] vikash  Say
[13:51] vikash  I take the input for transformation rules, if use inputs it
then its cool, else I will use schema.org as default
[13:51] vikash  *user inputs
[13:51] vikash  ?
[13:51] vikash  Also … "If you are talking about the schema.org tool, that's
slightly different because you don't have a JSON document to start with."
???
[13:53] mlnt    the demo uses GitHub's API
[13:54] mlnt    so it's similar to your task transforming LinkedIN's API
[13:54] vikash  yup.
[13:54] vikash  And...
[13:55] mlnt    let's keep using GitHub's API for the moment because it
doesn't need authentication
[13:55] mlnt    do you have curl installed?
[13:55] vikash  yup!
[13:55] mlnt    ok
[13:56] mlnt    run
[13:56] mlnt    curl -k https://api.github.com/users/lanthaler
[13:56] mlnt    you will get back a JSON object
[13:56] vikash  did that. also by testing ?callbacks :0
[13:56] vikash  :)
[13:59] mlnt    no, we don't need that for the moment
[13:59] mlnt    ok, so the goal now is to translate that into valid JSON-LD
[13:59] vikash  yes...
[14:00] mlnt    to do so you have to write some transformation rules
[14:00] mlnt    let me set-up a jsfiddle
[14:00] vikash  thanks'
[14:01] vikash  all this will be very useful in docs and tutorials :)
[14:07] mlnt    ok
[14:07] mlnt    here's the JSfiddle: http://jsfiddle.net/9ZU8q/
[14:07] mlnt    it just renders the GitHub data as is at the moment
[14:08] vikash  yes! JSON.stringify() :-)
[14:08] mlnt    no we can start describing the transformation
[14:08] mlnt    let's start by adding a context
[14:09] vikash  Yo \o/
[14:09] vikash  ...
[14:10] mlnt    http://jsfiddle.net/9ZU8q/1/
[14:11] mlnt    ok, so this adds a @context which just sets @vocab
[14:13] mlnt    that's already valid JSON-LD
[14:14] mlnt    you have two tasks
[14:14] mlnt    a) write the transformation rules for the LinkedIN API
[14:14] mlnt    b) create a UI which makes it simple to create such
transformation rules
[14:16] vikash  Yes, so thats pretty evident now. But say I am planning to
work on the first part! just the creator app. For that I can ask the user
his @vocab?
[14:17] vikash  so the creator tool will just be
http://jsfiddle.net/9ZU8q/1/ with a @vocab either provided as user input or
schema.org which becomes pretty evident?
[14:17] vikash  which could be one way
[14:19] vikash  I will ask few more questions but before that can you please
throw some light on "nt. It would be nice if you could describe the forms
using JSON-LD as well instead of hard-coding them into HTML."
[14:21] mlnt    no, for the creator app you would predefine a couple of
things for certain schema.org types
[14:21] vikash  yes..
[14:22] vikash  intact lets do one thing.
[14:22] mlnt    so you don't ask the user for @vocab but for the type.. so
Person, Event, etc.
[14:22] vikash  Let me implement something small
[14:22] vikash  depending upon which we shall improve.
[14:23] vikash  I think that ways I will understand better.
[14:23] vikash  mlnt, can you explain the last point.. the hard-coding one..
[14:24] vikash  and mlnt sorry if I woke you up early morning :D
[14:25] mlnt    it's 2pm here
[14:25] mlnt    I'm in Europe
[14:25] mlnt    so all right :-)
[14:26] vikash  hehe! Oh! I thought you are also staying in US or MIT Boston
:-P
[14:27] mlnt    no
[14:27] mlnt    Austria at the moment
[14:27] mlnt    but I'm from Italy originally
[14:27] mlnt    btw. here's the final jsfiddle: http://jsfiddle.net/9ZU8q/2/
[14:27] mlnt    transforms the data to json-ld and then expands it
[14:28] vikash  mlnt, and .. just small bits of info on "create a UI which
makes it simple to create such transformation rules"
[14:28] vikash  mlnt, Did you complete the github App for JSON to JSON-LD :P
[14:28] vikash  ?
[14:29] vikash  mlnt, And thanks! This is very helpful :)
[14:29] mlnt    sorry, I don't understand your question
[14:31] vikash  mlnt, Does this last fiddle the entire JSON to JSON-LD
conversion tool for github api?
[14:33] mlnt    no, it just converts GitHub user representations
[14:33] vikash  mlnt, because thats one of my brownie tasks :-) *wink*
[14:33] mlnt    so just for urls starting with https://api.github.com/users/
[14:33] vikash  Cool! That was very insightful!
[14:33] mlnt    you have to look up the API documentation and do something
similar for the other representations
[14:34] vikash  expanded representations are always required?
[14:34] vikash  mlnt, ^
[14:34] mlnt    perhaps it is possible to create a single context that you
can then simple reference in your transformation rules
[14:34] vikash  yes.. I saw that example in spec :)
[14:34] mlnt    no, expanded representation just shows you nicely how it is
interpreted in JSON-LD
[14:34] mlnt    it shows you how all the properties etc. are expanded to
URLs
[14:35] vikash  Yes!
[14:35] mlnt    in the example you'll see that name is expanded to
http://example.com/name because I used @vocab: http://example.com/
[14:35] vikash  yes! I could figure that out :-)
[14:36] mlnt    It also shows you that all URLs are still just interpreted
as values (strings) instead of URLs, e.g., followers_url
[14:36] mlnt    so you would have to adjust your context or transformation
rules accordingly
[14:40] vikash  mlnt, and where shall be the code mirror used?
[14:41] vikash  will read the macro js, jsonld.js, future.js and other
related files and will implement something now!
[14:43] mlnt    ok
[14:44] mlnt    doesn't matter where the code will be used
[14:44] mlnt    we will find a place
[14:44] vikash  cool! so I am keeping that aside for time being
[14:44] vikash  :D
[14:47] vikash  mlnt, Thanks a tonne :D
[14:50] mlnt    your are welcome