JSON-LD Telecon Minutes for 2012-08-21

The minutes from last week's call are now available here:


Full text of the discussion follows including a link to the audio

JSON-LD Community Group Telecon Minutes for 2012-08-21

   1. DrupalCon discussion on JSON-LD
   2. Property Generators and .compact() API
   3. Alternative to combinatorial .compact()
   1. The group is committed to support language maps and
      property generators in JSON-LD 1.0.
   2. Support optional features in .compact(). Any optional
      feature that is implemented, MUST be implemented in a specific
      way as outlined in the JSON-LD API.
   Manu Sporny
   Gregg Kellogg
   Gregg Kellogg, Manu Sporny, Markus Lanthaler, Niklas Lindström,
   François Daoust, Dave Longley

Gregg Kellogg is scribing.
Gregg describes talk he's giving at NoSQL Now!, slides are above,
   good feedback from folks.

Topic: DrupalCon discussion on JSON-LD

Manu Sporny: http://groups.drupal.org/node/249128
Manu Sporny:  Lin has asked what JSON-LD features should be used
   in Drupal.
   … Two issues that were important have been re-opened: property
   generators and language map.
   … When they were re-opened they became nervous about stability
   of JSON-LD.
   … We need to discuss why they were re-opened, and what to do
   going forward.
   … The message is that the features are unstable, and could go
   away, which is not true.
Markus Lanthaler:  The main reason they were opened was because
   the algorithms haven't been updated, and we haven't discussed how
   compaction works.
   … In the property generators issue, there was a comment that
   this was supported, but the array-position wasn't.
   … I thought we might want to discuss the array-position issue
   … The main reason was to make sure that the algorithms are
   updated before the issue is closed.
Markus Lanthaler:  I had some question about our support for
   property generators.
Manu Sporny:  Perhaps we should discuss how compaction should
   … the main thing is to signal to Drupal that we're not pulling
   the rug out from under them.
   … We re-opened to make sure that all the work was completed,
   not that the feature was in doubt.
Markus Lanthaler:
Manu Sporny:  It's not just for drupal, we have a similar
   mechanism in RDFa, and other companies have indicated their need
   for it too.
   … It's not just for drupal, but they did provide motivation.
Niklas Lindström:  I understand that the mutliple-property
   feature could be seen as "un-pure" from a theoretical
   perspective. You should rely on inference and other things, but
   those are all ideals.
   … This case supports a fairly easy way for both producers and
   consumers to manage such properties.
   … The alternative would be to create different documents for
   different consumers, which isn't very useful.
   … It's un-pure, but useful.
Manu Sporny:  I don't hear any objection to keeping the feature,
   we just need to work out details, specifically in compaction.
Gregg Kellogg:  We may want to say right now that it's not
   addressed in compaction, it would allow us to close out this
   particular feature. [scribe assist by Manu Sporny]
Gregg Kellogg:  We could say that if we /do not/ describe
   anything happening in compaction, then we may not do anything.
   [scribe assist by Manu Sporny]
Gregg Kellogg:  The algorithm for compaction would be a bit
   different... more heavyweight. [scribe assist by Manu Sporny]
Gregg Kellogg:  I don't think we want to wrap these things
   together, we don't want to message that just because we haven't
   wrapped some of the API details up, that we're not going to
   support the feature. [scribe assist by Manu Sporny]
Niklas Lindström: +1 for that
Manu Sporny:  we do a proposal that says "support both
   property-generators and language-maps; close issues; create new
   issues for compaction implications".
Markus Lanthaler:  you can't make use of these things without the
   API, so we need all parts.
Gregg Kellogg:  I think we need to do the expansion... no
   controversy there, the issue comes in via compaction and in via
   framing... certainly, there is some open work to be done in the
   expansion algorithms. It's reasonable to link them to the
   existing issues. [scribe assist by Manu Sporny]
Gregg Kellogg:  Our practice has been to keep issues open until
   the language has been updated to reflect what's done. It's a
   book-keeping exercise. [scribe assist by Manu Sporny]
Manu Sporny:  we need to do something to tell them that there is
   a clear effort to support these features.
   … The proposal is to underscore that both property generators
   and language maps are supported features, and the group is
   working out the details of how the API supports them.
Manu Sporny:  we could post on drupal that we're supporting it,
   the issue is resolved, but the work's not all done.
Manu Sporny:  the reason we need to be more careful on how we use
   the issue tracker is because the the perception on outside

PROPOSAL:  The group is committed to support language maps and
   property generators in JSON-LD 1.0.

Gregg Kellogg: +1
Manu Sporny: +1
Niklas Lindström: +1
Markus Lanthaler: +1
François Daoust: +1

RESOLUTION: The group is committed to support language maps and
   property generators in JSON-LD 1.0.

Topic: Property Generators and .compact() API

Manu Sporny:  Markus' point is that we've typically dealt with
   things all in one issue, why is this different.
Gregg Kellogg:  I don't know where other people are wrt to this.
   The .compact() algorithm is getting to be almost unimplementably
   complex. [scribe assist by Manu Sporny]
Gregg Kellogg:  It becomes almost unacceptable in an algorithm,
   when in-fact, most people typically generate JSON-LD through
   their own customized mechanisms. I do make use of compaction, but
   mostly as a way of doing fairly standard normalizations. [scribe
   assist by Manu Sporny]
Gregg Kellogg:  I usually generate properties as arrays - so for
   example, dropping single value arrays. Making use of CURIEs, or
   other terms. I find compaction() useful for that stuff. I don't
   find .frame() useful at all. I usually do in-memory linking.
   [scribe assist by Manu Sporny]
Gregg Kellogg:  It would be too much work to get the API to do
   what I wanted... noticed this in .frame(). If it doesn't work, it
   doesn't work and there is no insight into why. Very hard to debug
   frames. [scribe assist by Manu Sporny]
Gregg Kellogg:  I think these things make it fairly useless,
   adding more complexity to .compact() is not something I'd like to
   do. [scribe assist by Manu Sporny]
Gregg Kellogg:  Which term is selected for a specific property is
   one of those things. [scribe assist by Manu Sporny]
Gregg Kellogg:  The ranking algorithm... the very act of
   splitting properties among multiple terms, is strange. [scribe
   assist by Manu Sporny]
Gregg Kellogg:  if you have a property that has an equivalent
   IRI, the datatypes are split among those terms.... if you are
   trying to be entirely symmetric between compaction and expansion.
   It leads to unexpected results. [scribe assist by Manu Sporny]
Manu Sporny:  At a high level, I agree on the complexity
   argument. That's one of the reasons in the beginning we said that
   when we talk about compaction, we leave a lot of the magic up the
   the implementation.
   … Maybe we should define a core-set of interoperability that
   you should expect from compaction, but it leaves serious concerns
   about interoperability.
Manu Sporny:  2 questions: 1) do we expect compaction to be
   interoperable. 2) were do we draw the line on interoperability?
   … property generators are a case where we may not want the
   algorithm to do anything when it goes to compact those values.
Niklas Lindström:  I also agree on a high level, especially
   w.r.t. property generators.
   … I do see the need for ranking, and tend to think that that
   should be interoperable.
   … The mapping from RDF to elastic search indexing of data. I'd
   like that to be as automatic as possible.
   … From my point of view, although I agree we should debate, I
   feel we should draw the line at where you cannot determine what
   term is used from the value alone.
   … That rules out property generators. Does not rule out
   ranking, and might not language maps, but I agree that it's on
   the edge.
Manu Sporny:  would we be opposed to some implementations
   implementing it, and others not.
Gregg Kellogg:  In RDFa, we have examples of optional features -
   implementing the feature is not required, but if it is required,
   it has interoperability requirements. Processor graphs is one
   such example. [scribe assist by Manu Sporny]
Gregg Kellogg:  Maybe we have a flag that allows best-effort
   compaction... maybe we can have a set of parameters that are
   associated with these optional features. Processors can declare
   whether they are supported or not. They're supported if the
   appropriate options are passed to the algorithm. It also makes
   testing easier. It can also reduce the computational complexity
   to support odd cases that are more likely to never come about.
   [scribe assist by Manu Sporny]
Gregg Kellogg:  In that case, maybe term selection is not
   optional, but it is a feature that is controlled through an API
   flag. [scribe assist by Manu Sporny]
Niklas Lindström:  I think flags are good for optional features.
   It should be possible to query the implementation via a
   .hasFeature() method. [scribe assist by Manu Sporny]
Niklas Lindström:  flags are good for optional features. We
   should have a "has feature" method, so you can take appropriate
Manu Sporny: +1 to niklasl
Manu Sporny:  I agree with the general direction of the
   discussion. We're discussing two things:
   … 1) the compact call will have a mechanism to turn off or on
   different features.
   … 2) if a processor implements a feature, it must be done in
   an interoperable way.
   … This would indicate that compaction for property generators
   would not be part of the base API, but people could implement it
   by providing a feature driven by a flag.
Markus Lanthaler:  I'm wondering if, in this case, it might not
   be too difficult to support.
   … It's difficult at the moment to distinguish between prefixes
   and terms, because every term can be a prefix. You could then
   have a prefix which would expand compact IRIs.
   … if we limit other options on terms that are property
   generators, we could simplify compaction.
Manu Sporny:  this would modify the feature because of
   complexity, and give poor signals to developers.
Markus Lanthaler:  we currently silently ignore all other
   definitions for a prefix if it's also a term.
   … I ran across some issues where a term was used as a prefix,
   or the other way, and other attributes of the term were not used.
Niklas Lindström:  we could actually say that things define with
   an expanded term definition are never used as prefixes.
Markus Lanthaler:  then I would need to keep track what's a
   prefix or what's not.
Gregg Kellogg:  We might say that if it uses an expanded
   definition, it only uses the @id key - if that, then it can be
   used as a prefix. [scribe assist by Manu Sporny]
Gregg Kellogg:  A term can be used as a term if it matches.
   [scribe assist by Manu Sporny]
Manu Sporny:  If we require a : at the end of a prefix this
   creates a micro-syntax.
   … This would make it work differently in the body of a
   document. The way we have it now, you need to understand that
   these terms would not be used as prefixes, but that's a detail
   developers need to worry about.
   … If we have something with a ':' in it, the general case is
   that they would need to educate themselves about this.
   … If we require a ':', it could be error prone. A
   single-character mistake could mess them up.
   … I don't like this for those reasons, and I think developers
   would have a hard time learning about this.
Niklas Lindström: .. "foaf": {"@id": "…", "@prefix": true}
Gregg Kellogg:  What are the issues where this happens in common
   practice? The only time this becomes an issue is in something
   like an OWL definition. [scribe assist by Manu Sporny]
Gregg Kellogg:  There are very few cases here where this happens.
   [scribe assist by Manu Sporny]
Niklas Lindström:  there were some issue when I did some
   RDFa/JSON-LD conversion; it's not clear from the context if I can
   use them for prefixes.
Gregg Kellogg:  Maybe converting from RDFa to JSON-LD... [scribe
   assist by Manu Sporny]
Manu Sporny:  the issue was how we would support property
   generator in compaction.
Manu Sporny:  do we think that there may be optional features in
   … If there optional features, we are going to say if you
   implement it, it has conformance requirements to guarantee
   … If a feature is optional, it does not need to be
   implemented, but if you do, it must conform.
   … In practice, you know if the feature is there. It's only
   when you have a modular system that this is useful.
Markus Lanthaler:  In General, I don't like this, but there may
   be some cases where it's useful.
   … I think the cost to processor implementors is small compared
   to consumer's needs.
Gregg Kellogg:  There is a computational concern, if these
   features are enabled by default, it could really slow down your
   workflow. [scribe assist by Manu Sporny]
Gregg Kellogg:  I have to do a combinatorial search over those
   properties for things like property generators... that is a
   computational burden. [scribe assist by Manu Sporny]
Gregg Kellogg:  That's for somebody that doesn't even want
   property generator compaction. [scribe assist by Manu Sporny]
Gregg Kellogg:  Perhaps these are features where the defaults can
   be defined by the presence of the feature. [scribe assist by Manu
Manu Sporny:  the more I think about it, the more I'm concerned
   about the memory/computational overhead of compacting property
   … Drupal needs this to use core properties and be compatible
   with schema.org.
   … The decision lies with the authors. It's not import for us
   to decide.
   … If not you're okay with the computational issues, you can
   turn it on or off.
Gregg Kellogg:  As a consumer of data, I don't specify a
   context... I use what is served to me. [scribe assist by Manu
Markus Lanthaler:  Then why are we concerned about compaction if
   people might not use it? [scribe assist by Manu Sporny]
Gregg Kellogg:  Because people do use .compact() and we don't
   want the computational complexity in the default case. [scribe
   assist by Manu Sporny]
Niklas Lindström:  I was thinking that if drupal started using a
   lot of properties for a given term, it could be that the
   consumers will react to that, and think that it's just noise. I
   wouldn't expect more than 3-5.
   … If consumers want to consume such data, they can consume it
   incrementally, triple by triple.
Manu Sporny:  we're going to need to do some "least-resistance"
   proposal to make some progress.
Manu Sporny: Proposals on the table: 1) Support optional features
   in .compact(), 2) Make all features required in .compact()
Markus Lanthaler: we already support optional features - optimize
Manu Sporny:  if we support optional features, we're likely to
   support property generation is high. If all are required,
   property generation support is low.
Manu Sporny: +1 to 1), -1 to 2)
Gregg Kellogg: +1 for 1), -1 to 2)
François Daoust: +1 to 1), -1 to 2)
Markus Lanthaler: -1 to 1), +1 to 2) (except optimizations)
Niklas Lindström: +1 to 1), -0.5 to 2)
Dave Longley: following along in IRC ... keep in mind that those
   that want to use the feature (eg: drupal) might shy away from
   JSON-LD anyway if they think it's "optional" (the compaction side
   of it)
Dave Longley: it needs to be sold that a processor that they can
   use will implement it ... otherwise what good is it?
Dave Longley: that said, i'm ok with the optional path, just
   pointing out (if it hasn't been) a possible issue.

PROPOSAL:  Support optional features in .compact(). Any optional
   feature that is implemented, MUST be implemented in a specific
   way as outlined in the JSON-LD API.

Gregg Kellogg: +1
Manu Sporny: +1
Markus Lanthaler: -1
François Daoust: +1
Niklas Lindström: +1

RESOLUTION: Support optional features in .compact(). Any optional
   feature that is implemented, MUST be implemented in a specific
   way as outlined in the JSON-LD API.

Topic: Alternative to combinatorial .compact()

Dave Longley: maybe a better solution would be to mark which term
   you want to use in the compaction, in the @context, so the
   compaction algorithm can just look there and pick it.
Dave Longley: that's really what i think people would want anyway
   ... instead of having to think through the algorithm in their
Dave Longley: and guess which term will be selected based on
   whichever combination is most efficient.
Dave Longley: {"@context": {"foo": {"compact: true", ... (other
   stuff here) ... }}
Dave Longley: that would make compaction super simple ... and it
   would do what you want.
Manu Sporny:  you have a document with a property generator
   already in compact form.
   … When it is expanded, three properties are expanded. When
   compaction, they don't get coalesced.
Manu Sporny: Proposals for property generators and .compact(): 1)
   Support property generator compaction as an optional feature, 2)
   Do not support compaction of property generators at all, 3)
   Support property generator compaction as a required feature, 4)
   Compact only the first value in the enumeration of the array
Gregg Kellogg: +1 to 1), +0 to 2), -1 to 3), -1 to 4)
Dave Longley: {"@context": {"foo": {"compact: true", ... (other
   stuff here) ... }}
Dave Longley:  the other option is to be explicit in the context
   for what to compact.
Dave Longley: "foo": {"@id": "whatever", ... property generator
   ..., "compact": true}
Dave Longley:  the problem has always been that you need to
   understand the algorithm. This allows you to be more clear and
   not have deep understanding.
   … We could put the ranking algorithm into the context. You
   could be very explicit about what terms to pick.
Manu Sporny:  my concern is that it's pretty complex, in that it
   doesn't hide any of the complexity from the developer.
   … it's much more complicated than the other features.
Markus Lanthaler:  if you want to be specific, then you only need
   to specify one.
Dave Longley: "foo": {"@id": [any match in here], "compact":
Dave Longley: "foo": {"@id": [any match in here], "compact": 1}
Dave Longley: "foo": {"@id": [any match in here], "compact": 5}
Dave Longley:  one of the issues is knowing what term to pick.
Manu Sporny:  I thought the way the algorithm was supposed to
   work is that every id must exist with the same value. Then and
   only then do you use foo.
   … what we're trying to do here is say how property generators
   are round-tripped.
   … this is a new way to do compaction.
Manu Sporny:  I think dave's option is interesting, but it is
   tangentially related to property generator compaction.
Dave Longley:  do they really care about this?
   … I would think that, in the general case, I want to say that
   all these properties mean the same thing. If that's true, I don't
   think we need the extra complexity.
Dave Longley:  if the intention is to generate multiple
   properties when you see a single key, I don't see there's an
Manu Sporny:  it does mean that it doesn't cleanly round-trip.
Manu Sporny: +1 to 1), -1 to 2), -1 to 3), -1 to 4), +0 to 5)
Markus Lanthaler: +0.5 to 1), -1 to 2) +1 to 3), -1 to 4)
+ 0.5 to 5)
François Daoust: +1 to 1), -1 to 2), -1 to 3), -1 to 4)
François Daoust: [also like 5)]
Manu Sporny:  this could send mixed messages to the drupal
Niklas Lindström: +1 to 1), -0.5 to 2), -1 to 3), -1 to 4), +0 to
Dave Longley:  it's more like we're fighting for their feature.
   We should ask them how they feel about 5)
Markus Lanthaler: don't really understand 5) to be honest... the
   way I would implement this is to rank a term for each of it @id's
   in compaction and leave all the rest the same
Manu Sporny:  this takes us into entailment, and we should tread
Dave Longley: +1 to 1) -1 to 2) +0.5 to 3) +0 to 4) +1 to 5)
   … If we start to add ranking flags, we should be careful.
Manu Sporny:  1) seems to have the strongest support.
Dave Longley:  I'm worried that people may feel this is license
   to not implement.
Manu Sporny:  once people see an implementation (PHP), they'll be
   fine with it.
Markus Lanthaler:  I can try an implementation of 1) to see how
   easy it is
Dave Longley:  the only way for the algorithm to work is if you
   check both properties and values (for 1)
   … when you expand "foo", it might expand into 3 different
   ... However, if when you expand, you might have something that
Markus Lanthaler: "foo": [ "a", "b" ] - foo: data - expand - a:
   data, b: data - compact - foo: data, data ??? Remove duplicates,
   round-trip OK?
Dave Longley:  you could end up choosing the wrong term, and
   create more data.
   … multiple property generators are even worse.
   … you might then need the ranking algorithm.
Markus Lanthaler:  we already have that problem. if you have two
   properties that map to the same IRI, they might not compact back
   the same.
Manu Sporny:  maybe a callback could help.
Manu Sporny:  for the drupal case, they definitely want to
   generate multiple properties on expansion.
Manu Sporny:  we need to reach out to the drupal community about
   … I'll touch base with Lin and Stephane about this.

-- manu

Manu Sporny (skype: msporny, twitter: manusporny)
President/CEO - Digital Bazaar, Inc.
blog: Which is better - RDFa Lite or Microdata?

Received on Tuesday, 28 August 2012 00:59:57 UTC