- From: Manu Sporny <msporny@digitalbazaar.com>
- Date: Mon, 27 Aug 2012 20:59:23 -0400
- To: Linked JSON <public-linked-json@w3.org>
- CC: RDF WG <public-rdf-wg@w3.org>
The minutes from last week's call are now available here:
http://json-ld.org/minutes/2012-08-21/
Full text of the discussion follows including a link to the audio
transcript:
--------------------
JSON-LD Community Group Telecon Minutes for 2012-08-21
Agenda:
http://lists.w3.org/Archives/Public/public-linked-json/2012Aug/0018.html
Topics:
1. DrupalCon discussion on JSON-LD
2. Property Generators and .compact() API
3. Alternative to combinatorial .compact()
Resolutions:
1. The group is committed to support language maps and
property generators in JSON-LD 1.0.
2. Support optional features in .compact(). Any optional
feature that is implemented, MUST be implemented in a specific
way as outlined in the JSON-LD API.
Chair:
Manu Sporny
Scribe:
Gregg Kellogg
Present:
Gregg Kellogg, Manu Sporny, Markus Lanthaler, Niklas Lindström,
François Daoust, Dave Longley
Audio:
http://json-ld.org/minutes/2012-08-21/audio.ogg
Gregg Kellogg is scribing.
http://greggkellogg.net/2012/08/21/json-ld-and-mongodb
Gregg describes talk he's giving at NoSQL Now!, slides are above,
good feedback from folks.
Topic: DrupalCon discussion on JSON-LD
Manu Sporny: http://groups.drupal.org/node/249128
Manu Sporny: Lin has asked what JSON-LD features should be used
in Drupal.
… Two issues that were important have been re-opened: property
generators and language map.
… When they were re-opened they became nervous about stability
of JSON-LD.
… We need to discuss why they were re-opened, and what to do
going forward.
… The message is that the features are unstable, and could go
away, which is not true.
Markus Lanthaler: The main reason they were opened was because
the algorithms haven't been updated, and we haven't discussed how
compaction works.
… In the property generators issue, there was a comment that
this was supported, but the array-position wasn't.
… I thought we might want to discuss the array-position issue
again.
… The main reason was to make sure that the algorithms are
updated before the issue is closed.
Markus Lanthaler: I had some question about our support for
property generators.
Manu Sporny: Perhaps we should discuss how compaction should
work.
… the main thing is to signal to Drupal that we're not pulling
the rug out from under them.
… We re-opened to make sure that all the work was completed,
not that the feature was in doubt.
Markus Lanthaler:
https://github.com/json-ld/json-ld.org/issues/142#issuecomment-7866816
Manu Sporny: It's not just for drupal, we have a similar
mechanism in RDFa, and other companies have indicated their need
for it too.
… It's not just for drupal, but they did provide motivation.
Niklas Lindström: I understand that the mutliple-property
feature could be seen as "un-pure" from a theoretical
perspective. You should rely on inference and other things, but
those are all ideals.
… This case supports a fairly easy way for both producers and
consumers to manage such properties.
… The alternative would be to create different documents for
different consumers, which isn't very useful.
… It's un-pure, but useful.
Manu Sporny: I don't hear any objection to keeping the feature,
we just need to work out details, specifically in compaction.
Gregg Kellogg: We may want to say right now that it's not
addressed in compaction, it would allow us to close out this
particular feature. [scribe assist by Manu Sporny]
Gregg Kellogg: We could say that if we /do not/ describe
anything happening in compaction, then we may not do anything.
[scribe assist by Manu Sporny]
Gregg Kellogg: The algorithm for compaction would be a bit
different... more heavyweight. [scribe assist by Manu Sporny]
Gregg Kellogg: I don't think we want to wrap these things
together, we don't want to message that just because we haven't
wrapped some of the API details up, that we're not going to
support the feature. [scribe assist by Manu Sporny]
Niklas Lindström: +1 for that
Manu Sporny: we do a proposal that says "support both
property-generators and language-maps; close issues; create new
issues for compaction implications".
Markus Lanthaler: you can't make use of these things without the
API, so we need all parts.
Gregg Kellogg: I think we need to do the expansion... no
controversy there, the issue comes in via compaction and in via
framing... certainly, there is some open work to be done in the
expansion algorithms. It's reasonable to link them to the
existing issues. [scribe assist by Manu Sporny]
Gregg Kellogg: Our practice has been to keep issues open until
the language has been updated to reflect what's done. It's a
book-keeping exercise. [scribe assist by Manu Sporny]
Manu Sporny: we need to do something to tell them that there is
a clear effort to support these features.
… The proposal is to underscore that both property generators
and language maps are supported features, and the group is
working out the details of how the API supports them.
Manu Sporny: we could post on drupal that we're supporting it,
the issue is resolved, but the work's not all done.
Manu Sporny: the reason we need to be more careful on how we use
the issue tracker is because the the perception on outside
groups.
PROPOSAL: The group is committed to support language maps and
property generators in JSON-LD 1.0.
Gregg Kellogg: +1
Manu Sporny: +1
Niklas Lindström: +1
Markus Lanthaler: +1
François Daoust: +1
RESOLUTION: The group is committed to support language maps and
property generators in JSON-LD 1.0.
Topic: Property Generators and .compact() API
Manu Sporny: Markus' point is that we've typically dealt with
things all in one issue, why is this different.
Gregg Kellogg: I don't know where other people are wrt to this.
The .compact() algorithm is getting to be almost unimplementably
complex. [scribe assist by Manu Sporny]
Gregg Kellogg: It becomes almost unacceptable in an algorithm,
when in-fact, most people typically generate JSON-LD through
their own customized mechanisms. I do make use of compaction, but
mostly as a way of doing fairly standard normalizations. [scribe
assist by Manu Sporny]
Gregg Kellogg: I usually generate properties as arrays - so for
example, dropping single value arrays. Making use of CURIEs, or
other terms. I find compaction() useful for that stuff. I don't
find .frame() useful at all. I usually do in-memory linking.
[scribe assist by Manu Sporny]
Gregg Kellogg: It would be too much work to get the API to do
what I wanted... noticed this in .frame(). If it doesn't work, it
doesn't work and there is no insight into why. Very hard to debug
frames. [scribe assist by Manu Sporny]
Gregg Kellogg: I think these things make it fairly useless,
adding more complexity to .compact() is not something I'd like to
do. [scribe assist by Manu Sporny]
Gregg Kellogg: Which term is selected for a specific property is
one of those things. [scribe assist by Manu Sporny]
Gregg Kellogg: The ranking algorithm... the very act of
splitting properties among multiple terms, is strange. [scribe
assist by Manu Sporny]
Gregg Kellogg: if you have a property that has an equivalent
IRI, the datatypes are split among those terms.... if you are
trying to be entirely symmetric between compaction and expansion.
It leads to unexpected results. [scribe assist by Manu Sporny]
Manu Sporny: At a high level, I agree on the complexity
argument. That's one of the reasons in the beginning we said that
when we talk about compaction, we leave a lot of the magic up the
the implementation.
… Maybe we should define a core-set of interoperability that
you should expect from compaction, but it leaves serious concerns
about interoperability.
Manu Sporny: 2 questions: 1) do we expect compaction to be
interoperable. 2) were do we draw the line on interoperability?
… property generators are a case where we may not want the
algorithm to do anything when it goes to compact those values.
Niklas Lindström: I also agree on a high level, especially
w.r.t. property generators.
… I do see the need for ranking, and tend to think that that
should be interoperable.
… The mapping from RDF to elastic search indexing of data. I'd
like that to be as automatic as possible.
… From my point of view, although I agree we should debate, I
feel we should draw the line at where you cannot determine what
term is used from the value alone.
… That rules out property generators. Does not rule out
ranking, and might not language maps, but I agree that it's on
the edge.
Manu Sporny: would we be opposed to some implementations
implementing it, and others not.
Gregg Kellogg: In RDFa, we have examples of optional features -
implementing the feature is not required, but if it is required,
it has interoperability requirements. Processor graphs is one
such example. [scribe assist by Manu Sporny]
Gregg Kellogg: Maybe we have a flag that allows best-effort
compaction... maybe we can have a set of parameters that are
associated with these optional features. Processors can declare
whether they are supported or not. They're supported if the
appropriate options are passed to the algorithm. It also makes
testing easier. It can also reduce the computational complexity
to support odd cases that are more likely to never come about.
[scribe assist by Manu Sporny]
Gregg Kellogg: In that case, maybe term selection is not
optional, but it is a feature that is controlled through an API
flag. [scribe assist by Manu Sporny]
Niklas Lindström: I think flags are good for optional features.
It should be possible to query the implementation via a
.hasFeature() method. [scribe assist by Manu Sporny]
Niklas Lindström: flags are good for optional features. We
should have a "has feature" method, so you can take appropriate
measure.
Manu Sporny: +1 to niklasl
Manu Sporny: I agree with the general direction of the
discussion. We're discussing two things:
… 1) the compact call will have a mechanism to turn off or on
different features.
… 2) if a processor implements a feature, it must be done in
an interoperable way.
… This would indicate that compaction for property generators
would not be part of the base API, but people could implement it
by providing a feature driven by a flag.
Markus Lanthaler: I'm wondering if, in this case, it might not
be too difficult to support.
… It's difficult at the moment to distinguish between prefixes
and terms, because every term can be a prefix. You could then
have a prefix which would expand compact IRIs.
… if we limit other options on terms that are property
generators, we could simplify compaction.
Manu Sporny: this would modify the feature because of
complexity, and give poor signals to developers.
Markus Lanthaler: we currently silently ignore all other
definitions for a prefix if it's also a term.
… I ran across some issues where a term was used as a prefix,
or the other way, and other attributes of the term were not used.
Niklas Lindström: we could actually say that things define with
an expanded term definition are never used as prefixes.
Markus Lanthaler: then I would need to keep track what's a
prefix or what's not.
Gregg Kellogg: We might say that if it uses an expanded
definition, it only uses the @id key - if that, then it can be
used as a prefix. [scribe assist by Manu Sporny]
Gregg Kellogg: A term can be used as a term if it matches.
[scribe assist by Manu Sporny]
Manu Sporny: If we require a : at the end of a prefix this
creates a micro-syntax.
… This would make it work differently in the body of a
document. The way we have it now, you need to understand that
these terms would not be used as prefixes, but that's a detail
developers need to worry about.
… If we have something with a ':' in it, the general case is
that they would need to educate themselves about this.
… If we require a ':', it could be error prone. A
single-character mistake could mess them up.
… I don't like this for those reasons, and I think developers
would have a hard time learning about this.
Niklas Lindström: .. "foaf": {"@id": "…", "@prefix": true}
Gregg Kellogg: What are the issues where this happens in common
practice? The only time this becomes an issue is in something
like an OWL definition. [scribe assist by Manu Sporny]
Gregg Kellogg: There are very few cases here where this happens.
[scribe assist by Manu Sporny]
Niklas Lindström: there were some issue when I did some
RDFa/JSON-LD conversion; it's not clear from the context if I can
use them for prefixes.
Gregg Kellogg: Maybe converting from RDFa to JSON-LD... [scribe
assist by Manu Sporny]
Manu Sporny: the issue was how we would support property
generator in compaction.
Manu Sporny: do we think that there may be optional features in
compaction?
… If there optional features, we are going to say if you
implement it, it has conformance requirements to guarantee
interoperability.
… If a feature is optional, it does not need to be
implemented, but if you do, it must conform.
… In practice, you know if the feature is there. It's only
when you have a modular system that this is useful.
Markus Lanthaler: In General, I don't like this, but there may
be some cases where it's useful.
… I think the cost to processor implementors is small compared
to consumer's needs.
Gregg Kellogg: There is a computational concern, if these
features are enabled by default, it could really slow down your
workflow. [scribe assist by Manu Sporny]
Gregg Kellogg: I have to do a combinatorial search over those
properties for things like property generators... that is a
computational burden. [scribe assist by Manu Sporny]
Gregg Kellogg: That's for somebody that doesn't even want
property generator compaction. [scribe assist by Manu Sporny]
Gregg Kellogg: Perhaps these are features where the defaults can
be defined by the presence of the feature. [scribe assist by Manu
Sporny]
Manu Sporny: the more I think about it, the more I'm concerned
about the memory/computational overhead of compacting property
generators.
… Drupal needs this to use core properties and be compatible
with schema.org.
… The decision lies with the authors. It's not import for us
to decide.
… If not you're okay with the computational issues, you can
turn it on or off.
Gregg Kellogg: As a consumer of data, I don't specify a
context... I use what is served to me. [scribe assist by Manu
Sporny]
Markus Lanthaler: Then why are we concerned about compaction if
people might not use it? [scribe assist by Manu Sporny]
Gregg Kellogg: Because people do use .compact() and we don't
want the computational complexity in the default case. [scribe
assist by Manu Sporny]
Niklas Lindström: I was thinking that if drupal started using a
lot of properties for a given term, it could be that the
consumers will react to that, and think that it's just noise. I
wouldn't expect more than 3-5.
… If consumers want to consume such data, they can consume it
incrementally, triple by triple.
Manu Sporny: we're going to need to do some "least-resistance"
proposal to make some progress.
Manu Sporny: Proposals on the table: 1) Support optional features
in .compact(), 2) Make all features required in .compact()
Markus Lanthaler: we already support optional features - optimize
Manu Sporny: if we support optional features, we're likely to
support property generation is high. If all are required,
property generation support is low.
Manu Sporny: +1 to 1), -1 to 2)
Gregg Kellogg: +1 for 1), -1 to 2)
François Daoust: +1 to 1), -1 to 2)
Markus Lanthaler: -1 to 1), +1 to 2) (except optimizations)
Niklas Lindström: +1 to 1), -0.5 to 2)
Dave Longley: following along in IRC ... keep in mind that those
that want to use the feature (eg: drupal) might shy away from
JSON-LD anyway if they think it's "optional" (the compaction side
of it)
Dave Longley: it needs to be sold that a processor that they can
use will implement it ... otherwise what good is it?
Dave Longley: that said, i'm ok with the optional path, just
pointing out (if it hasn't been) a possible issue.
PROPOSAL: Support optional features in .compact(). Any optional
feature that is implemented, MUST be implemented in a specific
way as outlined in the JSON-LD API.
Gregg Kellogg: +1
Manu Sporny: +1
Markus Lanthaler: -1
François Daoust: +1
Niklas Lindström: +1
RESOLUTION: Support optional features in .compact(). Any optional
feature that is implemented, MUST be implemented in a specific
way as outlined in the JSON-LD API.
Topic: Alternative to combinatorial .compact()
Dave Longley: maybe a better solution would be to mark which term
you want to use in the compaction, in the @context, so the
compaction algorithm can just look there and pick it.
Dave Longley: that's really what i think people would want anyway
... instead of having to think through the algorithm in their
minds
Dave Longley: and guess which term will be selected based on
whichever combination is most efficient.
Dave Longley: {"@context": {"foo": {"compact: true", ... (other
stuff here) ... }}
Dave Longley: that would make compaction super simple ... and it
would do what you want.
Manu Sporny: you have a document with a property generator
already in compact form.
… When it is expanded, three properties are expanded. When
compaction, they don't get coalesced.
Manu Sporny: Proposals for property generators and .compact(): 1)
Support property generator compaction as an optional feature, 2)
Do not support compaction of property generators at all, 3)
Support property generator compaction as a required feature, 4)
Compact only the first value in the enumeration of the array
Gregg Kellogg: +1 to 1), +0 to 2), -1 to 3), -1 to 4)
Dave Longley: {"@context": {"foo": {"compact: true", ... (other
stuff here) ... }}
Dave Longley: the other option is to be explicit in the context
for what to compact.
Dave Longley: "foo": {"@id": "whatever", ... property generator
..., "compact": true}
Dave Longley: the problem has always been that you need to
understand the algorithm. This allows you to be more clear and
not have deep understanding.
… We could put the ranking algorithm into the context. You
could be very explicit about what terms to pick.
Manu Sporny: my concern is that it's pretty complex, in that it
doesn't hide any of the complexity from the developer.
… it's much more complicated than the other features.
Markus Lanthaler: if you want to be specific, then you only need
to specify one.
Dave Longley: "foo": {"@id": [any match in here], "compact":
true}
Dave Longley: "foo": {"@id": [any match in here], "compact": 1}
Dave Longley: "foo": {"@id": [any match in here], "compact": 5}
Dave Longley: one of the issues is knowing what term to pick.
Manu Sporny: I thought the way the algorithm was supposed to
work is that every id must exist with the same value. Then and
only then do you use foo.
… what we're trying to do here is say how property generators
are round-tripped.
… this is a new way to do compaction.
Manu Sporny: I think dave's option is interesting, but it is
tangentially related to property generator compaction.
Dave Longley: do they really care about this?
… I would think that, in the general case, I want to say that
all these properties mean the same thing. If that's true, I don't
think we need the extra complexity.
Dave Longley: if the intention is to generate multiple
properties when you see a single key, I don't see there's an
issue.
Manu Sporny: it does mean that it doesn't cleanly round-trip.
Manu Sporny: +1 to 1), -1 to 2), -1 to 3), -1 to 4), +0 to 5)
Markus Lanthaler: +0.5 to 1), -1 to 2) +1 to 3), -1 to 4)
+ 0.5 to 5)
François Daoust: +1 to 1), -1 to 2), -1 to 3), -1 to 4)
François Daoust: [also like 5)]
Manu Sporny: this could send mixed messages to the drupal
community.
Niklas Lindström: +1 to 1), -0.5 to 2), -1 to 3), -1 to 4), +0 to
5)
Dave Longley: it's more like we're fighting for their feature.
We should ask them how they feel about 5)
Markus Lanthaler: don't really understand 5) to be honest... the
way I would implement this is to rank a term for each of it @id's
in compaction and leave all the rest the same
Manu Sporny: this takes us into entailment, and we should tread
carefully.
Dave Longley: +1 to 1) -1 to 2) +0.5 to 3) +0 to 4) +1 to 5)
… If we start to add ranking flags, we should be careful.
Manu Sporny: 1) seems to have the strongest support.
Dave Longley: I'm worried that people may feel this is license
to not implement.
Manu Sporny: once people see an implementation (PHP), they'll be
fine with it.
Markus Lanthaler: I can try an implementation of 1) to see how
easy it is
Dave Longley: the only way for the algorithm to work is if you
check both properties and values (for 1)
… when you expand "foo", it might expand into 3 different
IRIs.
... However, if when you expand, you might have something that
maps.
Markus Lanthaler: "foo": [ "a", "b" ] - foo: data - expand - a:
data, b: data - compact - foo: data, data ??? Remove duplicates,
round-trip OK?
Dave Longley: you could end up choosing the wrong term, and
create more data.
… multiple property generators are even worse.
… you might then need the ranking algorithm.
Markus Lanthaler: we already have that problem. if you have two
properties that map to the same IRI, they might not compact back
the same.
Manu Sporny: maybe a callback could help.
Manu Sporny: for the drupal case, they definitely want to
generate multiple properties on expansion.
Manu Sporny: we need to reach out to the drupal community about
this.
… I'll touch base with Lin and Stephane about this.
-- manu
--
Manu Sporny (skype: msporny, twitter: manusporny)
President/CEO - Digital Bazaar, Inc.
blog: Which is better - RDFa Lite or Microdata?
http://manu.sporny.org/2012/mythical-differences/
Received on Tuesday, 28 August 2012 01:00:00 UTC