Re: Compacting on ActivityStreams receive in the API?

☮ elf Pavlik ☮ writes:

> On 11/23/2014 09:18 PM, Christopher Allan Webber wrote:
>> Hello,
> Hi Christopher o/
>
>> 
>> I'm working on a minimal experiment that I'm hoping to use to track the
>> standards as we develop them in this working group (it's currently a toy
>> rather than a real implementation... I'm not sure that it'll be a fully
>> useful implementation yet, but it'll be nice if it will be).
> Awesome! How do you plan to persist data? (relational | document |
> key-value | graph | ... )

Well, here's how it is at present:

PostgreSQL has query'able, index'able JSON support in their latest
release that rivals or maybe even beats out projects like MongoDB
(nicely structured in the database, not just a text blob... queryable
across multiple tables).  It's pretty neat!

  http://clarkdave.net/2013/06/what-can-you-do-with-postgresql-and-json/

So I'm using that + sqlalchemy + aiopg to mostly store activities as
big ol json dumps where important fields are indexed.  Right now it's
just an activities table... I figure if I implement users, etc, maybe
that'll be more "normal SQL", but I'm not sure.

>> This lead me to hit a point we've discussed, but I don't think have
>> specified for sure (?)... what's the plan around compacting incoming
>> data?  It's reasonable to assume that we should compact any json-ld
>> before sending across the wire in the federation and social APIs... but
>> what about incoming data in the social API?  Should we "just assume"
>> it's already compacted, or should we run it through the compacting tool
>> anyway to enforce that it is?
> To my understanding we don't want to require everyone to use JSON-LD
> processing algorithms[1]. If some system treats AS2.0 as regular JSON,
> it will pretty much always work with representation equivalent to
> JSON-LD /compacted/ form.

So it definitely seems (as I figured) that we'd go with compacted by
default... and that this would be mandatory (yes?) for whatever APIs we
expect to implement?  Any json-ld encoded activitystreams across the
wire *must* be compacted then?  The reason I'm asking is: I should never
have to compact incoming data, because I should already expect that it
is compacted, because that's what our API will likely mandate?

> On the other hand, systems understanding RDF will often work with
> other JSON-LD profiles[2]. For example, myself I persist all the data
> in a triple store, later when serializing certain subset of those
> triples to JSON-LD I need to output it in /compacted/ form. This way
> other systems, which once again don't have to support JSON-LD
> algorithms, can expect it in a form which they can work with.

In the land of everything being in the platonic ideal of data, RDF /
triple stores, I suppose it doesn't matter; the above is as complex as
you need to know, and then you're done.  You query around in your graph,
and your graph is itself the ideal representation, and export to json-ld
again when you need to communicate with the outside world.

I'm not doing that. :)  I think realistically whatever spec we put
together is going to be used by people storing things in document stores
and relational databases also, especially since we've been targeting the
"RDF is not mandatory", so I want to think about how to handle that.

> While you can work with AS2.0 ignoring JSON-LD processing, I would
> encourage you to still give a shot to some of the libraries listed on
> http://json-ld.org/
>
> HTH

I've used json-ld before, and I'm using pyld here, though it's still
somewhat new to me.

I'm still a bit unsure about at what times when people give extended
data beyond the default activitystreams stuff when we'll want to expand
the document, or do any validation.  I guess that's something that will
become clearer as I do implementation.

 - Chris

Received on Monday, 24 November 2014 02:04:52 UTC