Re: The Linked Data Event Streams specification

My bad, I was thinking about something else: of course, LDN allows any kind of resources to be LDN Notifications.

–Andrew

On 2021-03-15 , at 21:22, Andrii Berezovskyi <andriib@kth.se<mailto:andriib@kth.se>> wrote:

Dear Pieter,

I am sure you are aware of W3C LDN. You may also be interested to check out TRS, an OSLC spec (W3C LDP grew out of the early OSLC efforts):

https://archive.open-services.net/wiki/core/TrackedResourceSet-2.0/

https://oslc-op.github.io/oslc-specs/specs/trs/tracked-resource-set.html (v3.0 draft)

The TRS spec is essentially defining a paged set of "base" resources and a "truncateable" change log. TRS is used by tools like IBM Jazz and others to publish a stream of changes to the engineering artefacts (e.g. requirements, bug reports).

I had a brief look at LDES and it seems like a great effort beyond OSLC TRS and W3C LDN (if you persist the stream of LDN Notifications). Thank you for a great effort! I think the biggest difference is that both TRS and LDN only represent resource change events, referring the consumers to the resources themselves for the updated state while LDES goes further by allowing any resource to be part of the stream. On our part, the ability to include trs:Creation/Modification/Deletion events ([1] specifies their OSLC shapes, which was a precursor to SHACL in some sense [3]) should be enough to represent all TRS events in the LDES stream.

The only thing that may need special attention is the retention policy. TRS specifies a server-triggered rebase paired with the truncation of the log. As a result, the trs:cutoffEvent property gets updated to move ahead. I don't think either of the two currently defined LDES retention policies would allow to represent TRS log truncation faithfully.

I think it would be even more beneficial to allow LDN Senders to publish an LDES Stream of LDN Notifications for durability reasons.

Looking forward to see LDES to take off!

Best regards,
Andrew
--
Andrew Berezovskyi
KTH Royal Institute of Technology
OSLC PGB co-chair
Eclipse Lyo project lead

[1]: https://oslc-op.github.io/oslc-specs/specs/trs/trs-shapes.ttl

[2]: https://archive.open-services.net/wiki/core/TrackedResourceSet-2.0/#Truncated-Change-Logs

[3]: http://book.validatingrdf.com/bookHtml009.html#sec49


On 2021-03-15 , at 20:36, Pieter Colpaert <pieter.colpaert@ugent.be<mailto:pieter.colpaert@ugent.be>> wrote:

Dearest public LOD mailing list,

I was looking for a specification, a bit like RSS, that would allow me to tell potential data consumers that:

 - I’m maintaining a collection of members adhering to a certain shape,

 - that they they can find and download all of the members, and

 - that I want them to stay in sync with the latest changes.

As I did not find a solution in for example LDP, Hydra or Activity Streams 2, I made my own. You can find the Linked Data Event Streams (LDES) specification over here: https://w3id.org/ldes/specification. It does not do much: it just defined an event stream as a collection of an ever-growing collection of objects that never change (“version objects”).

It is based on the TREE specification (https://w3id.org/tree/specification) which in its turn allows more than simple one-dimensional paging. It allows to make your collection searchable using qualified links to following pages (or tree:Nodes). This means you can for example geospatially tile your LDES [1] or expose a file-based substring index [2]. Of course, you can also just load the LDES in a triple store and make sure your copy always stays in sync with the authoritative data source. TREE also foresees compatibility with hypermedia specs and collection designs in LDP, Hydra, Activity Streams 2, Shape Trees and Triple Pattern Fragments. Over the coming months we are planing to publish some governmental base registries with this specification.

Are there any other specs LDES should be compatible to?

Kind regards,

Pieter

P.S. Tomorrow I’m speaking at this online event called ENDORSE about this specification and the ideas behind it. More info: https://op.europa.eu/en/web/endorse/programme


[1] We last year published about a geospatial fragmentation in this paper: https://hdelva.be/articles/geospatial-linked-connections/


[2] Check out an autocompletion demo on top of a TREE collection fragmented by substring here: https://treecg.github.io/treemunica_typeahead_demo/ - I can send a preprint of a paper we wrote about this if interested.

--
https://pietercolpaert.be/#me