Re: Search Algorithms for Hydra

Hello,

I am quoting two previous messages I had sent here, according to the matter
of search and semantic discoverability.
I did implement a demo of semantic discoverability as part of my thesis
"Call by Context - Autodiscovery of Web services utilizing the Semantic
Web".
The demo app is semi documented on github [4].
A screencast demonstrating part of its functionalities [5].
Unfortunately my thesis is in Greek, if there is interest I could translate
parts of it.
Sorry for my huge and draft response but I am short in time.
I believe you will find this email interesting.

Regards,
Dimitris

Summary:

A network of servers offering Web services is implemented. Their content is
described with terms from the HYDRA vocabulary. The data model is built
upon schema.org terms rather than being simply annotated.
The generic client is able to understand those terms and communicate with
the servers with the aid of RDF graphs, instead of direct calls to their
URL.
In the middle of this communication lies the API-Resolver, a server
equipped with an RDF parser and a SPARQL endpoint aspiring to resolve and
match the requests from the client to the desired server.
The goal of the current thesis is twofold. The evaluation of this proof of
concept implementation and the exhibition of the Semantic Web potential.
However, only with its adoption in large scale, the automation of many
processes and the extension of the functionality of the current Web will
become feasible.


Message 1:
"""

I am trying to automate the process of API discovery using HYDRA +
schema.org.
The API provider describes it's capabilities using terms from those
vocabularies.
The API consumer describes the requested APIs using the same vocabularies
in a simple JSON-LD format.
There is a server between those two named API-Resolver, acting like a
search engine.
The queries and answers are in the form of RDF graphs. Behind the
API-Resolver, lies an instance of Jena/Fuseki as our SPARQL endpoint.
Each API provider is supposed to feed his jsonld complete description to
the API-Resolver in order to be discoverable.
The description of the server contains all the available classes and their
available methods, or better Actions in terms of schema.org.

The process so far is as follows:

1. The client describes the requested APIs with a simple graph
//part of clients description
    },
    "postalAdress": {
      "@type": "PostalAddress",
      "addressCountry": '',
      "addressLocality": ''
    },
    "airport": {
      "@type": "Airport",
      "iataCode": '',
      "address": { "@type": "PostalAddress" },
      "geo": {"@type": "GeoCoordinates"},
      "potentialAction": {
        "@type": "searchAction",
        "object": "schema:Airport",
        "result": "schema:Airport",
        "target": "schema:Airport",
        "query": {"@type": "schema:PostalAddress"}
      }
    },
//
2. Sends the request to API-Resolver who is already known.
3. API-Resolver creates the appropriate SPARQL queries for the triplestore.
//Sample
      DESCRIBE ?class ?target
      WHERE  {
        ?class rdf:type <http://schema.org/Airport>.
        ?server hydra:supportedClass ?class.
        ?server hydra:entrypoint ?entrypoint .
        ?class hydra:supportedProperty ?prop1 .
          ?prop1 hydra:property ?prop1_IRI .
          ?prop1_IRI rdf:type schema:address .
          ?prop1_IRI rdfs:range schema:PostalAddress .
          ?class hydra:supportedProperty ?prop2 .
          ?prop2 hydra:property ?prop2_IRI .
          ?prop2_IRI rdf:type schema:geo .
          ?prop2_IRI rdfs:range schema:GeoCoordinates .
          ?class hydra:supportedProperty ?prop3 .
          ?prop3 hydra:property ?prop3_IRI .
          ?prop3_IRI rdf:type schema:iataCode .
          ?action4_IRI schema:object ?class .
          ?action4_IRI schema:query ?query .
          ?query rdf:type ?queryClass .
          ?queryClass rdf:type schema:PostalAddress .
          ?action4_IRI schema:result ?result .
          ?result rdf:type <http://schema.org/Airport> .
          ?action4_IRI schema:target ?target .
          }
4. The API-Resolver reconstructs those graphs and responds to the client
with a single graph.
5. Client parses this graph and stores it locally using rdflib.js
6. Creates the appropriate classes based on the requested data and binds
the Actions to functions using a utility lib.
7. Inside our main client now, we are using something like this to search
the Airport Collection:

  var schydra = schydraAngular.init(rawJSON,resolverServer,graph);
  schydra.then(function(hydraClass){
    var searchAirport = new hydraClass['Airport'];
    searchAirport.label = 'Search Airport';
    searchAirport.set('iataCode','');
    searchAirport.set('iataCode','IATA code','label');
    searchAirport.set('address',searchAddress);
    searchAirport.set('geo',location);

   searchAddress.set('addressLocality',searchQuery);
   searchAirport.actions.searchAction(graph,'').then(function(response){ ...


The whole chain is already working in this simple manner but I can not
publish it just yet.
I tried to abstract the whole RDF thing from the developer's perspective.
The only thing he needs to know is the vocabulary he is about to use in
order to describe his data model.
What do you think about that?



"""

"""
I would like to share with you my diploma thesis, which utilized HYDRA. I
am sharing this to you, as you may find some interest and check out other
perspectives. The problem at hand was API automated discovery. The problem
is based on a paper named Call by Meaning [1].

I ended up with an implementation better named as Call by Context. I used
HYDRA to describe REST APIs and schema.org for the data and Actions
(methods).
API-Platform [2] instances offer services/data and a custom HYDRA/schema.org
client consumes them. In between something like a semantic search engine
(API-Resolver) matches desired APIs to available ones [3].

You can have a look at http://vps454845.ovh.net/schydra/playground.html .
I do know that my implementation suffers from bugs and some decisions I
made in order to proceed may not be the best. I would like to fix some bugs
before sharing the code.
"""

I had created a

Thanks,
Dimitris

[1] http://www.vpri.org/pdf/tr2014003_callbymeaning.pdf
[2] https://api-platform.com/
[3] https://drive.google.com/open?id=0BzKpYQSejhXxM2ljU0tfODFGZFk
[4] https://github.com/cr3a7ure/api-resolver
[5] https://www.youtube.com/watch?v=LxRaZSl3ANE

Στις Σάβ, 22 Φεβ 2020 στις 11:36 μ.μ., ο/η Karol Szczepański <
karol.szczepanski@gmail.com> έγραψε:

> Christian
>
> > This is the first response a possible client will get. Is this correct?
> > Or will the client directly ask the vocab for the semnatic API
> > description? If so: How does the Client know that the client speaks with
> > the right service? This is basically my research question, because I
> > want to make Hydra Services semantically discoverable.
>
> As Tomasz already stated - client can start in any place within the
> API, and each API powered by hydra should provide in each response an
> API documentation link which at least should provide an entrypoint for
> the service.
>
> Just to add to what Tomasz already wrote - having the RDF and Linked
> Data as the foundations of hydra, we can operate with hypermedia (RDF
> has some capabilities as it has a notion of a link, Hydra adds rest of
> those hypermedia capabilities).
> This is something that goes way farther that API documentation as a
> service can provide hypermedia controls within each service response
> so the client is guided from each state transition to another.
>
> > I don't expect an implementation from you guys, I just want to know if I
> > am on the right way and if hydra could suit my requirements.
>
> No worries - we just want to make sure hydra is at least enabled for
> some features we might not see clearly today.
>
> Regards
>
> Karol
>
> pt., 21 lut 2020 o 16:42 Tomasz Pluskiewicz <tomasz@t-code.pl> napisał(a):
> >
> > Hello Christian
> >
> > Extending what I wrote in my earlier email, while Hydra Core may not
> necessarily readily specify directly how you would define that information.
> >
> > However, there are definitely building blocks which allow it. Let me
> elaborate inline.
> >
> > Tom
> >
> > On 21 February 2020 at 12:36:55, Christian Rebischke (
> christian.rebischke@tu-clausthal.de) wrote:
> > > Hi Karol,
> > >
> > > Thanks for this explanation.
> > >
> > > What I am missing in the Hydra specification right now is basic meta
> > > data like I've pointed out already:
> > >
> > > * The name of the service
> > > * a short description in natural language
> >
> > These two should be simple. Hydra uses its hydra:label and
> hydra:description in other contexts.
> > Also, IIRC, analogous rdfs/schema terms are also explicitly allowed.
> > It would seem only logical to explicitly mention in the spec that the
> service as a whole can be annotates the same way.
> >
> > > * License being used
> >
> > This probably falls out of scope of the spec itself, which aims to
> mainly provide hypermedia control descriptions.
> > Nothing stops you from inspecting an API for cc:license [1] or
> schema:license [2].
> > Again, we might add a paragraph to suggest that, yet without a string
> mandate IMO.
> >
> > [1]: https://prefix.zazuko.com/cc:license
> > [2]: https://prefix.zazuko.com/schema:license
> >
> > > * Categories on which the Web Service operate (for example
> "Car-Manufacturing”)
> >
> > I find this one a little vague but if the purpose is essentially
> “tagging”, then again sounds like schema.org might define some useful
> terms, such as schema:category [3]
> >
> > [3]: https://prefix.zazuko.com/schema:category
> >
> > > * Preconditions and Effects (Does the Service depend on another
> Service?)
> >
> > This is turn is quite specific and at the same time more interesting.
> >
> > >
> > > For example the entrypoint of the event API demo:
> > > (http://www.markus-lanthaler.com/hydra/event-api/)
> > >
> > > ```
> > > {
> > > "@context": "/hydra/event-api/contexts/EntryPoint.jsonld",
> > > "@id": "/hydra/event-api/",
> > > "@type": "EntryPoint",
> > > "events": "/hydra/event-api/events/"
> > > }
> > > ```
> > >
> > > This is the first response a possible client will get. Is this correct?
> > > Or will the client directly ask the vocab for the semnatic API
> > > description? If so: How does the Client know that the client speaks
> with
> > > the right service? This is basically my research question, because I
> > > want to make Hydra Services semantically discoverable.
> >
> > Not necessarily "the first”. A client can begin from any resource URL
> (think browser bookmarks). The response should include a HTTP Link header
> with the hydra:apiDocumentation relation. This is where most of the
> service’s capabilities will be described.
> >
> > >
> > > As far as I understood you, you said that the client don't need to
> > > 'find' any service, because it's hardcoded and Hydra just provides a
> > > same vocabulary for decoupling.
> > >
> > > My question is: Wouldn't it be possible to 'add' a few meta tags to the
> > > entrypoint like this:
> > >
> > > ```
> > > {
> > > "@context": "/hydra/event-api/contexts/EntryPoint.jsonld",
> > > "@id": "/hydra/event-api/",
> > > "@type": "EntryPoint",
> > > "events": "/hydra/event-api/events/",
> > > "name": "Event-API",
> > > "description": "A simple example for a Hydra based event API",
> > > "license": "GPLv3"
> > > "categories": ["events", "Halloween"]
> > > ".... maybe a few more.. even with proper context"
> > > }
> > > ```
> > >
> > > I don't expect an implementation from you guys, I just want to know if
> I
> > > am on the right way and if hydra could suit my requirements.
> >
> > Definitely. As have shown, by using Linked Data technologies at its
> core, you are free to
> > extend your APIs with any non-standard metadata, which we may later
> standardise or recommend
> > if they prove useful and generic enough.
> >
> > I think it would be very useful if your research results in an analysis
> of possible RDF terminology (vocabularies/ontologies) which enable the
> goals you describe and we could take it from there.
> >
> > That said, standardisation is hardly necessary on a global level if one
> intends to operate in a closed environment,
> > such as within an organization.
> >
> > >
> > > christian
> > >
> > >
> > >
> > >
> > > On Thu, Feb 20, 2020 at 10:44:25PM +0100, Karol Szczepański wrote:
> > > > Hi Christian
> > > >
> > > > In short, hydra is supposed to let both client and server talk using
> > > > same protocol expanded with some common vocabulary so the server can
> > > > drive the client resulting in client being smart and decoupled.
> > > > How the server can drive a client depends on that client - it will be
> > > > something different for an automated one (i.e. some details allowing
> > > > automatic decision making) and something different for human operated
> > > > app (i.e. what can be done and how it should be displayed).
> > > >
> > > > Having this in mind, while search algorithm may not fall into that
> > > > circle of interest, "meta service description" may fit.
> > > > Hydra is a meta service description to some degree - hypermedia
> > > > controls introduced by hydra vocabulary are additional pieces of
> > > > information interleaved with raw data that can be used by the clients
> > > > to interact with the data and the server.
> > > >
> > > > Feel free to deliberate more on your ideas - maybe there is something
> > > > we could make hydra better.
> > > > In worst case scenario hydra may introduce some "plugs" that could
> get
> > > > that vocabulary opened for some extensions bringing features you
> > > > desire.
> > > >
> > > > Best regards
> > > >
> > > > Karol
> > > >
> > > > czw., 20 lut 2020 o 20:58 Christian Rebischke
> > > > napisał(a):
> > > > >
> > > > > Hello Tomasz,
> > > > >
> > > > > thank you for your mail. Let me explain you my current work:
> > > > >
> > > > > I am working in a small research group that want to enable
> developers to
> > > > > upload semantic described Web services for emergent genesis.
> > > > >
> > > > > The goal is that a user can describe their requirements and the
> > > > > plattform will semantically search for different services that
> meet this
> > > > > requirements and compose them to an emergent service.
> > > > >
> > > > > We thought that Hydra is might a good choice for solving this
> problem.
> > > > > At least the part about semantic input/output and API description.
> > > > >
> > > > > I am the person who is responsible for service discovery, hence I
> had a
> > > > > look on different semantic Web service descriptions like OWL-S,
> WSMO,
> > > > > Hydra, WebAPI on schema.org and many others.
> > > > >
> > > > > OWL-S seem to have anything what we would need, but it's old and
> > > > > therefore there are no existing libraries for it anymore. That's
> why I
> > > > > thought Hydra is might a good choice for the project.
> > > > >
> > > > > Yesterday I had a proper look on the API description and the Web
> Service
> > > > > Vocabulary. The input and output API description looks fine for
> me, the
> > > > > only thing what is missing is meta data for the service like
> "Name",
> > > > > "Description", "License", maybe some sort of "Preconditions" or
> > > > > "Effects".
> > > > >
> > > > > What are "preconditions" and "effects"? This is a concept from
> OWL-S.
> > > > > It describes adjustments or changes in the physical world. For
> example,
> > > > > when you trigger a smarthome web service and the smarthome web
> service
> > > > > is going to adjust the temperature in a room, this might be an
> effect.
> > > > > Another service can have as precondition, that the room
> temperature is
> > > > > not too low, this would mean that I would need a sensor service in
> my
> > > > > "Service composition" for measuring the room temperature first.
> > > > >
> > > > >
> > > > > So much about my project. My next question is:
> > > > >
> > > > > Is this actual from interest for this group? So, are you guys
> interested
> > > > > in some sort of search algorithm or meta service description for
> hydra
> > > > > or is this out of scope of Hydra?
> > > > >
> > > > > Best regards,
> > > > >
> > > > > Christian
> > > > >
> > > > >
> > > > > On Wed, Feb 19, 2020 at 07:28:49PM +0100, Tomasz Pluskiewicz wrote:
> > > > > > Hello Christian
> > > > > >
> > > > > > That is a very interesting question and indeed a subject which
> comes back occasionally
> > > in the context of Hydra and other API specifications.
> > > > > >
> > > > > > At the moment there does not appear to be a ready
> one-size-fits-all solution which
> > > would discover Hydra that way. I cannot really comment on the linked
> package though.
> > > > > > Hydra itself is probably not expressive enough to facilitate
> that.
> > > > > >
> > > > > > However, because about Hydra is built on top of RDF and Linked
> Data technologies,
> > > it would be fairly easy to start implementing a custom extension to
> allow such discovery.
> > > > > >
> > > > > > Please let us know a little more about your work:
> > > > > >
> > > > > > 1. What kinds of requirements would the API expose, in human
> terms perhaps
> > > > > > 2. Would those be public APIs or a closed environment (such as
> multiple APIs within
> > > an organization)
> > > > > >
> > > > > > Best,
> > > > > > Tom
> > > > > >
> > > > > > On 19 February 2020 at 14:32:56, Christian Rebischke (
> christian.rebischke@tu-clausthal.de)
> > > wrote:
> > > > > > > Hi,
> > > > > > >
> > > > > > > is there any library/client for searching for hydra web
> services and
> > > > > > > matching them against a fixed set of requirements?
> > > > > > >
> > > > > > > Imagine the following:
> > > > > > >
> > > > > > > I have a fixed set of requirements for a Web Service (either
> described
> > > > > > > semantically or natural language or whatever) and i want to
> find a
> > > > > > > matching service for this requirement.
> > > > > > >
> > > > > > > Is there something for hydra?
> > > > > > >
> > > > > > > What I've found so far is an `api-doc-parser`[1]. But i am not
> sure if
> > > > > > > this is enough for finding a matching hydra service.
> > > > > > >
> > > > > > > Grüße
> > > > > > > Christian
> > > > > > >
> > > > > > > [1] https://github.com/dunglas/api-doc-parser
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> > >
> >
>
>