Re: Hydra and the uniform interface

On Dec 13, 2013, at 10:15 AM, Markus Lanthaler <markus.lanthaler@gmx.net> wrote:

> On Thursday, December 12, 2013 5:12 AM, Ryan J. McDonough wrote:
>> I’ve been trying to parse this thread for the past few hours and I
>> think I understand what Mark is getting at, and wanted to add my 2¢
>> to see if it helps any. 
> 
> Thank you very much for taking the time to do so Ryan. Much appreciated.
> 

You’re welcome :)

> 
>> On Dec 11, 2013, at 5:37 PM, Markus Lanthaler wrote:
>>> On Friday, December 06, 2013 9:59 PM, Mark Baker wrote:
>>> 
>>> On Thu, Dec 5, 2013 at 3:38 PM, Markus Lanthaler wrote:
>>> 
>>>>> You can either define them in a media type specification and expose
>>>>> them by typing the requests/responses with that media type, 
>>>>> as e.g. AtomPub does, or use some other mechanism. Hydra is an attempt
>>>>> to make these contracts machine-readable and discoverable at runtime.
>>>> 
>>>> I understand the objective, as I shared it when I developed RDF Forms
>>>> over 10 years ago. It too was type based like Hydra (something I've
>>>> mentioned I wouldn't do again). But it made no attempt to change the
>>>> HTTP contract, because it didn't need to.
>>> 
>>> I'm trying to understand why you think Hydra changes the HTTP contract
>>> but I'm afraid I still can't follow you. 
>> 
>> I think what Mark has been trying to point out is Hydra, in it’s current
>> form, suggests some level of coupling. Using an example from earlier:
>> 
>> {
>>    "@context": ...,
>>    "@id": "#order",
>>    "@type": "Link",
>>    "supportedOperations": [
>>        {
>>            "@type": [
>>                "schema:OrderAction",
>>                "Operation"
>>            ],
>>            "method": "POST",
>>            "expects": "schema:Product",
>>            "returns": "#Order"
>>        }
>>    ]
>> }
>> 
>> 
>> HTTP says that a successful POST operation could return a 200 or 201
>> response code that may include a response body. In this case, Hydra is
>> building on the HTTP response codes but also asserting that for this
>> particular URI, it’s also always going to return an #Order instance in the
>> response body.
> 
> Hmm.. no, not exactly. The spec says
> 
>    Optionally, it is also possible to describe what information the
>    server expects or returns, including additional information about HTTP
>    status codes that might be returned. This helps a developer to
>    understand what to expect when invoking an operation. This information
>    has, however, not to be considered as being complete; *it is merely a
>    hint*. Developers should, e.g., expect that other HTTP status codes
>    might be returned and program their clients accordingly.
> 
> and the definition of returns says
> 
>    hydra:returns
>    The information returned by the Web API on success
> 
> So it is really just a hint. Probably this has to be clarified or
> highlighted somehow in the spec. I've raised ISSUE-21 to track this.


My bad, I was mainly referring to the example and not the fact that the Hydra spec makes this optional. My trouble with hints like this is that people don’t always read specs and will assume return types are required, as opposed to hints, and will build clients that don't expect other response codes or entities with something other than an #Order.. Furthermore, this particular example flow a lot like a method call and fits nicely into code generators. I don’t know that I find returns to especially useful. But even so, I think the spec needs to be open to having more that one entity type on a success. In my example, I was getting that a successful response could be either an #Order or an #Invoice on success. 

The return type model also feels a bit limiting as well. While Hydra can provide a hint that a given operation might return a specific type, it also seems to assumed that I will always getting back a JSON-LD response. There really needs to more flexibility here as a service may not always return a JSON-LD response. Let’s say we have a LOL Cat picture service allows me to place text on an image. The POST operation would return an image. Hydra doesn’t appear to be able to describe non-JSON-LD responses very well. 

> 
> 
>> What I believe Mark is getting at is that the client can’t
>> reliably assume that a 200 (Ok) response could would always return an
>> #Order instance. At any time, the sever must be free to change the flow
>> and return an #Invoice instance instead.
> 
> Fully agreed.
> 
> 
>> Browsers work well because they
>> only know that the next link or form post will return “something."
>> Browsers don’t even assume that they’ll get back HTML either.
> 
> No, but humans controlling them have expectations about what they get back
> when they follow a link or fill out a form. If you break those too often,
> the user will give up and use a different site instead. In other words, you
> site's usability sucks in that case.

I think you’ve misunderstood what I was getting at. No doubt changing things too often will result in usability issue. That’s not what I am getting at. However, it’s not uncommon to follow a link and expected to see the article I wanted but was greeted with an Ad. Or more importantly, asking me to agree to new “terms of service” or a new privacy policy. Browsers don’t break when these changes occur. But users  do have to react to these changes. 

I know Stu chimed on on this thread a few weeks back, but I’d highly encourage you to listen to his WS-REST talk:

http://www.stucharlton.com/blog/archives/2011/03/ws-rest-keynote-slides-and-rec.html

The idea of sensors and state machines to reach the desired goal is a very powerful idea. This also gets around the option to specify return types. The agent would then be able to react to the response by sensing what it got back by looking at the Content-Type and optionally one or more link headers (maybe a profile link relation) to determine if they have ability to process this response. This a lot like how enterprise messaging systems work. You may only be able to handle certain types of messages and you have to determine if you can can process the message by looking at headers or message properties to determine if it’s worth parsing the body. I see a property hypermedia client working in a slightly similar way. 

> 
> 
>> The browser
>> knows what media types it can understand and determines if it can render
>> it by looking at the content-type header. If it can't display it, it asks
>> the user what to do with it. Each link does not scribe what’s on the other
>> end of the link, or what it might look like, its just a link.
> 
> It does, otherwise the user wouldn't be able to select the right link to
> follow. It is just that machines can't understand the labels or images used
> to convey that information.

The point I was getting is that a non-human agent can only be coded to things that it knows and understands. I think there’s little doubt that an automated agent would have to be coded with some upfront knowledge of the data model and some basic event types. This where I’m drawing the similarity; the agent would know the different types (i.e. #Order, #Product, etc.) before embarking on achieving some type of goal. With the understanding of what the different types mean, the agent can react to different responses. When you go there, you don’t really need to specify return types. Put it another way: I think the ideal way to approach this is to frame this in an event-based design rather a synchronous method call with a return type. 

Taking it up another level, I think another way to approach Hyrda would be to look what a an ideal Hydra Client would look like. There are too many “REST” or Hypermedia specs out there that define what things look like from the server side, but do very little to to describe how a client should consume this stuff. I think once we can define what the client programming model looks like, it would make the service description that much easier.

> 
>> This detail also invites itself to germinated client bindings that would
>> ultimately couple clients to specify server implementations.
> 
> Yeah, that's one of the things I'm most concerned with. I tried to address
> that by not including any URLs in a Hydra API description which allow to do
> that in a practical manner. If you need to find out the URLs at runtime
> anyway, it's that much more work to process the API description at the same
> time.


Well I’m glad you parse what “germinated client bindings” meant :) Stupid auto correct.  

> 
> 
>> WADL sets up
>> a nice environment for describing an XML IDL that can be used to generate
>> clients from. One of the things that a WADL Document would describe is
>> both what a given URI would return in a successful response as well as the
>> types of errors that the client could get.
> 
> Right, in WADL the coupling is, as in most current API libraries (and
> documentations for that matter), on the URL structure.


It’s not just URL structure, WADL also binds input and return types as well as response codes. To complicate matters, response messages were baked into the WADL descriptor rather than the  getting the message from the server. Even if you address URL bindings, fixed entities base on a descriptor also binds clients to specific workflows and properties and don’t get clients to to consider that they may get response entities with more information than they expected and the fact that may not be talking directly to the service they are connected to and might get responses from the intermediary. 

> 
> 
>> The generated clients only know
>> how to deal the error defined in the WADL document. In reality, a client
>> really ought to be able to handle all of the HTTP error code and redirect
>> codes. It’s the client that needs to handle the errors. It’s not the
>> servers job to define every error for every condition. The server needs to
>> be free to make changes as then need too.
> 
> Again, fully agreed. I probably did a bad job explaining that in the spec.
> 
> 
>> Mark, is this kind of where you were going?
> 
> Of course I'd like to know that as well :-)
> 
> 
> Thanks a lot again Ryan for trying to help here. At least we've found
> something that has to be made much clearer in the spec. I remember you were
> working on a little prototype. How is that going?


That’s been on hold for the moment. Day job getting in the way with all of the end of year crunch going on. Hopefully I’ll have some time over the holiday’s to get back on that.

> 
> 
> Cheers,
> Markus
> 
> 
> --
> Markus Lanthaler
> @markuslanthaler
> 
>