Re: ISSUE-31: Are Operations violating REST's uniform interface constraint?

On Feb 17, 2014, at 3:03 PM, Markus Lanthaler <markus.lanthaler@gmx.net> wrote:

> On Sunday, February 16, 2014 8:14 PM, Ryan J. McDonough wrote:
>> On Feb 16, 2014, at 11:27 AM, Thomas Hoppe wrote:
>> 
>>>> The @type of "Search" offers less value than the resource defined in
>> the "expects" property. It's here when're it's telling me how to format
>> the message to the server. The operation subclass doesn't do much in
>> helping me achieve that. The majority of request formatting is
>> expressed in #SearchRequest and not the #Search @type.
>>> 
>>> Hmm, for generic modelling of operations (or say forms because that
>> would be the analogy in HTML)
>>> I think you are right -- in that case I don't care about the @type.
>>> But what If a client is specifically interested in a search operation
>> of a Web resource.
>> 
>> I still believe that client doesn't gain much with the @type of the
>> operation. What the client must care about is how to prepare the
>> request and deliver it over the correct HTTP method. The operation is
>> being carried out by the information in the search request message, not
>> the annotations around it.
> 
> OK. Let's try to use a more interesting example than search. Let's talk
> about ordering goods. So, you are arguing that the client would need to send
> a do-order-this-stuff message, right?

Exactly. The idea being that the request message is clear about the users intent and that the client does hold any information that the server can’t see. 

> Likely, the server will transform that
> to an Order. So, the question is, can I reuse a Order (let's say, e.g.,
> http://schema.org/Order) to order some stuff or do I need to design a
> separate do-order-this-stuff message type?

I think it depends. The schema.org example actually represents the receipt, or the end state of an activity of making a purchase. This would be more applicable to a response type after executing the order. But that aside, a shared data model like schema.org or Good Relations would serve as a foundational piece rather but may not be usable without some extension to a base model.

Even a company like Demandware can’t really get every catalog and checkout forms to be consistent across all of their customers. This has little to do with Demandware’s excellent platform, but has everything to do with the fact that each customer application has different requirements and interacts with different backend systems (i.e. payment systems, address verification systems, etc..). A 100% consistent data model between different storefronts is a tall order. In some cases, it might work but I think for 98% of the use cases, you’re not going to be able to reuse a vanilla http://schema.org/Order instance out of the box.


> This now goes back to something I
> discussed with Mark a while ago, but if the server I'm talking to doesn't
> deliver any goods but just stores files. Do I need another
> store-this-order-representation message type?

I’d approach it more like this: you have a catalog of things you browse and search through. I may be able to use http://schema.org/Product to describe all of the Products I’m selling in the the catalog. I can also use Hydra collections to paginate lists within the catalog. Obviously, there needs to be a cart where I collect and store the things I intend to purchase. The cart or basket lives on the server and my client can query it at any time. Now the part where I think things get interesting is how we get items into the cart. There are two ways I can think of off the top my head that we could do this:

* The client issues a POST request with the http://schema.org/Product instance or URI to add to the cart
* The client uses an AddItemToCartRequest message that includes the the product ID and the optional quantity and possibly the product variant, and the request is sent over POST

The first option is weak beer because it doesn’t covey intent. POST does not mean “Add to Cart” its just means POST. 

Most HTML forms follow the second pattern and you see hidden forms like so:

<form action=“/me/cart" 
          method="post" 
          id="product_addtocart_form">
	<input type="hidden" name="productId" value="1789">

	<select id=“productVarientId" style="display: none;">
		<option selected="selected" value="no">Select an option</option>
		<option value="1640">M.Zuiko Digital ED 40-150mm f4.0-5.6 R (Silver) $149.99</option>
		<option value="1641">M.Zuiko Digital ED 40-150mm f4.0-5.6 R (Black) $149.99</option>
	</select>
       <input type=“text" name=“quantity" value="">
       
</form>

This is nice and small and I’ve got additional details that should never exist in a Product such as quantity, etc.. Obviously, the HTML Forms example can’t convey why we’re sending this form and I need a special URI to post to.

In my mind, the equivalent in Hydra would be something like so:

{
    "@context": "http://www.w3.org/ns/hydra/context.jsonld",
    "@id": "http://example.com/me/cart",
    "operation": {
        "method": "POST",
        "expects": {
            "@type": "AddItemToCartRequest",
            "supportedProperty": [
                {
                    "property": {
                        "@id": "productId",
                        "@type": "rdf:Property",
                        "range": "http://www.w3.org/2001/XMLSchema#integer"
                    },
                    "required": true
                },
                {
                    "property": {
                        "@id": "productVarientId",
                        "@type": "rdf:Property",
                        "range": "http://www.w3.org/2001/XMLSchema#integer"
                    },
                    "required": true
                },
                {
                    "property": {
                        "@id": "quantity",
                        "@type": "rdf:Property",
                        "range": "http://www.w3.org/2001/XMLSchema#integer"
                    },
                    "required": true
                }
            ]
        }
    }
}

I can use @type to be more specific about the intent of this message. The server would see this message come as a call to action and there’s likely a handler on the server side to process an “ AddItemToCartRequest.” The ability for me to annotate the “operation” has little value to everyone but the client. The client doesn’t get much out of and the server and intermediaries can’t see it. They’d only be getting the AddItemToCartRequest and can’t have visibility to anything beyond the message being sent to the server. More importantly, I know what the intent of the user is because they sent me a message requesting to add the product to the cart as opposed to POSTing a http://schema.org/Product instance to a URL. 

The only actor I see who gets value out of @type’d Operations is the client. The client needs to be able to have a handle on which operation to select. Do I want to add a thing to my cart or remove a thing? As Mark mentioned, a predicate-based approach might work better. The hydra:search is a good example, we have’t been debating IriTemplate subclasses but we do have hydra:TemplatedLink sub properties. This doesn’t seem to be all that contentious. 

What if we made hydra:Operation a property like hydra:TemplatedLink so that we can have other “links” that do more than HTTP GET requests like so:

{
    "@context": "http://www.w3.org/ns/hydra/context.jsonld",
    "@id": "http://example.com/me/cart",
    "addItemToCart": {
        "method": "POST",
        "expects": {
            "@type": "AddItemToCartRequest",
            "supportedProperty": [
                {
                    "property": {
                        "@id": "productId",
                        "@type": "rdf:Property",
                        "range": "http://www.w3.org/2001/XMLSchema#integer"
                    },
                    "required": true
                },
                {
                    "property": {
                        "@id": "productVarientId",
                        "@type": "rdf:Property",
                        "range": "http://www.w3.org/2001/XMLSchema#integer"
                    },
                    "required": true
                },
                {
                    "property": {
                        "@id": "quantity",
                        "@type": "rdf:Property",
                        "range": "http://www.w3.org/2001/XMLSchema#integer"
                    },
                    "required": true
                }
            ]
        }
    },
    "removeItemFromCart": {
        "method": "POST",
        "expects": {
            "@type": "RemoveItemFromCartRequest",
            "supportedProperty": [
                {
                    "property": {
                        "@id": "productId",
                        "@type": "rdf:Property",
                        "range": "http://www.w3.org/2001/XMLSchema#integer"
                    },
                    "required": true
                }
            ]
        }
    }
}

I don’t think we loose anything here but we’re no longer (possibly) suggesting that POST has additional semantics beyond what HTTP provides. 

> 
> I think I'd agree that it's purer to do so from a theoretical POV but I
> doubt makes that much sense (or difference, for that matter) in practice.

How so? I build systems like this on a fairly regular basis. The HTML example I posted above does exactly that and it’s pretty much the same model most online stores use.

Getting the intent of requestor is important. There’s a huge loss of fidelity from a SQL Update statement to place an order vs an XML message that asserts what the requestor wants to buy, how much and how quickly they need to execute it. 

> More than likely, intermediaries will treat all of these message exactly the
> same way but just looking at the used HTTP method. On the other hand, the
> client needs to know whether it is going to be charged when it POSTs an
> Order or whether that just results in a "file" to be created on some server.

For starters, the client doesn’t care if the request is stored to disk. That’s an implementation detail of the server the the client doesn’t need to bother themselves with. Second of all, I am asking the server to place an order and have no idea how they might accomplish that. 

The difference with the “AddItemToCartRequest” approach I am talking about is that the client intent is visible all the way through from the client, any intermediaries, and finally to the origin server. As a client, all I asked the server to do was to add an item to my cart by issuing a POST with the AddItemToCartRequest as the message body. I don’t care if they wrote files on disk or created a new row in an RDBMS. I just want the server t add the item to my cart, how they do that is their business. But once the message leaves my client and sent over HTTP, all of Hydra goodness is gone and I just have an HTTP request with a message body that is application/ld+json. The server now knows my intent by looking at the type of message being sent and can pass this message off to the appropriate handler to add this item to my cart. 

The hydra:Operation sub classes loose this information once it leaves the client. The only way to get it is to look back at the ApiDocumention as well as having ll parties be Hydra-aware. Operation types are only ever visible to the client issuing the request. 

> 
> 
> 
>>> In that case he might look for a well established operation like
>> http://some.vocab.ns/Search
>>> and inspect how to format a request message in the second place.
>> 
>> Sure, but I can guarantee you that if this service is running over
>> HTTP, there'd be only two operations to perform: GET or POST. I'm
>> leaving out PUT, PATCH, and DELETE as they're not applicable to a
>> search use case. What the client needs to know now is:
>> 
>> * How do I send a "search" message to the service in order to get a
>> response back?
>> * Do I need to send the message via GET or POST?
>> 
>> It's not so much about describing the "operation" because we already
>> know all of the HTTP operations that we can act on. At present, that's
>> about 5 in total (i'm excluding the WebDAV methods) that would be
>> applicable to most applications. For me, it now boils down to
>> formatting and transmission; how does the client prepare the message
>> and what HTTP method do they send it with. I know I keep harping on
>> HTML forms, XForms, and OpenSearch, but it's because all of these
>> formats operate on the same set of assumptions.
> 
> Right. But they have either been created for a very specific use case
> (OpenSearch) or rely on the intelligence of the user to understand the
> consequences of POSTing a set of key-value pairs (HTML forms). With Hydra I
> would like to enable clients which can emulate at least parts of that
> intelligence by understanding a subset of possible consequences. I did
> choose a rather pragmatic approach by simply typing operations. Schema.org
> independently (I think) came up with basically the same design.

I still think you can achieve what you’re looking to do, but these details need to be pushed down into the message rather than the around the message. 

> 
> 
>>> Use-case: A generic client could use this information to present the
>>> user a search input when
>>> he browses a resource based on the existence of a search operation.
>> 
>> I still think this can be achieved. Checkout the JSONary project [1]
>> and how they use JSON Hyper Schema [2] to dynamically generate HTML
>> forms. If you take a gander at the Hyperschema docs, the link object
>> for Hyperschema seems to follow the HTML approach and doesn't give have
>> any additional semantics around the Link and all of the descriptive
>> effort is put into the "schema" property:
>> 
>> {
>>    "links": [
>>        {
>>            "rel": "search",
> 
> Well, I would argue the "rel": "search" provides the additional semantics
> here. 
> 
> 
>>            "href": "/search",
>>            "method" : "POST",
>>            "schema": {
>>                "type": "object",
>>                "properties": {
>>                    "q": {
>>                        "type": "string"
>>                    },
>>                    "count": {
>>                        "type": "integer",
>>                        "minimum": 10,
>>                        "default": 20
>>                    },
>>                    "start": {
>>                        "type": "integer",
>>                        "default": 0
>>                    }
>>                },
> 
> The tokens used in "schema" mean nothing at all without out of band
> knowledge.

To a JSON Schema capable processor, this information is not out of band. If they’re out of band, then so is Hydra IMHO. In JSON HyperSchema, the “schema” property is almost functionally equivalent to “expects.”

> 
> 
>>                "required": [ "q" ]
>>            }
>>        }
>>    ]
>> }
>> 
>> Sure, it doesn't look like much and JSON Schema itself still has a lot
>> of holes. But it's still plenty functional enough to build a generic
>> client. It's also simple and easy to work with and companies like
>> Heroku [3] and GitHub are using this today. But if you squint a little,
>> you can see that the "schema" is not very different than hydra:expects.
> 
> Agreed.. I would, however, go a step further and say that "rel": "search" in
> the example above isn't very different from "@type": "Search" either..
> 
> 
>> Arguably, the link should probably be the value the "rel" property such
>> that you'd have:
>> 
>> {
>>    "links": {
>>        "rel": {
> 
> You probably wanted to use "search" instead of "rel" in this example, right?

Yep! :) Thanks.

> 
> 
>>            "href": "/search",
>>            "method": "POST",
>>            "schema": {
>>                "type": "object",
>>                "properties": {
>>                    "q": {
>>                        "type": "string"
>>                    },
>>                    "count": {
>>                        "type": "integer",
>>                        "minimum": 10,
>>                        "default": 20
>>                    },
>>                    "start": {
>>                        "type": "integer",
>>                        "default": 0
>>                    }
>>                },
>>                "required": [  "q"  ]
>>            }
>>        }
>>    }
>> }
>> 
>> But that's likely for another forum.
>> 
>> Ryan-
>> 
>> [1] http://jsonary.com/
>> [2] http://json-schema.org/latest/json-schema-hypermedia.html
>> [3]
>> https://blog.heroku.com/archives/2014/1/8/json_schema_for_heroku_platfo
>> rm_api
> 
> If you look at [3], the only link I see in the example is a "create" link.
> Now that looks *very* much like a hydra:Operation.

Agreed.

> I also often see the same
> pattern used with HAL. It's certainly an option but I don't find that any
> better.

Actually, HAL is more like the last example whereby we have:

{
    "_links": {
        "search": {
            "href": "/search”,
            ...
        }
    }
}

I find HAL’s structure a bit more sensible. 

> Actually, I find it worse than explicitly allowing to describe
> operations on a resource. A link relation type should define the
> relationship between two resources. A relation type of "create" thus doesn't
> make much sense IMO.. and neither does the "edit" link relation in AtomPub
> btw. What would make sense instead is a "create-form" or "source" relation
> type. But that's really a different story :-)

100% Agreed. The link relation to a “create-form” is somewhat pointless when you have the ability to format a message that offers the same utility as the create-form?

The link relation model has some utility, but it was really build around a read-only model and it doesn’t fit super great for write operations. The hydra:search property is likely more akin to link relations than hydra:Operation. 

> 
> 
> 
> --
> Markus Lanthaler
> @markuslanthaler
> 

Ryan-

+-----------------------------------------------+
    Ryan J. McDonough
    http://damnhandy.com
    http://twitter.com/damnhandy

Received on Tuesday, 18 February 2014 00:05:46 UTC