Re: ISSUE-31: Are Operations violating REST's uniform interface constraint?

On Feb 18, 2014, at 10:55 AM, Markus Lanthaler <markus.lanthaler@gmx.net> wrote:

> On Tuesday, February 18, 2014 12:22 AM, Ryan J. McDonough wrote:
>> On Feb 17, 2014, at 3:03 PM, Markus Lanthaler wrote:
>>> On Sunday, February 16, 2014 8:14 PM, Ryan J. McDonough wrote:
>>>> I still believe that client doesn't gain much with the @type of the
>>>> operation. What the client must care about is how to prepare the
>>>> request and deliver it over the correct HTTP method. The operation
>>>> is being carried out by the information in the search request
>>>> message, not the annotations around it.
>>> 
>>> OK. Let's try to use a more interesting example than search. Let's
>>> talk about ordering goods. So, you are arguing that the client would
>>> need to send a do-order-this-stuff message, right?
>> 
>> Exactly. The idea being that the request message is clear about the
>> users intent and that the client does hold any information that the
>> server can't see.
>> 
>>> Likely, the server will transform that to an Order. So, the 
>>> question is, can I reuse a Order (let's say, e.g.,
>>> http://schema.org/Order) to order some stuff or do I need to design a
>>> separate do-order-this-stuff message type?
>> 
>> I think it depends. The schema.org example actually represents the
>> receipt, or the end state of an activity of making a purchase. This
>> would be more applicable to a response type after executing the order.
> 
> OK, agreed. That was probably a bad example.
> 
> 
> [...]
> 
>>> This now goes back to something I discussed with Mark a while ago,
>>> but if the server I'm talking to doesn't
>>> deliver any goods but just stores files. Do I need another
>>> store-this-order-representation message type?
>> 
>> I'd approach it more like this: you have a catalog of things you browse
>> and search through. I may be able to use http://schema.org/Product to
>> describe all of the Products I'm selling in the the catalog. I can also
>> use Hydra collections to paginate lists within the catalog. Obviously,
>> there needs to be a cart where I collect and store the things I intend
> 
> I don't find that so obvious. You can just as well collect these things
> entirely on the client

Not sure I follow. You’d need some type of structure somewhere that the client can add items too. It could live on the server or the client, but there needs to be something. 

> 
> 
>> to purchase. The cart or basket lives on the server and my client can
>> query it at any time. Now the part where I think things get interesting
>> is how we get items into the cart. There are two ways I can think of
>> off the top my head that we could do this:
> 
> Let's try to keep the example simple and order things directly (similar to
> Amazon's one-click ordering)
> 
> 
>> * The client issues a POST request with the http://schema.org/Product
>> instance or URI to add to the cart
> 
> Let's concentrate on this scenario. There are no quantities.. A client can
> order just one piece of each product and each product has to be ordered
> separately.
> 
> 
>> * The client uses an AddItemToCartRequest message that includes the the
>> product ID and the optional quantity and possibly the product variant,
>> and the request is sent over POST
>> 
>> The first option is weak beer because it doesn't covey intent. POST
>> does not mean "Add to Cart" its just means POST.
> 
> Right. The question is, do intermediaries need to be able to distinguish
> that or is it enough if the server and the client know that a POST with a
> product to a specific resource means that the client wants to order that
> product? The server obviously knows it because he was it who informed the
> client about that possible operation. 
> 
> 
>> Most HTML forms follow the second pattern and you see hidden forms like
>> so:
>> 
>> <form action="/me/cart"
>>          method="post"
>>          id="product_addtocart_form">
>> 	<input type="hidden" name="productId" value="1789">
>> 
>> 	<select id="productVarientId" style="display: none;">
>> 		<option selected="selected" value="no">Select an
>> option</option>
>> 		<option value="1640">M.Zuiko Digital ED 40-150mm f4.0-5.6 R
>> (Silver) $149.99</option>
>> 		<option value="1641">M.Zuiko Digital ED 40-150mm f4.0-5.6 R
>> (Black) $149.99</option>
>> 	</select>
>>       <input type="text" name="quantity" value="">
>> 
>> </form>
>> 
>> This is nice and small and I've got additional details that should
>> never exist in a Product such as quantity, etc.. Obviously, the HTML
>> Forms example can't convey why we're sending this form and I need a
>> special URI to post to.
> 
> Right. The same key-value pairs could be used for other things as well. And
> if we reduce this to just the product's URI (or just the ID as in the HTML
> form above) this becomes even more apparent.
> 
> 
>> In my mind, the equivalent in Hydra would be something like so:
>> 
>> {
>>    "@context": "http://www.w3.org/ns/hydra/context.jsonld",
>>    "@id": "http://example.com/me/cart",
>>    "operation": {
>>        "method": "POST",
>>        "expects": {
>>            "@type": "AddItemToCartRequest",
> 
> Yeah, but without that type information.


Yes.

> 
> 
>>            "supportedProperty": [
>>                {
>>                    "property": {
>>                        "@id": "productId",
>>                        "@type": "rdf:Property",
>>                        "range":
>> "http://www.w3.org/2001/XMLSchema#integer"
>>                    },
>>                    "required": true
>>                },
>>                {
>>                    "property": {
>>                        "@id": "productVarientId",
>>                        "@type": "rdf:Property",
>>                        "range":
>> "http://www.w3.org/2001/XMLSchema#integer"
>>                    },
>>                    "required": true
>>                },
>>                {
>>                    "property": {
>>                        "@id": "quantity",
>>                        "@type": "rdf:Property",
>>                        "range":
>> "http://www.w3.org/2001/XMLSchema#integer"
>>                    },
>>                    "required": true
>>                }
>>            ]
>>        }
>>    }
>> }
>> 
>> I can use @type to be more specific about the intent of this message.
>> The server would see this message come as a call to action and there's
>> likely a handler on the server side to process an "
>> AddItemToCartRequest."
> 
> I can't help, but to me this looks like a perfect example of RPC:
> 
>  addItemToCart(productId, productVarientId, quantity)

How did you arrive at that? If you want to look at it that way, the sure. However, I see it more like this:

curl -X POST -H "Content-Type: application/ld+json" -d '{"@context": "http://example.com/cart", "@type": "AddItemToCartRequest", "productId": "1789", "productVarientId": "1641", "quantity": 2}' http://example.com/my/cart

Or like this:

 var client = new XMLHttpRequest();
  client.open("POST", "http://example.com/my/cart/my/cart");
  client.setRequestHeader("Content-Type", "application/ld+json");
  client.send({ "@context": "http://example.com/cart",
				"@type": "AddItemToCartRequest",
				"productId": "1789",
				"productVarientId": "1641",
				"quantity": 2 });

Sure, there’s nothing stopping you from sending wrapping either of the above in a function like you’ve described, but that function would still have to format the message and send it over HTTP POST. I’m still working with the uniform interface of HTTP. The only difference is that the message being sent indicates intent. What I want to do is expressed in the message rather than around it.

> 
> In REST, you would much rather want to modify its state by PUTing a new
> representation.

Why must this always the case? As I’ve illustrated with the HTML forms example already, this is not the model that HTML forms work with today.  Unless of course you disagree with the statement that HTML exhibits a number of RESTful qualities. 

> 
> The problem I'm trying to bring across is that changing the state of a
> specific resource might trigger other processes that yield to consequences
> outside of the limited world view that HTTP offers; they are entirely
> defined by the server. With the file example I meant that a server whose
> functionality is simply to store files is fundamentally different to an
> e-commerce application. Nevertheless, both might expose exactly the same
> resource representations and both might support exactly the same
> interactions. The outcomes, however, will be completely different. One
> service just stores the representations it gets whereas the other will also
> debit credit cards.
> 
> Intermediaries don't need to be aware of these differences and thus these
> "implementation details" are hidden behind a uniform interface. That,
> however, doesn't mean that the client doesn't need to know them.
> 
> 
>> The ability for me to annotate the "operation"
>> has little value to everyone but the client.
> 
> Sure, but why should that be a problem in practice? Obviously, the client
> and the server need to have a shared understanding of the side effects that
> invoking operations entails but, as long as HTTP semantics aren't violated,
> intermediaries will continue to work flawlessly.

I know you wanted to keep out quantities, etc., but if you’re working under the assumption that we’re adding items to a cart representation and using an HTTP PUT to modify the state of cart, it’s not immediately obvious that we’ve added something to the cart, changed quantities, or removed an item from a cart. 

> 
> 
>> The client doesn't get
>> much out of and the server and intermediaries can't see it. They'd only
>> be getting the AddItemToCartRequest and can't have visibility to
>> anything beyond the message being sent to the server.
> 
> Sure, and that's fine IMHO.
> 
> 
>> More importantly,
>> I know what the intent of the user is because they sent me a message
>> requesting to add the product to the cart as opposed to POSTing a
>> http://schema.org/Product instance to a URL.
> 
> POST is always tricky as it has so little semantics. If we would replace
> that example with PUTing a new shopping cart representation to the URL of my
> shopping cart, the situation would be quite different. The intent would be
> quite clear without requiring a special AddItemToCartRequest message.

I assuming that the interaction you’re talking about is that the client dereferences a representation of the current state of the cart. When they want to add something to that, the modify the cart locally and issue a PUT to replace the current state of the cart, correct?

Assuming that’s true, the server has some work to do to determine what happened to the cart. Did I add an item? Did I change the quantity? Did I change the color of the thing I ordered? Did I remove the item? And “UpdateCartOperation” is just too vague and the expectations that a client may have on this operation could yield unexpected results. 

> 
> 
>> The only actor I see who gets value out of @type'd Operations is the
>> client.
> 
> Right, exactly the same is true for the rest of the form or operation
> description.
> 
> 
>> The client needs to be able to have a handle on which operation
>> to select. Do I want to add a thing to my cart or remove a thing? As
>> Mark mentioned, a predicate-based approach might work better. The
>> hydra:search is a good example, we have't been debating IriTemplate
>> subclasses but we do have hydra:TemplatedLink sub properties. This
>> doesn't seem to be all that contentious.
>> 
>> What if we made hydra:Operation a property like hydra:TemplatedLink so
>> that we can have other "links" that do more than HTTP GET requests like
>> so:
> 
> You can have that already. Hydra Links and TemplatedLinks allow you to
> associate a number of supportedOperations to such link relations aka
> properties.
> 
> 
>> {
>>    "@context": "http://www.w3.org/ns/hydra/context.jsonld",
>>    "@id": "http://example.com/me/cart",
>>    "addItemToCart": {
>>        "method": "POST",
>>        "expects": {
>>            "@type": "AddItemToCartRequest",
>>            "supportedProperty": [
>>                {
>>                    "property": {
>>                        "@id": "productId",
>>                        "@type": "rdf:Property",
>>                        "range":
>> "http://www.w3.org/2001/XMLSchema#integer"
>>                    },
>>                    "required": true
>>                },
> [...]
>>    },
>>    "removeItemFromCart": {
>>        "method": "POST",
>>        "expects": {
>>            "@type": "RemoveItemFromCartRequest",
>>            "supportedProperty": [
> [...]
>>            ]
>>        }
>>    }
>> }
>> 
>> I don't think we loose anything here but we're no longer (possibly)
>> suggesting that POST has additional semantics beyond what HTTP
>> provides.
> 
> Maybe it's just me but I can't see how this is any different. Intermediaries
> won't see the "addItemToCart" property either. The downside of this approach
> is that you can't collect operations that easily anymore. Before you had a
> single property and an array of operations (mainly talking about JSON-LD
> here but in principle the same is true for any other serialization format),
> with this model you have multiple properties you need to look into. Don't
> get me wrong. It's not a big deal but I can't see any advantage of doing so.

Putting a straight JSON approach, it’s a tad easier to use the property approach because I don’t have to scan the array for a specific operation type. The other thing it buys is consistency with hydra:TemplatedLink, etc.. There’s a lot of similarity between hydra:TemplatedLink and hydra:Operation. 

> 
> 
>>> I think I'd agree that it's purer to do so from a theoretical POV but
>>> I doubt makes that much sense (or difference, for that matter) in
>>> practice.
>> 
>> How so? I build systems like this on a fairly regular basis. The HTML
>> example I posted above does exactly that and it's pretty much the same
>> model most online stores use.
> 
> We are going in circles now, but let me nevertheless say it one more time:
> all of those HTML forms are accompanied with prose that describes the
> consequences of sending those key-value pairs. That information is just as
> invisible to intermediaries as the operation's type is. I would argue that
> doesn't matter in >99% of the cases. In the few where it does, the form
> typically contains a hidden field or a drop-down for the user to select the
> desired operation. This makes the intent then more explicit. We can have
> both in Hydra as well.
> 
> It's not so that we have a PostOperation, PutOperation etc. and are
> subclassing them. We have a very generic Operation class which says nothing
> about the HTTP method and specialize that. So the operation defines the
> abstract task a client might want to carry out. That abstract task is then
> *mapped* to an HTTP request template (method / expects
> 
> 
>> Getting the intent of requestor is important. There's a huge loss of
>> fidelity from a SQL Update statement to place an order vs an XML
>> message that asserts what the requestor wants to buy, how much and how
>> quickly they need to execute it.
> 
> Agreed.
> 
> 
>>> More than likely, intermediaries will treat all of these message exactly
> the
>>> same way but just looking at the used HTTP method. On the other hand,
> the
>>> client needs to know whether it is going to be charged when it POSTs an
>>> Order or whether that just results in a "file" to be created on some
>>> server.
>> 
>> For starters, the client doesn't care if the request is stored to disk.
>> That's an implementation detail of the server the the client doesn't
>> need to bother themselves with. Second of all, I am asking the server
>> to place an order and have no idea how they might accomplish that.
> 
> [...]
> 
> As said above, I think you misunderstood what I was trying to say here. I
> hope the explanation above clarified that.
> 
> 
>>>>> In that case he might look for a well established operation like
>>>> http://some.vocab.ns/Search
>>>>> and inspect how to format a request message in the second place.
>>>> 
>>>> Sure, but I can guarantee you that if this service is running over
>>>> HTTP, there'd be only two operations to perform: GET or POST. I'm
>>>> leaving out PUT, PATCH, and DELETE as they're not applicable to a
>>>> search use case. What the client needs to know now is:
>>>> 
>>>> * How do I send a "search" message to the service in order to get a
>>>> response back?
>>>> * Do I need to send the message via GET or POST?
>>>> 
>>>> It's not so much about describing the "operation" because we already
>>>> know all of the HTTP operations that we can act on. At present, that's
>>>> about 5 in total (i'm excluding the WebDAV methods) that would be
>>>> applicable to most applications. For me, it now boils down to
>>>> formatting and transmission; how does the client prepare the message
>>>> and what HTTP method do they send it with. I know I keep harping on
>>>> HTML forms, XForms, and OpenSearch, but it's because all of these
>>>> formats operate on the same set of assumptions.
>>> 
>>> Right. But they have either been created for a very specific use case
>>> (OpenSearch) or rely on the intelligence of the user to understand the
>>> consequences of POSTing a set of key-value pairs (HTML forms). With
> Hydra I
>>> would like to enable clients which can emulate at least parts of that
>>> intelligence by understanding a subset of possible consequences. I did
>>> choose a rather pragmatic approach by simply typing operations.
> Schema.org
>>> independently (I think) came up with basically the same design.
>> 
>> I still think you can achieve what you're looking to do, but these
>> details need to be pushed down into the message rather than the around
>> the message.
> 
> So you say sending an AddItemToCartRequest is preferable than PUTing a new
> ShoppingCart representation?

If I had my druthers, yes this is exactly what I’d do. And I’d send it over POST and not PUT. You can imagine that by adding n items to my cart, this may or may not have an affect on available inventory to there potential shoppers. Perhaps my store will reserve the quantity in your cart for some period of time until you check out? Since this operation will now have side effects on other resources, say the catalog and available inventory, PUT is not appropriate. This is pretty much the same way HTML forms work today, but you’re only dealing with key/value pairs. But because I’m affecting other resources, I want to use POST. 

From a performance perspective, smaller messages tend be faster over the wire. Using a PUT to update the entire cart would get slower because now my cart grows with every addition and will continually get bigger. Furthermore, I don’t know why someone sent back their cart using a PUT. All I know is that they issued a PUT. I you haven’t guessed yet, I’m a bit more of fan of Domain Driven Design and CQRS. In short, I don’t like making my write model the same as my read model. I also can optimize writes differently than I do for my write model. 

But we’re going off the rails a bit here, and I want to go back to Mark’s original point. This debate we are having go back to Mark’s DELETE/Clear example. What your solution seems to be advocating is collecting the items selected by the client and adding them to a data structure that models a ShoppingCart. In your solution, your client would be be issuing a PUT to the cart URI. You might call this “UpdateCartOperation” but all you’ve really done is replace the state of cart with no content of what the update entailed. This operation lacks visibility in that one cannot determine if I added a new item to the cart, removed an item from the cart, or simply updated a quantity. These are 3 distinct activities that affect the same concept. If we called your operation “ UpdateCartOperation”, developers will have different expectations around the kind of response that gets returned. 

> 
>>> Actually, I find it worse than explicitly allowing to describe
>>> operations on a resource. A link relation type should define the
>>> relationship between two resources. A relation type of "create" thus
>>> doesn't make much sense IMO.. and neither does the "edit" link
>>> relation in AtomPub btw. What would make sense instead is a
>>> "create-form" or "source" relation type. But that's really a
>>> different story :-)
>> 
>> 100% Agreed. The link relation to a "create-form" is somewhat pointless
>> when you have the ability to format a message that offers the same
>> utility as the create-form?
> 
> Why? You can embed it instead of just referencing it.

Yes, that was what I was saying :) Atom never had functionality like HTML Forms, whereas Hydra does. Thus, a create-form operation in Hydra is pointless.

> 
> 
> 
> --
> Markus Lanthaler
> @markuslanthaler
> 
>