Re: LDP paging: spec'ing Sandro's proposal

I don't see how HTTP range requests are useful for paging.
We are dealing with items in collections, so all you can request is a range
of items, either by ordinal number in a collection -- send me items 10 to 20,
or by a range of key values they were ordered on.
All the best, Ashok

On 6/11/2014 5:55 PM, Sandro Hawke wrote:
> Quick response, not having reviewed your changes yet:
>
> On 06/10/2014 02:46 PM, John Arwe wrote:
>> Moving this on-list, since to the best of everyone's ability/memory amongst the progenitors and editors the overall intent has been captured to first order.
>>
>> > This document describes...
>>
>> I tweaked it very modestly, probably not enough to get you happy ... model > protocol basically.  "access portions of state" sounds like range requests; "that impl this spec" seems superfluous.  But frankly I'd rather concentrate on getting the normative meat straight first and revisit this later.
>> Not allergic to Modules, although as a reader of HTML4 (?) modules, YIKES was that impenetrable.
>>
>> > too large to fit in a single HTTP response
>>
>> tweaked to give an example.  The limits come from implementations (memory etc), not HTTP AFAIK.
>>
>> > Single-page resource ...terribly confusing name. ... Partial page?   Sub-page? Sequence-page?    Traversal Page? ... All of those would be much better.
>>
>> I hate naming generally.  Let's raise the issue/wiki/whatever, get consensus, change the drafts Once more.
>>
>> > LDP Paging Clients ... affordances missing, ... clients must ask for paging
>>
>> Issue-worthy.   Personally I don't see the difference between this and ignoring an Accept header, and (so far) the Net has not ended due to the latter - despite it breaking some clients, with zero doubt.  But maybe I'm missing something in your argument.  I wonder if you're looking for a "stronger than 'both use HTTP'" level of interop guarantee.
>>
>> > 4.1.1
>>
>> Well there is the very next line in the draft: TODO: Confirm this client MUST for server-initiated paging
>> In other words, even in this draft's red-headed-step-child current state Steve and/or I questioned it too.
>>
>
> Yes, sorry.
>
>> > read committed isolation ...
>>
>> TallTed has vocally asked for that kind of material more than once; IIRC some related proposals were -1'd.  Let's raise an issue, get some consensus, and change it One last time.
>>
>> > 333
>>
>> Since the TAG discussions have matured a bit, I tweaked it quickly to 2NN per item 2 in their draft comments so at least the class will match current thinking.
>>
>> > HTTP dialog trace, showing the actual headers
>>
>> As Steve and I mentioned a week or two ago on the weekly call, we'd already concluded that was better for LDP proper and the intent was to be consistent.  In the specific case of Paging, I've started noting html-comment level todos but I want to take ONE pass through them AFTER we have the bulk of the normative material agreed to.  Was not about to ripple through those changes until you said that your proposal was essentially captured.
>>
>> > Do we say how jpg and html and zip files can be paged...?   Are we suggesting they can be?
>>
>> No, and we're allowing for the possibility that some other/later spec might define how paging works interoperably for non-LDP-RSs.  That was a WG consensus at the time we split it off, I've had to check that myself when doing some of the initial drafting.
>>
>
> I wonder if there's any way to connect this to HTTP Ranges (rfc7233).   One could perhaps view paging as giving ranges URLs.
>
> (mostly I'd prefer we avoid thinking about anything other than RDF, since it's outside our expertise.)
>
>> > 5.2.3.8 LDP ..., but SHOULD only add pages to the end of a sequence.   SH: Doesn't that contradict ordering?
>>
>> This was added based on WG discussions (Ashok/TallTed IIRC) when we were focused more on "stability" of traversal and all the snapshot stuff was floating around.  I agree that certain kinds of ordered containers (like alphabetic sort on some key under client control) would either need to not implement this Should, or cease advertising themselves as ordered.  Other kinds of ordering (change log/anything time-ordered) plays fine with it, so they're not automatically in opposition.
>>
>> > 5.2.3.11/...13
>>
>> Each is only for one kind of traversal.  We require clients to support both traversal directions.
>> Squinting, I don't see that we say that any given page-sequence Must be either forw or backward-linked, although I *thought* we had agreed to that.  Is that the way others remember it?
>>
>> > 5.2.3.15 type=ldp:Page header ... This is one mechanism by which clients know ... I don't think that's true.    The client knows because...
>>
>> As stated, it's definitionally true (one way).  I think you're arguing that it's not the only way, so we're not being minimal.
>
> Yeah, ooops, I was overtired and simply misread the spec as saying "this is the mechanism".   Sorry I led you astray on this!
>
>> You bring up how the client obtained the link; given that links can be passed around freely and/or bookmarked (the wisdom of doing so aside), I don't think you can *depend* on that.
>> You bring up the canonical link; *if* one assumes that only LDP servers mint canonical links, then you might treat them as equivalent.  Given that we're not defining that link relation as new, nor if we did would we likely argue that it's LDP-only, I don't think you can *depend* on that.  6596 "recommends" only one canonical link per resource, although not in obviously normative language.
>> It Might well be that "knowing" that a resource is a Page is simply uninteresting outside of our little context.  If we don't have scenarios requiring a client to be *reliably* able to traverse from any random page to the corresponding paged resource, it could be that we say "follow the canonical link, and maybe it's The Paged Resource/maybe not" = good enough.
>> I.e. I could look at it as: if the only thing you get from knowing that some URI handed to you identifies "a page" is that there's Some Bigger Related Thing out there, somewhere, and some options for finding pages related to the SBRT, and (maybe, via one of the canonical link(s)) the SBRT's URI, ho hum.
>>
>> The closest I can come quickly to imagining a problem is thinking about paging when the paged-resource itself is part of a sequence.  E.g. the paged resource is a container, and a VCS links to successive versions of that paged resource via the same "sequence" headers we use for LDP paging.  I could still find my way through that (v=n.lastpage.next = nil/omitted, do a Head on v=n to get v=n.next -> v=(n+1) ).
>> Based on that I can't put my finger on what breaks absent the type=ldp:Page header, so I'd be amenable at this point if you propose removing it.
>>
>> > which would defeat the purpose of pagination ... If that's all pagination is for, then we'd only use it with sorted resources.
>>
>> I have a TODO in the source (which you would not have seen) to similar effect.  I think sharing ordering of a container's partitioning of members to pages is an optimization for clients working with servers that do/can order the members to avoid repeating work already done on the server (allows for a more efficient client-side implementation).
>
> I perhaps need to get more comfortable with this ordering mechanism.   The problem is, I only work with generic LDP servers, for which this mechanism is useless.  What would make it useful is if clients could request it as well, and maybe get redirected to a resource that had that ordering.   You guys ever think about something like that?
>
>        -- Sandro
>
>>
>>
>>
>> Best Regards, John
>>
>> Voice US 845-435-9470 BluePages <http://w3.ibm.com/jct03019wt/bluepages/simpleSearch.wss?searchBy=Internet+address&location=All+locations&searchFor=johnarwe>
>> Cloud and Smarter Infrastructure OSLC Lead
>>
>>
>>
>>
>> From: Sandro Hawke <sandro@w3.org>
>> To: John Arwe/Poughkeepsie/IBM@IBMUS
>> Cc: Steve K Speicher/Raleigh/IBM@IBMUS, Arnaud Le Hors/Cupertino/IBM@IBMUS
>> Date: 06/08/2014 11:41 PM
>> Subject: Re: LDP paging: spec'ing Sandro's proposal
>>
>> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>>
>>
>> On 05/20/2014 05:42 PM, John Arwe wrote:
>> I've done the majority of the initial writing in the live editor's draft, trying to keep this moving.  Please see if there's anything big-animal MIA.  The chat logs were not terribly detailed.
>> The following are all that I know of NOT in the draft yet.  There's a somewhat larger working list (that includes the DONEs) at very top of the html source.
>>
>>         TODO: _http://www.w3.org/2013/meeting/ldp/2014-04-17_resolution 4: page size hint
>>        TODO: _http://www.w3.org/2013/meeting/ldp/2014-04-17_resolution 5: contains+member triple always on same page
>>
>> Best Regards, John
>>
>> Voice US 845-435-9470 _BluePages_ <http://w3.ibm.com/jct03019wt/bluepages/simpleSearch.wss?searchBy=Internet+address&location=All+locations&searchFor=johnarwe>
>> Cloud and Smarter Infrastructure OSLC Lead
>>
>>
>> Sorry I took so long -- I confirm these changes look correct to me, as does the handling of change during traversal.
>>
>> I find several other problems, though, that have probably been around a long time, as well as some stylistic things I would love to see changed.
>>
>> Feel free to quote this & reply on list, if you want.
>>
>> > This document describes a model for clients and servers to be able to efficiently retrieve large Linked Data Platform Resource representations by splitting up the responses into separate URL-addressable page resources.
>>
>> Not fond of "model".
>>
>> How about:  This document specifies a way for Linked Data Platform (LDP) Clients to access portions of the state of large Linked Data Platform Resources on servers which implement this specification.
>>
>> Actually, I'd love to say it's an LDP Module, but the group hasn't really talked about that.
>>
>> > to deal with large resources.
>>
>> RDF Sources?
>>
>> (Ah, coming back later, I see in general paging is for everything.  That's kind of weird, but okay, I guess...)
>>
>> > too large to fit in a single HTTP response
>>
>> conveniently?
>>
>> There's no limit to HTTP response size, is there?
>>
>> > Single-page resource
>>
>> I still think this is terribly confusing name.
>>
>> Partial page?   Sub-page?     Sequence-page?    Traversal Page?
>>
>> All of those would be much better.
>>
>> > LDP Paging Clients
>>
>> I have a real problem with this, but I don't think I can explain this yet.   Basically, there have to be affordances for everything in HTTP, and there's none here.
>>
>> I think the actual rule is: EVERY *LDP CLIENT* MUST TRANSPARENTLY HANDLE FORWARD AND BACKWARD TRAVERSAL.
>>
>> Anything less means that servers can't initiate paging without breaking some clients.  IE they can't initiate paging.
>>
>> Without that rule, the only time to do paging is when the client asks for it.
>>
>> > 4.1.1 LDP Paging clients MUST be paging-aware in cases where LDP Paging servers initiate paging.
>>
>> Aside from the above point, I can't figure out what this could mean.
>>
>> > read committed isolation ...
>>
>> I happen to find these links make it all much more complex and confusing than necessary.   The behavior we actually want is simpler than this sounds.
>>
>> > 333
>>
>> I'd think we should use a 2xx code as the placeholder, probably 209.
>>
>> > Example 1
>>
>> I think this would be more clear as an HTTP dialog trace, showing the actual headers.....
>>
>> > any resource (LDP-RS or not) as a paged resource.
>>
>> What's the benefit of non-RS paging?    Do we say how jpg and html and zip files can be paged...?   Are we suggesting they can be?
>>
>> > 5.2.3.8 LDP Paging servers MAY add or remove single-page resources to a paged resource's sequence over time, but SHOULD only add pages to the end of a sequence.
>>
>> Doesn't that contradict ordering?
>>
>> > 5.2.3.11
>>
>> Isn't that only for forward traversal?
>>
>> > 5.2.3.13
>>
>> Isn't this only for backward traversal?
>>
>> > 5.2.3.15 LDP Paging servers MUST provide an HTTP Link header whose target URI is _http://www.w3.org/ns/ldp#Page_, ... This is one mechanism by which clients know that the resource is one of a sequence of pages.
>>
>> I don't think that's true.    The client knows because it got the URL via a link first/last/previous/next header. Also the "canonical" link.
>>
>> I don't see any value to this type, aside from not liking it's name.
>>
>> > which would defeat the purpose of pagination
>>
>> If that's all pagination is for, then we'd only use it with sorted resources.  I'm not sure what else paginating is for, but people seem to want it.  (I guess it's for client access to flow control, when it can't touch TCP.)
>>
>> That's it for now.   *shrug*
>>
>> I guess I could make these changes, if you want.    Probably a few issues that need WG discussion there, though.
>>
>>        -- Sandro
>>
>