Re: LDP paging: spec'ing Sandro's proposal

Moving this on-list, since to the best of everyone's ability/memory 
amongst the progenitors and editors the overall intent has been captured 
to first order.

> This document describes...

I tweaked it very modestly, probably not enough to get you happy ... model 
> protocol basically.  "access portions of state" sounds like range 
requests; "that impl this spec" seems superfluous.  But frankly I'd rather 
concentrate on getting the normative meat straight first and revisit this 
later.
Not allergic to Modules, although as a reader of HTML4 (?) modules, YIKES 
was that impenetrable.

> too large to fit in a single HTTP response

tweaked to give an example.  The limits come from implementations (memory 
etc), not HTTP AFAIK.

> Single-page resource ...terribly confusing name. ... Partial page? 
Sub-page?     Sequence-page?    Traversal Page? ... All of those would be 
much better.

I hate naming generally.  Let's raise the issue/wiki/whatever, get 
consensus, change the drafts Once more.

> LDP Paging Clients ... affordances missing, ... clients must ask for 
paging

Issue-worthy.   Personally I don't see the difference between this and 
ignoring an Accept header, and (so far) the Net has not ended due to the 
latter - despite it breaking some clients, with zero doubt.  But maybe I'm 
missing something in your argument.  I wonder if you're looking for a 
"stronger than 'both use HTTP'" level of interop guarantee.

> 4.1.1

Well there is the very next line in the draft: TODO: Confirm this client 
MUST for server-initiated paging
In other words, even in this draft's red-headed-step-child current state 
Steve and/or I questioned it too.

> read committed isolation ...

TallTed has vocally asked for that kind of material more than once; IIRC 
some related proposals were -1'd.  Let's raise an issue, get some 
consensus, and change it One last time.

> 333

Since the TAG discussions have matured a bit, I tweaked it quickly to 2NN 
per item 2 in their draft comments so at least the class will match 
current thinking.

> HTTP dialog trace, showing the actual headers

As Steve and I mentioned a week or two ago on the weekly call, we'd 
already concluded that was better for LDP proper and the intent was to be 
consistent.  In the specific case of Paging, I've started noting 
html-comment level todos but I want to take ONE pass through them AFTER we 
have the bulk of the normative material agreed to.  Was not about to 
ripple through those changes until you said that your proposal was 
essentially captured.

> Do we say how jpg and html and zip files can be paged...?   Are we 
suggesting they can be?

No, and we're allowing for the possibility that some other/later spec 
might define how paging works interoperably for non-LDP-RSs.  That was a 
WG consensus at the time we split it off, I've had to check that myself 
when doing some of the initial drafting.

> 5.2.3.8 LDP ..., but SHOULD only add pages to the end of a sequence. SH: 
Doesn't that contradict ordering?

This was added based on WG discussions (Ashok/TallTed IIRC) when we were 
focused more on "stability" of traversal and all the snapshot stuff was 
floating around.  I agree that certain kinds of ordered containers (like 
alphabetic sort on some key under client control) would either need to not 
implement this Should, or cease advertising themselves as ordered.  Other 
kinds of ordering (change log/anything time-ordered) plays fine with it, 
so they're not automatically in opposition.

> 5.2.3.11/...13

Each is only for one kind of traversal.  We require clients to support 
both traversal directions.
Squinting, I don't see that we say that any given page-sequence Must be 
either forw or backward-linked, although I *thought* we had agreed to 
that.  Is that the way others remember it?

> 5.2.3.15 type=ldp:Page header ... This is one mechanism by which clients 
know ... I don't think that's true.    The client knows because...

As stated, it's definitionally true (one way).  I think you're arguing 
that it's not the only way, so we're not being minimal.
You bring up how the client obtained the link; given that links can be 
passed around freely and/or bookmarked (the wisdom of doing so aside), I 
don't think you can *depend* on that.
You bring up the canonical link; *if* one assumes that only LDP servers 
mint canonical links, then you might treat them as equivalent.  Given that 
we're not defining that link relation as new, nor if we did would we 
likely argue that it's LDP-only, I don't think you can *depend* on that. 
6596 "recommends" only one canonical link per resource, although not in 
obviously normative language.
It Might well be that "knowing" that a resource is a Page is simply 
uninteresting outside of our little context.  If we don't have scenarios 
requiring a client to be *reliably* able to traverse from any random page 
to the corresponding paged resource, it could be that we say "follow the 
canonical link, and maybe it's The Paged Resource/maybe not" = good 
enough.
I.e. I could look at it as: if the only thing you get from knowing that 
some URI handed to you identifies "a page" is that there's Some Bigger 
Related Thing out there, somewhere, and some options for finding pages 
related to the SBRT, and (maybe, via one of the canonical link(s)) the 
SBRT's URI, ho hum.

The closest I can come quickly to imagining a problem is thinking about 
paging when the paged-resource itself is part of a sequence.  E.g. the 
paged resource is a container, and a VCS links to successive versions of 
that paged resource via the same "sequence" headers we use for LDP paging. 
 I could still find my way through that (v=n.lastpage.next = nil/omitted, 
do a Head on v=n to get v=n.next -> v=(n+1) ).
Based on that I can't put my finger on what breaks absent the 
type=ldp:Page header, so I'd be amenable at this point if you propose 
removing it.

> which would defeat the purpose of pagination ... If that's all 
pagination is for, then we'd only use it with sorted resources.

I have a TODO in the source (which you would not have seen) to similar 
effect.  I think sharing ordering of a container's partitioning of members 
to pages is an optimization for clients working with servers that do/can 
order the members to avoid repeating work already done on the server 
(allows for a more efficient client-side implementation). 



Best Regards, John

Voice US 845-435-9470  BluePages
Cloud and Smarter Infrastructure OSLC Lead




From:   Sandro Hawke <sandro@w3.org>
To:     John Arwe/Poughkeepsie/IBM@IBMUS
Cc:     Steve K Speicher/Raleigh/IBM@IBMUS, Arnaud Le 
Hors/Cupertino/IBM@IBMUS
Date:   06/08/2014 11:41 PM
Subject:        Re: LDP paging: spec'ing Sandro's proposal



On 05/20/2014 05:42 PM, John Arwe wrote:
I've done the majority of the initial writing in the live editor's draft, 
trying to keep this moving.  Please see if there's anything big-animal 
MIA.  The chat logs were not terribly detailed. 
The following are all that I know of NOT in the draft yet.  There's a 
somewhat larger working list (that includes the DONEs) at very top of the 
html source. 
        TODO: http://www.w3.org/2013/meeting/ldp/2014-04-17 resolution 4: 
page size hint 
        TODO: http://www.w3.org/2013/meeting/ldp/2014-04-17 resolution 5: 
contains+member triple always on same page 
Best Regards, John

Voice US 845-435-9470  BluePages 
Cloud and Smarter Infrastructure OSLC Lead 

Sorry I took so long -- I confirm these changes look correct to me, as 
does the handling of change during traversal.

I find several other problems, though, that have probably been around a 
long time, as well as some stylistic things I would love to see changed.

Feel free to quote this & reply on list, if you want.

> This document describes a model for clients and servers to be able to 
efficiently retrieve large Linked Data Platform Resource representations 
by splitting up the responses into separate URL-addressable page 
resources. 

Not fond of "model". 

How about:  This document specifies a way for Linked Data Platform (LDP) 
Clients to access portions of the state of large Linked Data Platform 
Resources on servers which implement this specification.

Actually, I'd love to say it's an LDP Module, but the group hasn't really 
talked about that.

> to deal with large resources. 

RDF Sources? 

(Ah, coming back later, I see in general paging is for everything.  That's 
kind of weird, but okay, I guess...)

> too large to fit in a single HTTP response

conveniently?

There's no limit to HTTP response size, is there?

> Single-page resource

I still think this is terribly confusing name.

Partial page?   Sub-page?     Sequence-page?    Traversal Page?

All of those would be much better.

> LDP Paging Clients

I have a real problem with this, but I don't think I can explain this yet. 
  Basically, there have to be affordances for everything in HTTP, and 
there's none here. 

I think the actual rule is: EVERY *LDP CLIENT* MUST TRANSPARENTLY HANDLE 
FORWARD AND BACKWARD TRAVERSAL.

Anything less means that servers can't initiate paging without breaking 
some clients.  IE they can't initiate paging.

Without that rule, the only time to do paging is when the client asks for 
it.

> 4.1.1 LDP Paging clients MUST be paging-aware in cases where LDP Paging 
servers initiate paging. 

Aside from the above point, I can't figure out what this could mean.
 
> read committed isolation ...

I happen to find these links make it all much more complex and confusing 
than necessary.   The behavior we actually want is simpler than this 
sounds.

> 333

I'd think we should use a 2xx code as the placeholder, probably 209.

> Example 1

I think this would be more clear as an HTTP dialog trace, showing the 
actual headers.....

> any resource (LDP-RS or not) as a paged resource. 

What's the benefit of non-RS paging?    Do we say how jpg and html and zip 
files can be paged...?   Are we suggesting they can be?

> 5.2.3.8 LDP Paging servers MAY add or remove single-page resources to a 
paged resource's sequence over time, but SHOULD only add pages to the end 
of a sequence. 

Doesn't that contradict ordering?

> 5.2.3.11

Isn't that only for forward traversal?

> 5.2.3.13

Isn't this only for backward traversal?

> 5.2.3.15 LDP Paging servers MUST provide an HTTP Link header whose 
target URI is http://www.w3.org/ns/ldp#Page, ... This is one mechanism by 
which clients know that the resource is one of a sequence of pages. 

I don't think that's true.    The client knows because it got the URL via 
a link first/last/previous/next header.   Also the "canonical" link.

I don't see any value to this type, aside from not liking it's name.

> which would defeat the purpose of pagination

If that's all pagination is for, then we'd only use it with sorted 
resources.  I'm not sure what else paginating is for, but people seem to 
want it.  (I guess it's for client access to flow control, when it can't 
touch TCP.)

That's it for now.   *shrug*

I guess I could make these changes, if you want.    Probably a few issues 
that need WG discussion there, though.

        -- Sandro