This document describes a HTTP-based protocol for clients and servers to be able to efficiently retrieve large Linked Data Platform Resource representations by splitting up the responses into separate URL-addressable page resources.

The Working Group has addressed all raised issues, and other substantive changes have been made, including the decision to separate LDP Paging from the Linked Data Platform [[!LDP]]. A test suite has also been made available [[LDP-PAGING-TESTSUITE]].

Introduction

This specification provides a widely re-usable pattern to make the state of a large paged HTTP resource available as a list of smaller subset resources (pages) whose representations are less burdensome for servers to form. Paged resources must be LDP Resources (LDPRs) [[!LDP]]. Any LDPR can be paged, but certain aspects of paging like ordering are only well-defined for particular sub-classes of LDPRs, like LDP-RSs or LDPCs.

When a client attempts to retrieve a paged resource, the server either redirects the client to a "first page" resource or provides the representation of the "first page" resource in its response, depending on the client's request preferences and the server's capabilities. The response includes links to other page(s) in the sequence, as do subsequent pages. Paging-aware clients know how to combine pages of an LDP-RS, and possibly (via other specifications) other LDPRs. LDP Paging defines a mechanism by which clients can learn that the paged resource has been changed so that they can, for example, abandon a page traversal as early as possible. A detailed example of paging is provided in .

When a paged resource is also an LDPC, some additional efficiencies become possible in cases where the server predictably assigns members to pages and is able to communicate its assignment algorithm to clients. The defines a facility to communicate the sort order of member-to-page assignments, to handle that common implementation algorithm.

For context and background, it could be useful to read Linked Data Platform Use Case and Requirements [[LDP-UCR]].

Terminology

Terms re-used from the Linked Data Platform

This section is non-normative. It summarizes a subset of terms formally defined in [[LDP]] for the reader's convenience.

LDP server
A conforming HTTP server [[RFC7230]] that follows the rules defined by [[LDP]] when it is serving LDPRs and LDPCs.

LDP client
A conforming HTTP client [[RFC7230]] that follows the rules defined by [[LDP]] when interacting with a LDP server.

Linked Data Platform Resource (LDPR)
A HTTP resource whose state is represented in any way that conforms to the patterns and conventions in [[LDP]].

Linked Data Platform RDF Source (LDP-RS)
An LDPR whose state is fully represented in RDF [[LDP]].

Linked Data Platform Container (LDPC)
A LDP-RS representing a collection of LDPRs linked by membership triples and containment triples [[LDP]].

Membership triples
A set of triples that lists an LDPC's members [[LDP]].

Containment triples
A set of triples, maintained by an LDPC, that lists documents created by the LDPC but not yet deleted [[LDP]].

Terms normatively defined by this specification

The following terminology is based on W3C's Architecture of the World Wide Web [[!WEBARCH]], Hyper-text Transfer Protocol ([[!RFC7230]], [[!RFC7231]]) and Linked Data Platform [[!LDP]].

Paged resource
A LDPR whose representation may be too large for a server to construct in a single HTTP response (e.g. without running out of memory), for which an LDP Paging server offers a page sequence that allows clients to obtain subsets of its state over some period of time by making multiple HTTP requests.

For readers familiar with paged feeds [[RFC5005]], a paged resource is similar to a logical feed. Any resource could be considered to be a paged resource consisting of exactly one page, although there is no known advantage in doing so.

Page sequence
A sequence of related LDPRs P1 (first), P2, ...,Pn (last), each of which contains a subset of the paged resource's state.

When the representations of the sequence's resources are combined by a client, the client has a copy of the paged resource's state; if the paged resource changed while the client was retrieving the sequence's resources, then the client's copy of the paged resource's state can be incomplete or inconsistent with the state of the paged resource at any single instant in time. Thus, it is impossible to strongly guarantee that the result of this retrieval and combination process is the same state that the client would obtain using a single request (if that were possible), but LDP does provide a way for clients to detect when the paged resource's state changed during the retrieval process. As long as the paged resource's state did not change during the retrieval process, the client's copy of the paged resource's state will be as accurate as the server implementation allows it to be.

If a paged resource P is a LDP-RS, the representation of each in-sequence page resource contains a subset of the triples in P, and a client can merge those graphs to combine them. LDP allows paging of resources other than LDP-RSs, but does not specify how clients combine their representations. See for additional details.

For readers familiar with paged feeds [[RFC5005]], a page sequence is similar to a paged feed and many of the same consideration (echoed above) apply.

In-sequence page resource
One LDPR in a page sequence, which contains a subset of the state of another resource P, called the paged resource. For readers familiar with paged feeds [[RFC5005]], an in-sequence page resource is similar to a feed document.

Note: the choice of terms was designed to help authors and readers clearly differentiate between the resource being paged, and the individual page resources, in cases where both are mentioned in close proximity.

Note: page sequences are described and named with respect to how they are traversed starting from the paged resource, using only forward traversal. This does not imply anything more; the choice is arbitrary. For example, following forward links does not imply that resources later in the sequence are newer; the forward direction might correspond to moving forward or backward in time or along some other dimension, but any such relationship is server-specific. It is not implied by LDP Paging, absent additional concrete assertions like those defined later for LDPC representations.

first page link
A link to the first in-sequence page resource P1 (first) of a page sequence. The first page is the one that a LDP Paging server redirects to (303 response) in response to a retrieval request for the paged resource's URI. Syntactically, a HTTP Link <P1>; rel="first" header [[!RFC5988]].

next page link
A link to the next in-sequence page resource of a page sequence. Syntactically, a HTTP Link <Pi>; rel="next" header [[!RFC5988]] where the context URI identifies some Pi=1 (first)...n-1 (next to last) and the target URI identifies Pi+1.

last page link
A link to the last in-sequence page resource Pn (last) of a page sequence. The last page is the page that terminates a forward traversal, because it contains no next page link. Syntactically, a HTTP Link <Pn>; rel="last" header [[!RFC5988]].

previous page link
A link to the previous in-sequence page resource of a page sequence Syntactically, a HTTP Link <Pi>; rel="prev" header [[!RFC5988]] where the context URI identifies some Pi=2...n (last) and the target URI identifies Pi-1.

paging link
Any of the links defined by LDP Paging: first page links, next page links, last page links, previous page links.

forward traversal
The process of following next page links from some starting point.

Note: "forward" should not be read to mean anything more than a name for one direction through the page sequence. For example, following forward links does not imply that resources later in the page sequence are newer; the forward direction might correspond to moving forward in time, but any such relationship is server-specific not implied by LDP Paging absent additional concrete assertions like those defined later for LDPC representations.

backward traversal
The process of following previous page links from some starting point.

Note: "backward" should not be read to mean anything more than a name for one direction through the page sequence.

Conventions Used in This Document

The namespace for LDP Paging is http://www.w3.org/ns/ldp#.

Sample resource representations are provided in text/turtle format [[turtle]].

Commonly used namespace prefixes:

	
	@prefix dcterms: <http://purl.org/dc/terms/>.
	@prefix foaf:    <http://xmlns.com/foaf/0.1/>.
	@prefix rdf:     <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
	@prefix ldp:     <http://www.w3.org/ns/ldp#>.

The status of the sections of Linked Data Platform Paging 1.0 (this document) is as follows:

A conforming LDP Paging client is a conforming LDP client [[!LDP]] that follows the rules defined by LDP Paging.

A conforming LDP Paging server is a conforming LDP server [[!LDP]] that follows the rules defined by LDP Paging.

Example paging message exchanges

Example Co.'s customer relationship data is identified by the URI http://example.org/customer-relations, and is retrievable using the same URI.

Traditional flow without paging

A standard HTTP client that knows nothing about LDP paging obtains a representation of the resource in the usual way.

Request 
GET /customer-relations HTTP/1.1
Host: example.org
Accept: text/turtle

The server response is:

Response: 
HTTP/1.1 200 OK
Content-Type: text/turtle
ETag: "_87e52ce291112"
Link: <http://www.w3.org/ns/ldp#Resource>; rel="type"
Vary: Accept, Prefer
Allow: GET,OPTIONS,HEAD

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
@prefix dcterms: <http://purl.org/dc/terms/>.
@prefix foaf: <http://xmlns.com/foaf/0.1/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix o: <http://example.org/ontology/>.
@base <http://example.org/customer-relations>.

<>
   a o:CustomerRelations;
   dcterms:title "The customer information for Example Co.";
   o:client <#JohnZSmith>, <#BettyASmith>, <#JoanRSmith>,
            <#GlenWSmith>, <#AlfredESmith>. 

<#JohnZSmith>
   a foaf:Person;
   o:status o:ActiveCustomer;
   foaf:name "John Z. Smith".
   
<#BettyASmith>
   a foaf:Person;
   o:status o:PreviousCustomer;
   foaf:name "Betty A. Smith".
 
 <#JoanRSmith>
   a foaf:Person;
   o:status o:PotentialCustomer;
   foaf:name "Joan R. Smith".
 
<#GlenWSmith>
   a foaf:Person;
   o:status o:ActiveCustomer, o:GoldCustomer;
   foaf:name "Glen W. Smith".

<#AlfredESmith>
   a foaf:Person;
   o:status o:ActiveCustomer, o:BronzeCustomer;
   foaf:name "Alfred E. Smith".

As long as the resource being retrieved is small enough that the server can form its representation without running into implementation limits on memory etc., and as long as the client is likewise able to consume these representations, this pattern works fine. Indeed, it's by far the most common pattern on the Web.

At the point where server and/or client constraints or consumption preferences come into play, however, something else is needed. LDP Paging addresses this need by giving clients and servers the ability to partition representations into smaller pieces, transfer as many as the client needs (potentially a small subset of the resource's full state), and allows the client to reassemble the pieces (or not) as it prefers. One sees this pattern in contexts such as search page results, as one common example.

Simple paging flow using redirects

The simplest possible solution would be to redefine the meaning of the existing URI http://example.org/customer-relations from "all customer relations information" to "the first subset of all customer relations information". This would require changes to any existing clients whose code was built using the original definition however, so it's more likely that Example Co. would mint a new URI for "the first subset", define a way to find subsequent subsets, and have clients use this new "first subset" URI approach.

The "first subset" URI approach does not solve the problem of migrating existing clients from the old "all" to the new "first subset" semantic; neither would a 303 See Other redirect from the old to the new URI, given the widespread historical client practice of automatically following 303 redirects and treating the redirected-to resource as equivalent to the original, contrary to HTTP. The safe route is to have clients explicitly tell the server that they are capable of handling the "first subset" semantic on the redirected-to URI; this is what LDP Paging does.

A client aware of LDP paging obtains a representation of the resource in the usual way, and also signals its ability to handle redirection to a first-page resource:

Request 
GET /customer-relations HTTP/1.1
Host: example.org
Accept: text/turtle
Prefer: return=representation; max-triple-count="500"

The server's response contains a 303 See Other status code and a Location: http://example.org/customer-relations?page1 HTTP response header identifying the target resource, as shown below:

Response: 
HTTP/1.1 303 See Other
Vary:  Accept, Prefer
Location: <http://example.org/customer-relations?page1>

At this point the client does not know if the target resource is "all" or "the first subset"; it has to retrieve the resource to know.

Request 
GET /customer-relations?page1 HTTP/1.1
Host: example.org
Accept: text/turtle
Prefer: return=representation; max-triple-count="500"

The server's response is shown below:

Response: 
HTTP/1.1 200 OK
Content-Type: text/turtle
ETag: "_87e52ce291112"
Link: <http://www.w3.org/ns/ldp#Resource>; rel="type",
      <http://www.w3.org/ns/ldp#Page>; rel="type"
Link: <http://example.org/customer-relations?p=2>; rel="next"
Link: <http://example.org/customer-relations>; rel="canonical"; etag="customer-relations-v1"
Vary: Accept, Prefer
Allow: GET,OPTIONS,HEAD

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
@prefix dcterms: <http://purl.org/dc/terms/>.
@prefix foaf: <http://xmlns.com/foaf/0.1/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix o: <http://example.org/ontology/>.
@base <http://example.org/customer-relations>.

<>
   a o:CustomerRelations;
   dcterms:title "The customer information for Example Co.";
   o:client <#JohnZSmith>, <#BettyASmith>, <#JoanRSmith>. 

<#JohnZSmith>
   a foaf:Person;
   o:status o:ActiveCustomer;
   foaf:name "John Z. Smith".
<#BettyASmith>
   a foaf:Person;
   o:status o:PreviousCustomer;
   foaf:name "Betty A. Smith".
 <#JoanRSmith>
   a foaf:Person;
   o:status o:PotentialCustomer;
   foaf:name "Joan R. Smith".

From this response, the client knows that more data exists and where to find it. The server determines the size of the pages using application-specific methods not defined within this specification, with the client's max-triple-count preference as one input; the simplest method is to make the server's page size equal to the client's preference. In this example, the server chooses a smaller value so there is a second page.

The following example shows the message exchange for retrieving the next page:

Request 
GET /customer-relations?p=2 HTTP/1.1
Host: example.org
Accept: text/turtle
Prefer: return=representation; max-triple-count="500"

The server's response is shown below:

Response: 
HTTP/1.1 200 OK
Content-Type: text/turtle
ETag: "8_7e52ce291112"
Link: <http://www.w3.org/ns/ldp#Resource>; rel="type",
      <http://www.w3.org/ns/ldp#Page>; rel="type"
Link: <http://example.org/customer-relations>; rel="canonical"; etag="customer-relations-v1"
Vary: Accept, Prefer
Allow: GET,OPTIONS,HEAD

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
@prefix dcterms: <http://purl.org/dc/terms/>.
@prefix foaf: <http://xmlns.com/foaf/0.1/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix o: <http://example.org/ontology/>.
@base <http://example.org/customer-relations>.

<>
   o:client <#GlenWSmith>, <#AlfredESmith>. 
 
<#GlenWSmith>
   a foaf:Person;
   o:status o:ActiveCustomer, o:GoldCustomer;
   foaf:name "Glen W. Smith".

<#AlfredESmith>
   a foaf:Person;
   o:status o:ActiveCustomer, o:BronzeCustomer;
   foaf:name "Alfred E. Smith".

In this example, there are only two customers provided in the second page.

Linked Data Platform Paging Clients

All of the following are conformance rules for LDP Paging Clients.

General requirements

LDP Paging clients MUST advertise their ability to support LDP Paging on all retrieval requests that normally result in a response containing a representation.

LDP Paging clients MUST be capable of at least one of forward traversal and/or backward traversal.

LDP Paging clients MUST NOT assume that any in-sequence page resource's paging links will remain unchanged when the in-sequence page resource is retrieved more than once. Such an assumption would conflict with a server's ability to add pages to a sequence as the paged resource changes, for example.

LDP Paging clients MUST NOT assume that any in-sequence page resource's paging links will always be accessible.

Non-normative note: Such an assumption would conflict with a server's ability to remove pages from a sequence as the paged resource changes, for example. One consequence of this is that clients can receive responses with 4xx status codes when following page links, purely due to timing issues and without any error on the part of the client or the server. Such a response would be unusual, and would likely signal an error, if the response also indicates that the paged resource's state has not changed while the client was traversing its pages. Servers might also limit the lifetime of a particular page sequence, and client requests after the server has abandoned that sequence are likely to result in 410 Gone or other 4xx status codes.

LDP Paging clients SHOULD NOT treat a page sequence as equivalent to the paged resource when the paged resource changed during retrieval of the page sequence.

LDP Paging clients MUST NOT treat the target of a 303 See Other redirection as a replacement for the original resource. That is, they cannot treat a 303 status code as if it was a 307 Temporary Redirect [[!RFC7231]] or 308 Permanent Redirect [[!RFC7238]], as [[RFC7231]] makes clear. This is critical to a client's ability to distinguish between the representation of a single in-sequence page resource and that of the paged resource when a LDP Paging server uses redirection as a way to initiate paging.

Client preferences

LDP Paging clients must provide paging-related hints in order to influence an LDP Paging server's choices.

This specification introduces new parameters on the HTTP Prefer request header's return=representation preference [[!RFC7240]], used by clients to supply a preference that helps the server form a response that is most appropriate to the client's needs. The presence of any parameter defined in this section serves several purposes:

LDP clients SHOULD be capable of processing successful HTTP GET responses formed by a LDP server that independently initiated paging, returning a page of representation instead of full resource representation [[!LDP]].

LDP Paging defines max-triple-count, max-member-count, and max-kbyte-count as new parameters on the existing HTTP Prefer request header's return=representation preference [[!RFC7240]]; the presence of any of these parameters on the preference, not the preference alone, is what indicates to a server that the client supports LDP Paging. A client communicates its hint(s) to the server by adding the request header with a return=representation preference that includes any of the following preference parameters

max-triple-count The maximum decimal number of triples the client wishes to appear on each page.
max-kbyte-count The maximum decimal number of kilobytes (1024 byte units) the client wishes to receive as the page's representation.
max-member-count The maximum decimal number of members the client wishes to appear on each page. This parameter is only meaningful for paged LDPCs.

The generic preference BNF [[!RFC7240]] allows either a quoted string or a token as the value of a preference parameter.

Clients interested in receiving at most 500 RDF triples per page would use add this HTTP header on a GET request, as shown in : Prefer: return=representation; max-triple-count="500"

Linked Data Platform Resources

Paging considerations

As described previously, paging logically breaks up a paged resource into a list of in-sequence page resources (pages) whose representations the client can retrieve, and serves each page with links to other pages in the sequence. Clients inspect each response to see if additional pages are available; paging does not affect the choice of HTTP status code. Clients generally have no insight into the allocation of information to pages, although in some cases currently defined only for LDPCs the server can expose its algorithm.

Some clients will be interested in knowing whether or not competing requests altered the paged resource while the client was retrieving pages, since LDP paging does not guarantee that those alterations were reflected in any in-sequence page resource received by the client. Regardless of the server implementation, LDP only guarantees that while traversing a series of pages that the client observes data that is equivalent to a database that uses read committed isolation [[READ-COMMITTED]]; specifically, clients can observe non-repeatable reads [[NON-REPEATABLE-READS]] while traversing pages served by LDP Paging servers. LDP Paging does guarantee, however, that any information in the LDPR continuously present (neither added nor deleted) in the paged resource across the entire period of time when the client is retrieving pages will be present in at least one in-sequence page resource.

LDP Paging defines a mechanism by which clients can detect that the paged resource has been changed so that they can, for example, abandon a page traversal as early as possible. This provides a stronger guarantee in certain cases than the one described for paged feeds [[RFC5005]], but it does require the client to detect when the condition holds. Paging does not guarantee that it will ever be possible to successfully retrieve all the in-sequence page resources without the paged resource being added to or changed, so clients requiring such a guarantee may not find all paged resources usable in practice. A detailed example was provided in . See for server implementation considerations.

In order to allow web-based systems to correctly cache the responses of an LDP Paging server, the requirements of the current HTTP [[rfc2616]] and Prefer Header [[rfc7240]] specifications should be kept in mind. Notably, RFC7240 requires conforming implementations to a return a Vary header that includes at least Prefer in its value. As an LDP server must also support content negotiation for the format using the Accept request header, the response might look like: 

Vary: Accept, Prefer

HTTP GET

In addition to the requirements on HTTP GET for LDPRs [[!LDP]], LDP Paging servers must also follow the requirements in this section for all paged resources and their associated in-sequence page resources.

LDP Paging servers SHOULD allow clients to retrieve large LDP-RSs in pages.

LDP Paging servers MAY treat any resource (LDP-RS or not) as a paged resource.

LDP Paging servers MAY vary their treatment of any resource (LDP-RS or not) as a paged resource over time. In other words, given two attempts to retrieve the same resource at different points in time, the server can choose to return a representation of the first page at one time and of the entire resource at a different time. Clients distinguish between these cases based on the status code and response headers.

LDP Paging servers SHOULD respect all of a client's LDP Paging defined hints, for example the largest page size the client is interested in processing, to influence the amount of data returned in representations.

Non-normative note: LDP server implementers should be careful not to interpret a Prefer return=representation request header that lacks any parameters defined here as a client's request for a paged resource. A client's hints indicate LDP Paging support only when at least one of the preference parameters defined by this specification is present.
Non-normative note: LDP server implementers should carefully consider the effects of these preferences on caching, as described in section 2 of [[!RFC7240]].
Non-normative note: [[!RFC7240]] recommends that server implementers include a Preference-Applied response header when the client cannot otherwise determine the server's behavior with respect to honoring hints from the response content.

LDP Paging servers MAY ignore a page size hints whose value is zero, and process the request as if no maximum desired size was specified; in the latter case the server can select whatever page size it deems appropriate, or choose not to page the resource at all.

LDP Paging servers SHOULD NOT initiate paging unless the client has indicated it understands LDP Paging. LDP Paging servers initiating paging SHOULD respond to successful GET requests with any paged resource as the Request-URI in one of the following ways:

  • If the client supports paging, respond with status code 303 See Other and a Location response header that identifies the first in-sequence page resource. The only standard means defined by LDP paging for a client to signal a server that the client understands paging is via the client preference defined for this purpose; other implementation-specific means could also be used.
  • If the server is willing to provide a non-paged representation, respond with an appropriate status code (likely 200 OK) and the potentially large non-paged representation.
  • Either reject the request, most likely with a 4xx status code, or (keeping in mind the note below) initiate paging as described above with a 303 status code, or choose another status code appropriate to the specific situation.
Non-normative note: LDP Paging servers could choose to make any resource available only as a paged resource. In so doing, when interacting with clients unaware of LDP Paging, if the server initiates paging anyway then it runs the risk that an ill-behaved client will automatically follow a 303 See Other redirect and believe via the subsequent 200 OK that it has obtained a complete representation of the paged resource rather than of a single in-sequence page resource. LDP Paging clients will not follow redirects in this way, but some existing HTTP clients are known to treat 303 See Other redirects as if they were equivalent to the original request-URI, despite this being explicitly disclaimed in [[RFC7231]].

LDP Paging servers MUST ensure that all state present in the paged resource throughout a client's entire traversal operation is represented in at least one in-sequence page resource. In other words, whatever subset of the paged resource that is not added, updated, or removed during the client's traversal of its pages has to be present in one of the pages.

Non-normative note: As a consequence, if the paged resource does not change at all during the traversal, then the client has a complete view of its state as given by the negotiated response media type at the point in time when the final page was retrieved. If the paged resource does change during the traversal, then only the portions that were actually updated will differ; the client has no LDP Paging provided means for knowing in what way(s) its view differs from that of the server in this case.
Non-normative note: When the paged resource is an LDP-RS, the expectation is that all triples untouched by changes to the paged resource have been given to the client during the traversal; it is possible that some subset of the changed triples, including all or none of them, have been provided to the client, but the client has no way to know which.

LDP Paging servers MUST enable a client to detect any change to the paged resource that occurs while the client is retrieving pages by including a HTTP Link header on all successful HTTP GET responses, and SHOULD include the same header on all 4xx status code responses. The link's context URI identifies the in-sequence page resource being retrieved, target URI identifies the paged resource, link relation type is canonical [[!REL-CANONICAL]], and link extension parameters include the parameter name etag and a corresponding parameter value identical to the ETag [[!RFC7232]] of the paged resource. For example: Link: <http://example.org/customer-relations>; rel="canonical"; etag="customer-relations-v1"

Non-normative note: If the rel="canonical"; etag="..." value changes as the client retrieves pages, for example the value accompanying the first page's representation is rel="canonical"; etag="v1" and the value accompanying the second page's representation is rel="canonical"; etag="v2", the client can detect that the paged resource's state has changed. Some clients will ignore such changes, but others may choose to react to them, for example by restarting the traversal.

LDP Paging servers MAY add or remove in-sequence page resources to a paged resource's sequence over time. If the page sequence is ordered, then the ordering communicated by the server MUST be maintained; the server SHOULD only add pages to the end of an unordered sequence.

Non-normative note: Servers might abandon an ordered page sequence, resulting in 4xx status codes to subsequent requests, although it will be less disruptive to clients if the server either adds content to the appropriate existing page or adds new pages at the proper point in the sequence. Clients have no more efficient means than a conditional retrieval request on existing pages to detect the changed/added pages.
Non-normative note: As a result, clients retrieving any in-sequence page resource several times can observe its place in the sequence change as the state of the paged resource changes. For example, a nominally last page's server might provide a next page link when the page is retrieved again later. Similar situations arise when the page sequence crosses server boundaries; server A might host the initial portion of a sequence that links to the last page server A is aware of, hosted on server B, and server B might extend the sequence of pages. A nominally middle page M can become the last (or a non-existent) page if a competing request deletes part of the paged resource's content after the client retrieves M.

LDP Paging servers MAY provide a first page link when responding to requests with any in-sequence page resource as the Request-URI.

LDP Paging servers MAY provide a last page link in responses to GET requests with any in-sequence page resource as the Request-URI.

LDP Paging servers MUST provide a next page link in responses to GET requests with any in-sequence page resource other than the final page as the Request-URI. This is the mechanism by which clients can discover the URL of the next page.

LDP Paging servers MUST NOT provide a next page link in responses to GET requests with the final in-sequence page resource as the Request-URI. This is the mechanism by which clients can discover the end of the page sequence as currently known by the server.

LDP Paging servers MAY provide a previous page link in responses to GET requests with any in-sequence page resource other than the first page as the Request-URI. This is one mechanism by which clients can discover the URL of the previous page.

LDP Paging servers MUST NOT provide a previous page link in responses to GET requests with the first in-sequence page resource as the Request-URI. This is one mechanism by which clients can discover the beginning of the page sequence as currently known by the server.

LDP Paging servers MUST provide an HTTP Link header whose target URI is http://www.w3.org/ns/ldp#Page, and whose link relation type is type [[!RFC5988]] in responses to GET requests with any in-sequence page resource as the Request-URI. This is one mechanism by which clients know that the resource is one of a sequence of pages.

LDP Paging servers SHOULD indicate that they have abandoned a sequence by responding with at 410 Gone to HTTP requests to any of the in-sequence page resources with appropriate paging link response headers, for example, rel="first".

LDP Paging servers SHOULD support the max-triple-count client preference parameter, which expresses a page size limiting the number of RDF triples represented on a page. For example, max-triple-count="500" expresses a limit of 500 RDF triples per page.

LDP Paging servers SHOULD support the max-kbyte-count client preference parameter, which expresses a page size limit as kilobytes of representation size. For example, max-kbyte-count="1" expresses a limit of 1024 bytes per page.

LDP Paging servers MUST, if provided with multiple client preference parameters limiting page size, honor all hints that it recognizes. This has the net effect that the most restrictive hint for any given response governs the resulting page size. For example, max-kbyte-count="1" and max-triple-count="500" usually would result in pages whose size hits the 1 kilobyte limit first, since each triple very likely requires more than two bytes (500 triples/1024 bytes).

Linked Data Platform Containers

Requirements when paging LDP Containers

LDP Paging servers MUST ensure that the membership triple and containment triple for each member are part of the same in-sequence page resource, whenever both triples are present in the page sequence for a paged LDPC.

LDP Paging servers SHOULD support the max-member-count client preference parameter, which expresses a page size limiting the number of members returned on each page of a paged LDPC. For example, max-member-count="10" expresses a limit of 10 members per page.

Ordering of container members across pages

Ordering

There are many cases where an ordering of the members of a container is important. [[!LDP]] does not provide any particular support for server ordering of members in containers, because any client can order the members in any way it chooses based on the value of any available property of the members. In the example below, the value of the o:marketValue predicate is present for each member, so the client can easily order the members according to the value of that property. Applications that require a way for clients to specify the order, can do so with application-specific extensions.

Order becomes more important when containers are paginated. If the server respects ordering when constructing pages, clients needing a globally sorted set of members can reduce the effort required to merge pages. In cases where ordering is important, an LDP Paging server ensures that all the members on any single page have the proper sort order with relation to all members on any next and previous pages. This says nothing about the ordering of members within any single page. When the sort is ascending, all the members on a current page have a sort order no lower than all members on the previous page and sort order no higher than all the members on the next page; that is, it proceeds from low to high, but keep in mind that several consecutive pages might have members whose sort criteria are equal. When the sort is descending, the opposite order is used. Since more than one value may be used to sort members, the LDPC specification allows servers to assert the ordered list of sort criteria used to sort the members, using the ldp:pageSequence HTTP link relation and its ldp:pageSortCriteria predicate. Each member of the ordered list exposes one ldp:pageSortCriterion, consisting of a ldp:pageSortOrder, ldp:pageSortPredicate, and optionally a ldp:pageSortCollation.

Here is an example asset container described in [[LDP]] section 5.1:

Request 
GET /netWorth/nw1/assetContainer/ HTTP/1.1
Host: example.org
Accept: text/turtle
Prefer: return=representation; max-triple-count="500"

The server might respond with status code 200 OK, and the following representation:

Response: 
HTTP/1.1 200 OK
Content-Type: text/turtle
ETag: "_87e52ff291112"
Link: <http://www.w3.org/ns/ldp#Resource>; rel="type"
Link: <http://www.w3.org/ns/ldp#DirectContainer>; rel="type"
Vary: Accept, Prefer
Allow: GET,OPTIONS,HEAD

# The following is the ordered representation of
#   http://example.org/netWorth/nw1/assetContainer/

# @base <http://example.org/netWorth/nw1/assetContainer/> .
@prefix dcterms: <http://purl.org/dc/terms/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix o: <http://example.org/ontology/>.

<>
   a ldp:DirectContainer;
   dcterms:title "The assets of JohnZSmith";
   ldp:membershipResource <http://example.org/netWorth/nw1>;
   ldp:hasMemberRelation o:asset;
   ldp:insertedContentRelation ldp:MemberSubject;
   ldp:contains <a1>, <a2>, <a3>.

<http://example.org/netWorth/nw1>
   a o:NetWorth;
   o:asset <a1>, <a3>, <a2>.

<a1>
   a o:Stock;
   o:marketValue 100.00 .
<a2>
   a o:Cash;
   o:marketValue 505.00 .
<a3>
   a o:RealEstateHolding;
   o:marketValue 100.00 .

The client knows that LDP Paging was not used to form the response, because the HTTP status code is 200 OK; the absence of a Link: <http://www.w3.org/ns/ldp#Page>; rel="type" response header provides redundant confirmation of this. Since paging was not used, the server provides no information related to page sorting; providing page sorting information would have no value to the client, it would only waste resources.

Had the server responded instead with a redirect to a first page http://example.org/netWorth/nw1/assetContainer/?p=1, whose representation was (after following the redirect):

Response: 
HTTP/1.1 200 OK
Content-Type: text/turtle
ETag: "_87e52ff291112"
Link: <http://www.w3.org/ns/ldp#Resource>; rel="type",
      <http://www.w3.org/ns/ldp#Page>; rel="type"
Link: <http://example.org/netWorth/nw1/assetContainer/?p=2>; rel="next"
Link: <http://example.org/netWorth/nw1/assetContainer/>; rel="canonical"; etag="v1"
Link: <http://example.org/netWorth/nw1/assetContainer/sortedSequence/>; rel="http://www.w3.org/ns/ldp#pageSequence"
Vary: Accept, Prefer
Allow: GET,OPTIONS,HEAD

# The following is the ordered representation of
#   http://example.org/netWorth/nw1/assetContainer/ page 1 of 2

@base <http://example.org/netWorth/nw1/assetContainer/> .
@prefix dcterms: <http://purl.org/dc/terms/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix o: <http://example.org/ontology/>.

<>
   a ldp:DirectContainer;
   dcterms:title "The assets of JohnZSmith";
   ldp:membershipResource <http://example.org/netWorth/nw1>;
   ldp:hasMemberRelation o:asset;
   ldp:insertedContentRelation ldp:MemberSubject;
   ldp:contains <a1>, <a2>, <a3>.

<http://example.org/netWorth/nw1>
   a o:NetWorth;
   o:asset <a1>, <a3>, <a2>.

<a1>
   a o:Stock;
   o:marketValue 100.00 .
   
# The remainder of the content describes the sort order.
# Note that the new base URI here matches the target URI of the pageSequence Link header above.

@base <http://example.org/netWorth/nw1/assetContainer/sortedSequence/> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.

<> ldp:pageSortCriteria  ( <#Sort-o.marketValue-Ascending> ).

<#Sort-o.marketValue-Ascending>
   a ldp:pageSortCriterion;
   ldp:pageSortOrder ldp:Ascending;
   ldp:pageSortPredicate o:marketValue.

In this case the server does provide information on the assignment of members to pages.

  • A Link: rel="http://www.w3.org/ns/ldp#pageSequence" response header has been added, allowing the client to where to find information about the page sequence; its content includes a ldp:pageSortCriteria triple, allowing the client to know what sort criteria the server used when allocating members to pages.
  • The server also chose to include assertions about the sort criteria in the content for the first page, namely the triples occurring after the second @base directive. This is an optional exemplary optimization, and the client must decide for itself using means outside of LDP and HTTP whether or not to trust those assertions. If the client trusts those assertions, it need not retrieve them; retrieving them would in this case result in another HTTP GET request. For the purposes of this example, assume that the assertions match what the client would obtain by retrieving them itself.
  • The contents of the sort criteria resource asserts that the o:marketValue predicate will be used to assign sets of members to pages in ascending order. The server is telling the client that the values of o:marketValue of all assets on the first page are no lower than the values on subsequent pages. Since only one such value happens to fall on the first page, this is trivially satisfied.

When the client retrieves the second page by following the rel="next" link, its representation might be:

Response: 
HTTP/1.1 200 OK
Content-Type: text/turtle
ETag: "_87e52ff291112"
Link: <http://www.w3.org/ns/ldp#Resource>; rel="type",
      <http://www.w3.org/ns/ldp#Page>; rel="type"
Link: <http://example.org/netWorth/nw1/assetContainer/?pageSortOrder>; rel="prev"
Link: <http://example.org/netWorth/nw1/assetContainer/>; rel="canonical"; etag="v1"
Link: <http://example.org/netWorth/nw1/assetContainer/sortedSequence/>; rel="http://www.w3.org/ns/ldp#pageSequence"
Vary: Accept, Prefer
Allow: GET,OPTIONS,HEAD

# The following is the ordered representation of
#   http://example.org/netWorth/nw1/assetContainer/ page 2 of 2

@base <http://example.org/netWorth/nw1/assetContainer/> .
@prefix dcterms: <http://purl.org/dc/terms/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix o: <http://example.org/ontology/>.

<a2>
   a o:Cash;
   o:marketValue 505.00 .
<a3>
   a o:RealEstateHolding;
   o:marketValue 100.00 .
  • The same Link: rel="http://www.w3.org/ns/ldp#pageSequence" response header is present, allowing the client to know what page sequence, and hence what sort criteria, the server used.
  • The server chose not to include assertions about the sort criteria in the content for the second page. Since LDP Paging requires that the page sequence is sorted consistently, any client that already retrieved the first page has already seen the sorting criteria. If a client is given the URI of an in-sequence page resource through means other than following links, it always has the option to retrieve the sort criteria.
  • As the example shows, the values of o:marketValue all assets on the second page are greater than or equal to the values on earlier pages. The values within this page are not ordered according to the sort criterion, illustrating that this criterion applies only to the sorting of "assigning members to pages", not to the ordering within any single page's representation. Typically those representations have not guarantee of ordering and are generated by libraries, whose serializers do not provide a way to control how the order is conveyed.

It is up to the domain model and server to determine the appropriate predicate to indicate the resource’s order within a page (or globally), and up to the client receiving this representation to use that order in whatever way is appropriate to meet its needs, for example to sort the data prior to presentation on a user interface. Also, as it is possible for a container to have as its members other containers, the ordering approach doesn't change as containers themselves are LDPRs and the properties from the domain model can be leveraged for the sort criteria.

HTTP GET requirements for member ordering across pages

If a LDP Paging server chooses to use LDP Paging-defined mechanisms to tell clients the order it uses to assign LDPC members to pages, which is optional, then it does so as described in this section. LDP Paging does not specify ordering for pages in other cases. See for exemplary message exchanges.

LDP Paging servers MAY communicate the order used to allocate LDPC members to pages. If they choose to communicate the order, they MUST respond consistently to retrieval requests for each page in the same sequence. That is, they must communicate the same criteria (or absence of it) for all pages in the same sequence; if the criteria were allowed to vary, the ordering among members of a container across pages would be undefined.

Non-normative note: A server can offer different sequences for the same paged resource, for example, sequences that have differing sort criteria, and they can be offered serially or concurrently.

LDP Paging servers communicating order MUST expose page sort criteria resources that describe the allocation of members to pages; each resource consists of a rdf:List of ldp:pageSortCriterion resources. The first list entry provides the primary sorting criterion, any second entry provides a secondary criterion used to order members considered equal according to the primary criterion, and so on. The resulting page sort order MUST be as defined by SPARQL SELECT’s ORDER BY clause [[!sparql11-query]].

LDP Paging servers communicating order MUST expose page sort criteria resources that contain, in every ldp:pageSortCriterion list entry, a triple whose subject is the sort criterion identifier, whose predicate is ldp:pageSortPredicate and whose object is the predicate whose value is used to order members between pages (the page-ordering values). The only literal data types whose behavior LDP constrains are those defined by SPARQL SELECT’s ORDER BY clause [[!sparql11-query]]. Other data types can be used, but LDP assigns no meaning to them and interoperability will be limited.

LDP Paging servers communicating order MUST expose page sort criteria resources that contain, in every ldp:pageSortCriterion list entry, a triple whose subject is the sort criterion identifier, whose predicate is ldp:pageSortOrder and whose object describes the order used.

LDP defines two values, ldp:Ascending and ldp:Descending, for use as the object of this triple. Other values can be used, but LDP assigns no meaning to them and interoperability will be limited.

LDP Paging servers communicating order MAY expose page sort criteria resources that contain, in any ldp:pageSortCriterion list entry, a triple whose subject is the sort criterion identifier, whose predicate is ldp:pageSortCollation and whose object identifies the collation used. The ldp:pageSortCollation triple MUST be omitted for comparisons involving page-ordering values for which [[!sparql11-query]] does not use collations.

LDP defines no values for use as the object of this triple. While it is better for interoperability to use open standardized values, any value can be used.

When the ldp:pageSortCollation triple is absent and the page-ordering values are strings or simple literals [[!sparql11-query]], the resulting order is as defined by SPARQL SELECT’s ORDER BY clause [[!sparql11-query]] using two-argument fn:compare, that is, it behaves as if http://www.w3.org/2005/xpath-functions/collation/codepoint was the specified collation.

When the ldp:pageSortCollation triple is present and the page-ordering values are strings or simple literals [[!sparql11-query]], the resulting order is as defined by SPARQL SELECT’s ORDER BY clause [[!sparql11-query]] using three-argument fn:compare, that is, the specified collation.

Security Considerations

As with any protocol that is implemented leveraging HTTP, implementations should take advantage of the many security-related facilities associated with it and are not required to carry out LDP operations that may be in contradistinction to a particular security policy in place. For example, when faced with an unauthenticated request to replace system critical RDF statements in a graph through the PUT method, applications may consider responding with the 401 status code (Unauthorized), indicating that the appropriate authorization is required. In cases where authentication is provided fails to meet the requirements of a particular access control policy, the 403 status code (Forbidden) can be sent back to the client to indicate this failure to meet the access control policy.

Paging LDPRs without maintaining server-side session state

Server implementers naturally have concerns when they are expected to maintain per-client state because of the scalability limitations that per-client state implies. LDP Paging carries no such requirement, although this may not be obvious at first glance. Since URLs are opaque to clients [[WEBARCH]], a server is free to encode the information required for it to know where to start the next page inside its next page links, for example.

Observe that in the preceding examples, the page n URIs are all of the form http://example.org/customer-relations?p=n, where n is 2..n. This is likely true in general practice. If the server allocates o:client representations to pages randomly, it's not obvious how to avoid keeping per-client state of one kind or another on the server while still sending all the representations on at least one page. In the common case where the list has an ordering (either a natural one or one imposed by the implementation) however, it is easy to avoid keeping per-client state on the server.

One trivial case is "fixed order"; if the server always sends o:client representations in the order #JohnZSmith, #BettyASmith, #JoanRSmith, #GlenWSmith, #AlfredESmith (or indeed, in any predictable order), then it can put the "last seen" identifier in the next page link of each in-sequence page resource as it is retrieved by each client. The "session state" is, in effect, kept by the client. In this case, the next page link in the first page might be: 

		Link: <http://example.org/customer-relations?resumeafter=JoanRSmith>; rel="next"

If the server also supports backward traversal, then the second page's previous page link might be: 

		Link: <http://example.org/customer-relations?resumebefore=GlenWSmith>; rel="next"

Keep in mind that this is an exemplary server implementation decision; it is not prescribed by LDP Paging, and other choices are certainly possible. As long as the URIs used in links are opaque to clients, any choice within the constraints of all normative references is permissible.

Acknowledgements

The following people have been instrumental in providing thoughts, feedback, reviews, content, criticism and input in the creation of this specification:

Alexandre Bertails, Andrei Sambra, Andy Seaborne, Antonis Loizou, Arnaud Le Hors, Ashok Malhotra, Bart van Leeuwen, Cody Burleson, David Wood, Eric Prud'hommeaux, Erik Wilde, Gregory McFall, Henry Story, John Arwe, Kevin Page, Kingsley Idehen, Mark Baker, Martin P. Nally, Miel Vander Sande, Miguel Esteban Gutiérrez, Nandana Mihindukulasooriya, Olivier Berger, Pierre-Antoine Champin, Raúl García Castro, Reza B'Far, Richard Cyganiak, Roger Menday, Ruben Verborgh, Sandro Hawke, Serena Villata, Sergio Fernandez, Steve Battle, Steve Speicher, Ted Thibodeau, Tim Berners-Lee, Yves Lafon

Change History

Summary