Re: Pagination (ISSUE-42)

Wow some good discussion here.

Thank you for all your considered view points.

I agree that the sever can return whatever upper limit on the range it
wants, just as requesting a collection with no explicit offset and limit
does. This is how I do it currently.

As for meta data I don't think I would have previous and next page but
perhaps rangeBefore and rangeAfter or not even that if a template is used.

The semantics of the collection seem to imply that a knowledgeable client
can request any page, yet the only links the current proposal defines are
the previous and next pages.  My clients need random access to a range of
the collection, so that is the key semantic I think that needs to be
captured however the parameters for such access are expressed. Paging
doesn't really convey the right meaning IMO.

It seems a template expressing the parameters be they page or offset and
limit is something that should be considered. If partial/random access
collections can be solved this way without imposing a page vs offset/limit
preference then I think that is preferable. Other schemes could also be
possible including using index/rank to identify ranges. Taken to its
logical conclusion paging/limit/offset is just filtering the collection as
other query params may do.

Thinking about the main collection query concerns there is filtering,
grouping, ordering, ranging and selecting. They are important enough to
segregate so that generic clients can be more intelligent. Describing
collections needs to cover off *how* those concerns can be expressed in
hydra without necessarily prescribing the particular syntax or style,
although some reference approaches to ranging may be described.

Apologies if I am not using the correct terminology. I'm only new to hydra,
and quite honestly not interested going down the RDF rabbit hole (or
climbing the ivory tower depending on your perspective). I want to use
json-ld to describe resources and links and potentially use hydra to
formalise my json APIs. I see this paging approach as too prescriptive and
not adequately capturing the semantic of accessing and operating on
partial/ranged collections.

For the caching argument, it probably would be more effective in terms of
cache utilisation for a dumb intermediate cache using a prescribed  page
size so that independent requests for a subset of the collection map to the
same resource. But honestly I don't think that it is reasonable to fix a
page size, or to arbitrarily limit the range of results returned; as others
have said it should be the sever that decides any upper limit (which may be
a dynamic concern). Some clients may want 10 items per page whilst another
wants 100 items per page. When demand loading content in the client and
allowing for 'buffers' either side of the viewable content region so that
smooth scrolling is possible, it seems any fixed page size will be less
than ideal most of the time (and result in more requests or too much data -
pick your poison) which is why I think  'fix sized paging has better
cachability' has almost negligible applicability in practice and results in
sub optimal behaviour most of the time. Besides, most applications these
days are using TLS for integrity and confidentiality guarantees which makes
them inherently uncachable.

Regards,

Andrew
 On 4 Feb 2015 08:03, "Thomas Hoppe" <thomas.hoppe@n-fuse.de> wrote:

>
> On 02/03/2015 06:58 PM, Gregg Kellogg wrote:
>
>> Gregg Kellogg
>>
>>> On 2015-02-03 16:31, Ruben Verborgh wrote:
>>>
>>>> Hi Andrew,
>>>>
>>>>  In my api's and clients I much prefer to use offset and limit as a
>>>>> more fundamental concept to partial collections.
>>>>>
>>>> This has strong repercussions of the cacheability of answers,
>>>> which is crucial when many clients consume the API.
>>>> (Graphs: http://linkeddatafragments.org/publications/iswc2014.pdf)
>>>>
>>>>  I've also disliked limit/offset endpoints because they empower users
>>> to query for very large portions of data unless the limit parameter is
>>> constrained.
>>>
>> IMO, using limit/offset is natural, and expressing them as URI query
>> parameters creates resource identifiers useful for first/last/previous/next
>> links. In my implementations, the first page of the collection typically
>> has no limit/offset parameters, but the rest do.
>>
>>
> I fully agree to this. And yes, In case the limit is too high, I think the
> server could just responding with HTTP 416 Requested range not satisfiable,
> for example.
>
>