Re: Streamability II

On Tue, Jun 22, 2004 at 02:40:06PM +0100, Seaborne, Andy wrote:
> 
> 
> Tom - you didn't explicitly say that the pages were held on disk at any time
> - are they?  Just sometimes?
> 
> Janne - While the server may be storing state in the server-side ResultSet
> that is paged, this just a matter of how Tucana have chosen to implement
> their system.  There does not appear to be in the application-contract that
> means server state is necessary.  The client-side ResultSet class is going
> over the results one at a time - the fact that there is an implementation
> that has the client pulling pages on demand is hidden.  The key is that the
> interface to the application is result-by-result. 
> 
> The same thing could be achieved with protocol that just pumps results as it
> can and let the network block if the client can't keep up.  In this case, no
> server state is involved (outside the lifetime of the query) in the query
> processor.  Might still choose in the server to record result to disk
> sometimes (e.g. consistency in the presence of updates).
> 
> If it's a big set of results and the client can't keep up with the server,
> then something somewhere has to block.  It may be in the query result
> protocol; it may be in the TCP stack.  I prefer to keep it in the TCP stack
> until we have a need otherwise for a web facing system.  There is a
> Denial-Of-Service attack point in creating TCP connections for operations
> then doing nothing with them but this is a well known problem.  DAWG isn't
> making it any different - it's the same as asking for large images and not
> reading them.

Doesn't that still leave the server locking tables or making a copy of
them?

Server finds a zillion matches to a query.
It could, before another thread starts, dump them out over the network.
The TCP window goes to 0. The server has to just sit there locked, or
copy the answers off so it can handle updates.
It seems like at that point, we'd rather have the flow control at a
higher level so we could be cleverer about sharing state with the client.
I think anything that scrolls in arbitrary time makes this scenario
more likely.

> -------- Original Message --------
> > From: Janne Saarela <>
> > Date: 22 June 2004 10:01
> > 
> > Hi Tom
> > 
> > > As promised, for those concerned about implementability, here is an
> > > outline from one of our engineers of our approach to streaming
> > > results across the wire: 
> > > 
> > > "When a remote query is performed, the ResultSet is wrapped in a
> > > paging class that can return a page of results at a time. On the
> > > client side is a class that meets the ResultSet interface and calls
> > > across the network to this paging class. When this client-side
> > > ResultSet class is returned from a query it contains its first page.
> > > Iterating over the client-side ResultSet just iterates over the
> > > results in the internal page. Once a page is finished, the next page
> > > is automatically requested from the paging class on the server and
> > > iterating can continue." 
> > 
> > Isn't this effectively an approach that stores the state for each
> > individual client on the server side? I would agree with Andy's
> > comment in
> > 
> > http://lists.w3.org/Archives/Public/public-rdf-dawg/2004AprJun/0606.html
> > 
> > where the requirement to store state for each client is
> > considered an unnecessary burden.
> > 
> > Have I understood correctly your concept of a 'paging class'?
> > 
> > Janne
> > --
> > Janne Saarela <janne.saarela at profium.com>
> > Profium, Lars Sonckin kaari 12, 02600 Espoo, Finland
> > Internet: http://www.profium.com

-- 
-eric

office: +81.466.49.1170 W3C, Keio Research Institute at SFC,
                        Shonan Fujisawa Campus, Keio University,
                        5322 Endo, Fujisawa, Kanagawa 252-8520
                        JAPAN
        +1.617.258.5741 NE43-344, MIT, Cambridge, MA 02144 USA
cell:   +1.857.222.5741 (does not work in Asia)

(eric@w3.org)
Feel free to forward this message to any list for any purpose other than
email address distribution.