Re: Cache questions.

Hi,

On 20-Nov-98 Henrik Frystyk Nielsen wrote:
> At 14:30 11/20/98 -0600, olga wrote:
>>I understand the concepts of w3c design. Cache transparency it an important
>>issue. But the set of functions dealing with cache entries are made public.
>>Among them meta information update, finding and deleting cached entries and
>>so on (why are they public?).
> 
> You are absolutely right that they in fact shouldn't be public. The only
> reason is that for historic reasons, the cache related filters were put
> into another module and in order for these filters to access the cache,
> these methods had to be public. The last filter that I added in fact is
> local and so doesn't need any public functions.
> 
> Do you think it is worth fixing?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

        I think that it will make library less flexible. Though it the efficient
*pull* streams were impelented then it would be right thing to do. (At least for
my application. I would be happy not to mess with the cache...)



>>Also it would be very helpful is w3c could provide a function returning an
>>open stream handle which would allow operations like:
>>stream->ReadBytes(numBytes, buffer), stream->Reset(), stream->Seek(..). 
>>Implementing such functionality using w3c streams doesn't seem to be easy.
> 
> I have been going back and forth on this for a long time. One of the
> important differences between libwww streams and Java streams, say, is that
> the former are "push" streams whereas the latter are "pull" streams. That
> is, libwww streams are fed data automatically and can't refuse, pull
> streams actively read data from the next stream.

> However, a push stream can easily be converted to a pull stream by using
> some memory to buffer the contents. There is already a stream doing
> something close to this defined in 
> 
>       http://www.w3.org/Library/src/WWWStream.html
> 
> and that is the Stream to Chunk Conversion stream. I do actually believe
> that the HTChunk class should have methods like seek, get bag of bytes,
> etc. defined. I would be very interested if someone (you?) would help
> contributing this.
> 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

        I already looked at the possibility of using HTChunk. There are some
problems with that though:

        To allow seek and reset on HTChunk it should be kept in memory
completely, that is it can grow to the size of the document which is being
downloaded (in my app they can reach 30 MB). If many of such streams are kept
open simultaneously then we can run out of memory. 
        In push streams the putBlock or putChar functions are called by w3c
whenever data is available not when they are needed by client. So the data
should be buffered even if client would never ask for it.  
        Also such stream is in the middle between the w3c and application and
should work as both client and server. Some kind of interprocess communication 
should be used. 

        I think all this makes it inconvenient (and not easy) for user of w3c to
convert push stream into pull stream. And it would be inefficient. It would
me much better if w3c could provide ab-initio (and efficient) pull stream (not
to convert push into pull).

        My opinion is that pull stream is very necessary for w3c. (My colleagues
think the same.) As for contribution, I personally would not try to
implement such stream for w3c because I do not know internals of w3c well
enough to do a good job on that :-). But I vote for it to be implemented.
          
Thanks,

Olga Antropova.

Received on Monday, 23 November 1998 16:47:01 UTC