nntp: newsgroup listings

Hi,

we have a severe problem with getting article listings for a given
newsgroup out of libwww (urls like "nntp://news.server.com/agroup"):

With the original version of HTNews.c the result of the XOVER nntp
command is most often empty - the nearer the nntp server the more
probable the xover fails (!).

In any case the SendCommand(request, news, "XOVER", buf) was
successful but it seems that libwww closes the socket prematurely
before reading the body of the return. The funny thing is that the
traces give no hint for any error, it just reports that the socket is
closed after reading <headersize> bytes.

I found a bad hack that enhances the probability of getting a valid
result: I added a little pause after the HTHost_read() call (see below).
Failures still happen randomly but it seems that enlarging the pause
doesn't help any further.

Any clue?

w3c-libwww-5.3.1/Library/src/HTNews.c line 519:
-----------------------------------------------
case NEWS_NEED_XOVER:
  if (!news->sent) {
      char buf[20];
      sprintf(buf, "%d-%d", news->first, news->last);
      status = SendCommand(request, news, "XOVER", buf);
      if (status == HT_WOULD_BLOCK)
          return HT_OK;
      else if (status == HT_ERROR)
          news->state = NEWS_ERROR;
      news->format = WWW_NNTP_OVER;
      news->sent = YES;
  } else {
      status = HTHost_read(HTNet_host(net), net);
      usleep(5000); /* <<<<<<<<<<<<<<<<<<<<<<<<<<< hack <<<< */
      if (status == HT_WOULD_BLOCK)
          return HT_OK;
      else if (status == HT_OK)
          news->state = NEWS_NEED_BODY;
      else if (status == HT_LOADED) {
          if (news->repcode/100 == 2)
              news->state = NEWS_SUCCESS;
          else {
              news->format = WWW_NNTP_HEAD;
              news->state = NEWS_NEED_HEAD;
          }
      } else
          news->state = NEWS_ERROR;
      news->sent = NO;
  }
  break;


Best regards,
 Volker
-- 
Volker Börchers, kidata AG, Königswinter, Germany

Received on Tuesday, 28 November 2000 09:42:46 UTC