Strategy for merging NIR applications

Tim Berners-Lee (timbl@www3.cern.ch)
Tue, 10 Nov 92 16:42:26 +0100


Date: Tue, 10 Nov 92 16:42:26 +0100
From: Tim Berners-Lee <timbl@www3.cern.ch>
Message-Id: <9211101542.AA00530@www3.cern.ch>
To: Dan Connolly <connolly@pixel.convex.com>
Subject: Strategy for merging NIR applications
Cc: www-talk@nxoc01.cern.ch

At the end are some condensed thoughts about directions for merging  
applications.

|  Date: Tue, 03 Nov 92 14:08:54 CST
|  From: Dan Connolly <connolly@pixel.convex.com>
|  

|  >> Dan:
|  >>     Should WAIS clients grok URLs? I don't see any need.  Similarly for
|  >>     gopher clients.
|  > Tim:
|  >	If you want to keep the WAIS and Gopher worlds totally distinct,
|  >	then no.  But who wants to have all these separate clients around?
|  >	Noone: that is the big user puzzlement with NIR now -- why so
|  >	many systems when they all do the same thing? The first step
|  >	toward merging existing systems, *AND* allowing for future systems
|  >	smarter than the ones we have thought of now, is the URL.
|  Dan:
|  Well, you took the bait, but you didn't run with it very far.
|  "The first step towards merging..." -- so you _do_ have a deployment
|  strategy in mind. You just didn't put it in the RFC.

(Can you punt bait?)

The RFC is to define one piece of the whole. It is most important to
keep the peices distinct, or the whole thing will be too rigid. I suppose we  
could propose "All the world should use W3 and we should all discuss what W3  
should be like next" but that would never fly. We see that URLs are essential  
to allow communication between applications and different environments and for  
the introduction of new namespaces.

|  I'd like to see more motivation in the form of a strategy or at
|  least a vision for how this whole thing comes together. I'm trying
|  to piece it together myself, but I'm not having much luck.

That may be because the bits aren't available.  Like a global solid reliable
directory system, for example, and a global payment mechanism or two, and  
athentication systems to support it.  These just aren't globally deployed yet  
so we will have to wait for them and then plug them in.  What we do have is the  
internet, TCP/IP and DNS and we have to put hooks for everything else.

|  For example, I really don't see why different protocols should
|  share addressing schemes, or why the sheme:path syntax should
|  go any further than the WWW project.

It is the scheme:path syntax which allows the bridge. It need not be used
in closed applications, but in open applications it is needed. W3 will be able
to incrementally move to new name spaces. If other apps don't, too bad.
But I think they do.

|  >	Yes. Certainly a useful aspect of URLs. But not the only.
|  >	What about the URL pasteboard type? To allow "cut reference"
|  >	and "paste link" to work between applications?
|  

|  I've been trying to work out a design for this, and URLs don't seem
|  to help much. If you copy a WAIS URL and paste it into another
|  client, that client has to grok WAIS protocol in order to use it.

Or have access to any WAIS engine. There are 100 ways of keeping a
register for a user for what applications can handle what kinds of
namespace.  This will probably be something which is more difficult
to standardize on, as NeXT and Windows and Motif etc will all have different  
schemes for registering application capabilities, and different  
inter-application messaging techniques.

There is some analogy between capabilities to deal with UDIs and capabilities  
to deal with data formats here ... Mark Litwack (are you there, Mark?)
had a scheme in which the two merged.

|  Or it has to use a gateway. If it uses a gateway, it has to translate
|  the WAIS name into something suitable to that gateway, which is
|  something you said you didn't think was practical.

I don't think the client has to translate anything necessarily.
The http protocol allows one to send over a URL including the
prefix, which need not be http:.  For a protocol which does not deal
in generic URLs, some translation would be necessary.

|  Anyway, if you're
|  going to define a gateway system, then define a gateway system, not
|  just an addressing syntax.

The gateway system is a time differential of the set of protocols used.
It will always change to reflect the movements between protocols. I agree
that the HTTP protocol could perhaps have something in the spec to point out  
that the full UDI may be sent to a gateway.

|  I'm trying to figure out how URLs complete the picture, but I don't
|  see it.
|  


There will be a few things in the picture. URLs are the most important next  
step now, a timely requeirement for commonality. That's all. They will not  
solve all the problems.

You want a merging strategy. Suppose it is as follows, off the top of my head.

			  MERGING STRATEGY

The metastrategy fro developing strategies is to first, make a model of
a superset of the functionality of the various systems, then design protocols  
to implement them, then make applications which use both old and new. Beware  
that merging all the applications onto one is only one as seen from the user,  
hopefully there would be separately developed products on there with messaging  
between them. 


	Merging WWW and WAIS and Z39.50:

A new HTTP2 protocol is defined, back-compatible with the current one.
It involves sending a request which is a particular format. There are MIME-like  
formats defined for a simple document request, a basic WAIS-like query, and if  
necessary other Z39.50 queries. This means establishing a mapping between the  
Z39.50 parameters and an ASCII internet-mail-like format. We make the set of  
query types as open as the set of document format types and encoding methods,  
and allow for a registery of names like IANA.  The formats acceptable to the  
client are sent with the request [as mail header lines or in the body?].
The query types acceptable to a server are returned by the server when a  
queryable object is referenced.

We look at the parameters necessary for this process and add them to the WAIS  
protocol, so that Z39.50++ becomes a binary (and therefore perhaps more  
efficient) version of HTTP2. The binary and telnet-style versions run  
side-by-side, with fairly simple table-driven interfaces. The telnet version  
will be useful becasue of all those perl servers which will be eth leading  
edge.

	Merging with SQL:

This model will engulf query/retrieve systems, including e.g. SQL queries,
and allow for upgrade for new ideas of remote operations. In fact, the
query type will generalize to a typed set of input parameters for a arbitrary  
operation [method] on a remote object with the given URL, and so in principle  
can access a globally distributed OODB. In order to get a more manipulable  
version of the results of an SQL query or arbitrary OO operation, we will  
probably need a data tagging system [maybe ASN/1 based?] for structured  
[alpha]numeric data such as SQL handles, but this is just another data format,  
not our problem now. The question of converting data formats on parts of  
multipart messages [structures] will be addressed, nothing uinsurmountable. We  
would obviously like to see a mapping from any OOP's identifier space into URL  
space.

	Merging with Mail and News:

Now look at what we have. The presentation part of the W3 client is just the  
presentation part of MIME, with an added hypertext data class. MIME really  
should have picked HTML instead of text/richtext, but that we can add.
Now we address the intruiging area of the merging of mail, news, and NIR.
The same application must obviously be able to handle all of these, and they  
should all look the same to the user. That is, if you link an article to a list  
of articles, post it to a newsgroup, or mail it to a mailbox, the operation  
(adding message to list of messages) is beasically the same, and should be sen  
as such. Note that MIME has to be augmented with the addition of query type  
registration, there needs to be a format for the summarizing of capabilities  
(mailcap information) in with requests. The mailcap information needs some  
extension. A spinoff is that the HTTP protocol should run over mail too, which  
though excruciatingly slow is neat as a lot of people still use mail servers  
;-).  This occured to me when implementing
the W3 server.

The mechanisms underneath are currently many and varied in order to be optimal  
in the extreme cases of personal messaging, news broadcasting, and archive  
serving. However, as cache coherency protocols are added to HTTP, the functions  
like "give me a list of what is new in the following domain" will approach NNTP  
functions.  Within time it will become a decision made on the fly by the system  
as to whether a document [message, article] is held, sent, or broadcast. So I  
expect the protocols to converge.  An important issue here will be  
charactization of basic operations on sets [maillistarchives, newsgroups,  
directories].  The engines which speak the new combined protocol will have to  
speak the old ones too for back-compatibilty for a few years. This is
a smooth trasnsition strategy. We're not talking about vast librares, just  
supporting SMTP and NNTP and HTTP for a while (they are very similar) as well  
as a superset protocol.

	Stargazing

What else do we want to merge in?  Name servers of course are just HTTP2
servers which return "forward" results. File systems, I expect, too, will veer  
in the general direction. Alex-like systems will map the retrieval subset onto  
the subset. Maybe soft links could generalise to URLs. Maybe some smart  
manufacturers will allow a soft link to be a URL.  If not, file suffix which  
indicates a query or just .MIME would be enough to indicate that the document  
contains type information and could be a pointer or a query. However, there is  
basic functionality which the file sysyem does not support (That is, anything  
other than open, read write and close: no arbitrary methods here) so we will  
never map everything to fopen(). But by that time we'l all be using our  
favorite OO systems anyway. The exciting part will be to see whether the GUI  
builders will get to the point that the whole OO application can by built  
graphically, bridging the gap between the app builders and real programs (Can  
you imagine progamming in perl with a mouse? in Eiffel?)

	Tim
	

__________________________________________________________
Tim Berners-Lee                       timbl@info.cern.ch
World Wide Web initiative             (NeXTMail is ok)	
CERN                                  Tel: +41(22)767 3755
1211 Geneva 23, Switzerland           Fax: +41(22)767 7155