Re: AFTF requirements list with comments, pre-2003/01/28 telcon

Mark Jones writes:

> 
> AFTFers,
> 
> Here is another version of our draft list.  I've added
> in-line the comments received so far on the
> requirements so we can more easily consider the
> feedback we've gotten.

Excellent, this is a terrific help, Thanks!
 
> I've also appended a summary of three new draft
> requirements recently proposed by Jeff Schlimmer
> (Microsoft) and commented on by Sanjiva and John
> Barton.
> 
> --mark
> 
> ________________________________________________________________
> 
> 
> Concrete Attachment Feature Requirements
> ----------------------------------------
> 
> Considerations
> --------------
> 
> * The specification should not invent a packaging scheme.
> 
> <barton 
href="//http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0027.html">
> Perhaps I don't quite understand the meaning of "packaging scheme" but
> the way I interpret this is "the ARTF is going to pick between SwA and
> DIME", which isn't truly possible since neither are sound enough.

This is asserted but seemingly unproven.

> Perhaps you mean The specification should resemble existing packaging
> schemes.
> </barton>

Still, I think the concern is legitimate.  I would go with:  "If existing 
packaging schemes meet the requirements, or represent sensible tradeoffs, 
then the specification SHOULD use such existing schemes."

 
> * The specification should aid debugging with simple tools.
> 
> <chris 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0025.html">
> 
> This has me baffled. What is it that you have in mind
> in the way of tools, and more specifically, are you
> suggesting that the specification would define said
> tools or that the specification would define a concrete
> binding that had as a design consideration that an
> implementation would be inherently debuggable? Further,
> what manner of "debugging" are we talking about here?
> 
> </chris>
> 
> <barton 
href="//http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0027.html">
> 
> I think this one was added at my suggestion.  I would word it:
>   The specification should rely on plain ASCII headers.
> Plain ASCII (no not internationalized) makes debugging
> message systems considerably easier.  Compare anyone's
> experience in working with HTTP on the one hand and
> RPC/Jini/Corba on the other.  Please note that ASCII
> does not mean unformatted ASCII.  Processing many small
> messages (~packet size) would benefit from fixed
> formats.
> 
> </barton>

I think this is too specific for requirements.  ASCII is a potential 
implementation technique.  I get nervous when we have requirements that 
don't deal with tradeoffs.  If we want the whole thing to be easy to 
inspect we can bag attachments entirely and go with text (ASCII or 
UNICODE) for data as well sa control structures.  I don't feel I know 
where the tradeoffs are between binary or text in all cases, and I would 
rather not make an absolute requirement.  I would prefer something like:

"The specification should, where reasonably practical, be designed to 
facilitate debugging, tracing, and other diagnostic activities."

> <markH 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0029.html">
> I think there's a (perhaps not clearly made)
> distinction between packaging scheme and attachment
> specification. My take on 'not invent a packaging
> scheme' is that the attachment specification will use
> an existing technology like MIME or DIME or ZIP (or tar
> or jar or ...) as the underlying packaging technology
> rather than inventing everything from the ground
> up. The attachment specification would describe how to
> use the underlying packaging scheme for packaging SOAP
> messages and attachments.
> </markH>
> 
> <markJ 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0028.html">
> John Barton suggested this one and his reply to your note
> captures his intention.
> </markJ>
> 
> 
> 
> General Requirements
> --------------------
> 
> R8. The specification must describe its relationship to the
>      properties defined in Table 1 (att:SOAPMessage and
>      att:SecondaryPartBag) in the SOAP 1.2 Attachment Feature
>      specification.
> 
> R9. The specification must describe its points of extensibility.
> 
> R15. The specification should not unnecessarily preclude convenient
>       description by languages such as WSDL.
> 
> <chris 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0025.html">
> 
> Hmmm... Why wouldn't the specification provide a
> normative WSDL binding extension mechanism? Afterall,
> what authority is better suited to define the extension
> than that which specifies the concrete binding itself?
> 
> Yes, I realize this is the XMLP WG and not the WSDL WG,
> but the WSDL WG is not chartered with the specification
> of all WSDL extensions, just the WSDL core syntax,
> processing model, extension points and framework.
> 
> It seems to me that not defining the WSDL binding
> extension for this feature would be like the XMLP
> defering a schema definition of SOAP to the XML Schema
> WG. Clearly, we would not do that, why would we defer
> the definition of the WSDL? 
> </chris>
> 
> <jean-jacques 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0026.html">
> 
> If nothing else, this may be a timing issue. WSDL is
> evolving rapidly; the SOAP 1.2 support is still in a
> state of flux; it will take a little while before
> things are stable enough for the ARTF so start dealing
> with this issue.
> 
> Also, it may well turn out that we need WSDL extensions
> for dealing with attachments. It might make sense to
> built them into the core.
> 
> </jean-jacques>
> 
> <markJ 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0028.html">
> 
> Jean-Jacques's reply touched on some of this.  Noah
> suggested the somewhat convoluted wording to try to
> convey the sense that WSDL is still evolving and that
> it may need to stretch a bit also.  (We won't
> necessarily need the flexibility, but this gives us a
> litle wiggle room.)
> 
> </markJ>

Yeah, I was going to agree with the original.  You've reminded me that it 
was mine :-), so I guess I at least get points for consistently.  I think 
JJM captured my concern with the proposal that we take responsibility for 
a normative WSDL binding.  If the XMLP group is still in existence as WSDL 
stabilizes, it might or might not be appropriate for XMLP to at that time 
own the normative WSDL binding.  In the meantime, I think we should 
explore the issues, sketch solutions, and maybe jumpstart soapbuilders on 
early experimentation.
 
> 
> DR17. The specification must work with the SOAP 1.2 HTTP binding and
>       with as many other bindings as possible.
> 
> 
> 
> Representation
> --------------
> 
> DR1. The specification must define a means to carry multiple data parts.
> 
> DR2. The specification must define a means for parts to carry
>      arbitrary data, including non-XML data (e.g., binary data and XML
>      fragments).
> 
> DR3. The specification must admit a reasonably time-efficient means of
>      identifying parts.
> 
> <chris 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0025.html">
> I think that rather than "identifying" this is intended
> to refer to resolving or dereferencing, no? If not,
> then I guess I don't understand the requirement's
> indended interpretation.
> </chris>
> 
> <markJ 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0028.html">
> "identifying" in this sense is more tied to finding
> parts in the packaging -- byte lengths, boundary
> strings, etc.
> </markJ>

I think it's all of the above.  I think we mean:

DR3:  The specification MUST support efficient implementation of:
a) parsing the phsyical representation to separate and identify its 
constituent parts.
b) programming systems which efficiently resolve a URI to retrieve the 
data (and metadata) comprising the corresponding part.
c) ...I think we need to decide whether to be more specific about random 
access, streaming, etc. as requirements...

 
> DR4. The specification must use a reasonably space-efficient
>      representation.
> 
> DR5. The representation must efficiently support the addition and
>      deletion of parts.
> 
> <chris 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0025.html">
> 
> Hmmm... While it is clear that an implementation of the
> specification would likely carry this requirement, it
> is less than clear that the requirement is applicable
> to the specification itself. Further, one would imagine
> that by this statement, it would be the intended to
> cover the insertion or in-line deletion of parts, or
> had you only appending and truncation in mind?
> 
> Again, it isn't clear that this requirement, as written
> is either testable of a specification or relevant for a
> specification that is not intended to be
> implementation-specific.  </chris>
> 
> <markJ 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0028.html">
> 
> The point here was to make the spec relatively friendly to
> intermediaries that might need to modify the attachment bundle in
> straightforward ways.  (roughly resonant with the fact that insertions
> and deletions of headers in a SOAP envelope are pretty straightforward
> syntactically, for example). 
> </markJ>

If that's the goal, then I think we need to specifically say:

DR5. The representation SHOULD efficiently support the addition and 
deletion of parts by intermediaries.

Otherwise, I agree completely with Chris' concern.  Indeed, I am somewhat 
nervous that even at the intermediary the issues will be hard to pin down, 
and may relate to higher level constructs that we can't control.  After 
all, if you write an application that has to inspect the whole message 
before deciding what to insert of delete, then you almost surely have to 
buffer the whole thing at the intermediary.  Once you've done that, then 
Chris is right on even at the intermediary.  How can you tell what is or 
isn't efficient for me at such a buffering intermediary?  I've very 
probably stored the parts in ways you wouldn't easily guess (e.g. some 
relational DB fields.)

> 
> DR13. The specification must provide support for large parts.
> 
> <chris 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0025.html">
> And small ones as well one would imagine. How large? Arbitrarily 
> large? Just "pretty big", really, really large" or "incomprehensibly 
> large"? :) 
> 
> What about parts who's size is not known at the time that 
> the serialization is begun? 
> </chris>
> 
> <markJ 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0028.html">
> These points have been discussed briefly.  This one needs more work.
> </markJ> 
> 
> <barton 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0030.html">
> 
> The reason for this kind of requirement is the dominant
> impact of I/O and memory allocation on performance.
> For small messages, all attachment scheme will be equal
> since CPUs are infinitely fast.  "Large" of course
> changes over time as hardware resources improve.
> Design for messages between 1MB and 1GB.  5 years from
> now, when this standard is in use, allocators can bite
> off 1MB but 1GB will likely still call for disk.  You
> can shift these numbers around, but they will factor
> into the design: might as well discuss them explicitly.
> 
> In my opinion, parts whose size is not known should not
> be "attached" to SOAP messages.  Rather one should use
> messages to set up an out of band stream mechanism.
> 
> </barton>

I think the question with small is, do you care about relative overhead? 
Is it OK to add 200 bytes of overhead to a 5 byte attachment.  In some 
situations the answer is:  yes, the whole message is still only a few 
hundred bytes and as John says, it's hard on modern processors to get in 
trouble processing a single small message.  On the other hand, if you have 
thousands of parts per message, or thousands of messages per second, the 
overhead can indeed really add up.  So, I don't think it's obviously a 
non-issue.
 
> 
> Reference to Parts
> ------------------
> 
> DR6. The specification must permit parts to be identified by URIs.
> 
> <chris 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0025.html">
> Hmmm... I think that the specification should require
> that parts be identified by URI, but that they may be
> identified using other means as well. Of course, they
> could be identified by relative URI, not just absolute
> URI.
> </chris>

+1 except for the references to relative URI.  I think we want:  The 
specification must provide that each part be identified by an (at least 
one) absolute URI.

I think issues of relative should be above our level.  If some system 
(e.g. SOAP itself) wants to provide base URI and resolve relatives to 
absolute, that's fine, but we don't worry about that I think.  I would not 
want a part to be known at the deepest level as "../p".
 
> <markJ 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0028.html">
> We can consider your wording instead.
> </markJ> 
> 
> 
> DR7. The URI identification scheme must be robust under the addition
>      and deletion of parts -- i.e., it must not require that URIs to
>      other parts be altered, it must be relatively easy to avoid URI
>      conflicts, etc.
> 
> DR11. (a) The specification should permit an initial human readable
>           part.
>       (b) The specification should not specify a particular ordering
>           of parts.
>       [still noodling on which version to prefer]
> 
> <chris 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0025.html">
> Not sure I follow this... 
> </chris>
> 
> <markJ 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0028.html">
> There was some sentiment for flexibility in part
> ordering -- for example, having a text part preceeding
> even the SOAP message.
> </markJ>

Right.  I also think the notion of "initial" is fuzzy.  Is it within the 
first 100 bytes?  Is it no binary data between the start of message and 
this initial part (so you can use text tools to get that far).  Does it 
preclude interleaving?  I think this is too specific and we should drop 
it.
 
> 
> DR12. The SOAP message part should be readily locatable/identifiable.
> 
> <chris 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0025.html">
> Should it not be the case that ALL parts be identified, identifiable? 
> What would make the SOAP part unique in this regard? 
> </chris>
> 
> <markJ 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0028.html">
> We wanted to make sure if there were multiple SOAP
> message parts that we could identify which one was the
> primary part and which were attachments.  This may be
> an issue if order were arbitrary, for example.
> </markJ>

+1 but reword as"

DR12. The primary (SOAP) message part should be readily 
locatable/identifiable.

I think this correctly layers the packaging abstraction (part) from its 
use by SOAP.
 
> 
> DR16. The part identifier scheme to be determined by sending
>       application.
> 
> <chris 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0025.html">
> "scheme" seems to imply "URI", but my guess is that it
> does not.  Again, I would strongly recommend that parts
> be identified by URI (relative or absolute).
> </chris>
> 
> <markJ 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0028.html">
> URI is what I have in mind.
> </markJ>
>

No.  I think that URI schemes should be used according to their 
definition.  This should not be a round-about way of enabling the caching 
scenario (if that's what's intended.)  Cachcing can be enabled with a SOAP 
feature (mapping an HTTP: URI to a CID:, for example).  The part in the 
message is unlikely to be correcly id'd directly with an HTTP URI (unless 
we're doing lazy pull through an http network.)

> ________________________________________________________________
> 
> New proposed requirements:
> --------------------------
> 
> DR18. The specification must define a means to format messages for
> down-level receivers that do not understand the specification.
> 
> <sanjiva 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0034.html">
> How can any spec say something about those who don't understand the
> spec? I'm confused.
> </sanjiva>
> 
> <barton 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0033.html">
> Maybe you can clarify this one Jeff...the way I read it, it sounds
> impossible.
> </barton>

I'm confused too.
 
> 
> 
> DR19. The specification must enable efficient allocation of buffers by
> receivers.
> 
> <sanjiva 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0034.html">
> I'm again confused; while a statement like "this spec
> must be implementable as efficiently as possible" is
> reasonable (and motherhood-and-apple-pie IMO), speaking
> specifically about buffer allocation seems rather
> pointed.
> </sanjiva>
> 
> <barton 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0033.html">
> 
> This one motivates some of the other requirements but
> it implies that the sender understand the receiver's
> memory allocation capabilities.  On one extreme the
> requirement could amount to "give the content length of
> attachments up front", but at the other extreme it
> could require the interleaving of parts to achieve a
> serialization optimal for receiver processing.
> 
> 
> As an example of the latter, the UPNP Printing folks
> worried about how an extremely long XHTML doc with many
> inline images could be a printed with one page buffer.
> While that may seem like an example far from the one
> most SOAP folks consider, once you get to pipelined
> processing of composed
> 
> SOAP services the differences begin to fade.  These are
> cases you want to be able to handle and they are cases
> that non-XML systems deal with.
> 
> Of course the serialization of XHTML is well-defined.
> Serialization for arbitrary receiver processing isn't.
> That makes this requirement difficult to spell out
> absent information on the receiver buffer capability.
> Consequently one might go for a requirement that asks
> the spec. to allow attachments to be placed in the
> stream physically near their first point of XML
> reference rather than getting into buffers.  That would
> pick up the critical use case without getting mired in
> an open-ended problem. 
> </barton>

I think we can say: "Attention should be given to likely implementation 
optimizations. I agree with Sanjiva, going much beyond that is too 
specific.)

> 
> DR20. The specification must allow messages to be secured using the
> mechanisms defined in WS-Security.
> 
> <sanjiva 
href="http://lists.w3.org/Archives/Public/xml-dist-app/2003Jan/0034.html">
> WS-Security only applies to SOAP envelopes. This
> requirement would hence have the effect of precluding
> MIME/DIME style packaging ..
> </sanjiva>

+1

------------------------------------------------------------------
Noah Mendelsohn                              Voice: 1-617-693-4036
IBM Corporation                                Fax: 1-617-693-8676
One Rogers Street
Cambridge, MA 02142
------------------------------------------------------------------

Received on Wednesday, 29 January 2003 11:08:49 UTC