RE: updated proposal on issue #144 - array metadata in SOAP Encod ing

 Murali,
 I have some comments. 8-)

 > (not contiguous elements - otherwise it becomes partial)

 You seem to distinguish between sparse arrays (as in "relatively
few elements") and partially transmitted arrays. In my proposal
sparse arrays are taken as a subset of partially transmitted
arrays. I think your distinction is very artificial.

 I disagree with your conclusion that your usecases demonstrate
that

 > * There is a valid need for stamping an array as a sparse array.

 In my opinion it's true that

* There is a valid need for interpreting some arrays as sparse.

 In your usecases both client and server know to expect/send the
array as partially transmitted for it will result in much more
efficient representation. Also they know they can use a special
language construct instead of a normal array, again for the same
reason. So the application (client or server) can instruct its
SOAP implementation to send the array as sparse and to
deserialize the array into the special language construct. It
doesn't seem like your clients can benefit from the isSparse
flag.

 Anyway, as I showed with integers and bytes, I think using
attributes is the wrong and inconsistent approach here, using a
subtype is better. Subtypes also communicate the fact that sparse
array IS-A array.

 And even then, I don't like communicating a fact that is defined
very vaguely. With integers and bytes we have strict definitions
(of ranges), but given an instance of an array you can only say
that it's "very sparse" or "somewhat sparse" but there's no
dividing line.

 All this makes me think we'd best leave this decision on the
receiver and not try to communicate sparseness on the wire.

 Best regards,

                   Jacek Kopecky

                   Senior Architect, Systinet (formerly Idoox)
                   http://www.systinet.com/



On Tue, 13 Nov 2001, Murali Janakiraman wrote:

 > >>From: Jacek Kopecky [mailto:jacek@systinet.com]
 > >> I think the issue here is the true meaning of the term "sparse
 > >>array".
 > >> In math we have the term "sparse matrix" meaning "a matrix
 > >>containing mostly zeros". Mostly is a very vague word, I am not
 > >>aware of any stricter quantification of how many zeros must a
 > >>matrix contain to be considered sparse.
 > >>The same with SOAP arrays - how many members can be present for
 > >>the array to be still considered a sparse array?
 >
 > >>> [Murali]
 >
 >   I don't think we need to worry about the degree of sparseness. A
 > specification is only expected to provide a means to do something, if that
 > something is identified to be valid.
 >
 >   As a tools provider, I don't really make use of sparse arrays. However, I
 > have made an attempt here to try and provide a couple of use cases for
 > sparse arrays.
 >
 > Use case 1:
 >
 >   I think Sparse arrays as a separate type or as a data structure is used
 > mostly in the areas of representing spatial data, sonar, radar and
 > ultrasound imaging. They use sparse arrays because as you stated most of the
 > values are known or constant(0) or below or above some threshold.
 >
 >   In these cases, the user knows for sure it is a sparse array and they
 > benefit from using a specific implementation. They typically use separate
 > data structures and there are many algorithms and papers on how they can
 > optimize here. It seems to me that these use cases would benefit from
 > encoding an array as a sparse array as a sender of such an array would
 > expect the receiver to take advantage of the optimization that comes with
 > it.
 >
 >   It also seems to make sense in this case to make sparse array as a
 > separate type.
 >
 > Use case 2:
 >
 >   It seems that there are cases where a sender holds a regular array but
 > tries to encode only a few elements of the array (not contiguous elements -
 > otherwise it becomes partial) for what ever reasons that doesn't matter
 > here.
 >
 >   It also seems from some of the discussions in this list that there are
 > cases where a sender doesn't know before hand that the resulting
 > representation of the array is going to be sparse (Rich - is this your use
 > case?)
 >
 >   A possible example could be a monitoring system that is monitoring an
 > array of sensors and sending out different sensor values only if they are
 > above some threshold.
 >
 >   In a dynamic situation like this where the input feed is dynamic and the
 > output is streamed it is not possible to stamp the array as a sparse array
 > before starting to encode. Note again that for this to be true the output is
 > streamed meaning it has left the sender before the full array is encoded.
 >
 >   In this case a separate type for sparse array complicates matter and so
 > does an attribute that calls the array as a sparse array.
 >
 > Discussion:
 >
 > *  It seems to me that use case 1 is a valid case and users will benefit
 > from saying "enc:isSparse=Yes".
 >
 > *  Use case 2 seems to indicate a new state called "enc:isSparse=maybe" a
 > sparse array. Part of the users falling in this use case will still probably
 > be able to say a clean "enc:isSparse=Yes".
 >
 > *  I don't think there will be any arguments for a case where the answer is
 > "enc:isSparse=No" which clearly indicates a regular array representation.
 >
 >    I draw the following conclusions from the above.
 >
 > * There is a valid need for stamping an array as a sparse array.
 >
 > * Sparse array as a separate type for encoding has it's own issues.
 >
 > Proposal:
 >
 > * SOAP 1.1 encoding rules for sparse arrays (where position should be
 > present for all elements) seems to work fine for both cases.
 >
 > * As the recent proposal by Jacek has some advantages for some use cases and
 > if we are going to adopt that then it seems some of the user community would
 > be better served by,
 >
 >      1, either adding a new attribute enc:isSparse or
 > enc:encodedSparse(prefer this name though not that particular) with 3 states
 > (yes, no, maybe).
 >
 >      2, or make the enc:encodedSparse attribute optional with a bool
 > state(true/false). The absence of such an attribute would indicate a 'maybe'
 > state.
 >
 > I prefer the second as it leaves the burden to the user who really cares
 > about sparse arrays or regular arrays (penalize only those that require it).
 >
 > <<< [Murali]
 >
 > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Received on Wednesday, 14 November 2001 10:31:48 UTC