W3C home > Mailing lists > Public > xml-dist-app@w3.org > December 2001

RE: [soapbuilders] FEEDBACK REQUESTED - Issues regarding Array encoding for SOAP 1.2

From: Andrew Layman <andrewl@microsoft.com>
Date: Thu, 13 Dec 2001 10:00:41 -0800
Message-ID: <C3729BBB6099B344834634EC67DE4AE102623C40@red-msg-01.redmond.corp.microsoft.com>
To: <xml-dist-app@w3.org>
Regarding partially transmitted arrays, nulls, and related matters, you
might be interested in three mails I wrote a few weeks ago to the SOAP
Builders mailing list, as well as subsequent discussion by others.


The full thread, including mails from others commenting on my analysis,
begins at

For the convenience of the XML Dist App archives, I copy the text of the
three messages I wrote here:  


I support Alan Kent's suggestion that untransmitted members be
equivalent to
nulls. Both an untransmitted member and a null token represent omissions
information. Both indicate that no definite value is asserted. Neither
a meaning different from the other.

I believe that any apparent distinction between nulls and omitted
is due to confusion brought on by the following two factors:

1. Reification of null due to incorrect reasoning from programming
Programming languages of the Algol ilk and similar define structure
types in terms of the structure members, meaning creation of a small
namespace representing a set of potential properties and their names
associated with the structure type. These programming languages somewhat
conflate the ideas of namespace, memory allocation and value. In
memory, a contiguous space is reserved, with space for each potential
value. Members that do not have a known value nonetheless have space
reserved. This is filled with something. When the distinction between a
known value and an unknown value is retained, as in Java Integer as
contrasted with Java integer, the lack of a value is represented by a
certain bit pattern, just as an actual value would be. This pattern is
called "null". This leads to the idea that a null is a value, rather
the lack of one. But this idea is an error: a null is the way that these
languages represent the lack of a stated value given that they must
some memory.
Other representations do not require the allocation of memory for
unactualized member values. For example, XML and Lisp allow omitted
The token "null" also functions in many programming notations much the
same way that the word "nothing" functions in English. That is, rather
saying "delete the value of the member x" we instead write that
as "x = null".
However, the fact that null appears in a programming syntax in a similar
position to where an actual value would appear reflects syntactic
convenience only, it does not make null into a value. Null represents
absence of information.

2. Nothing is not anything.
There is no definite semantics to the absence of information. Nothing
well, nothing. Any way of representing lack of information is going to
context-dependent interpretation. It may mean that the information is
simply not known, it may indicate that a default value is appropriate,
may indicate that no new information is provided and any prior value, if
any, should be used, etc. Consequently, the semantics of a null in one
context will differ from the semantics of an absent member in a
context. This obscures the more general point that both null and absence
a member are different notation's ways of distinguishing the presence
the absence of a stated value.

Sometimes, also, it is appealing to distinguish, within one context,
complete ignorance from use of a default value from indication of the
why no value is known from a statement that no value exists, etc. In
cases where exactly two aspects of ignorance are needed in a model, it
appealing to think of null as distinct from absence of a member. For
example, we could then distinguish whether we are stating that we don't
the patient's age from the statement that we are not asking to have the
patient's age changed. Without disputing that two distinguished tokens
useful, I observe that three, four and five are, also, but the
of programming and database languages, as they are used in practice, at
only make one distinction between presence and absence of a stated
That is, the Algol-style programming languages use null-notation and do
support omission of a member while XML supports omission-notation and
has no
null token.

Alan suggests, if I understand him correctly, that we should treat null
a omission of an element as representing the same thing. I suggest that
this is correct, and that the "same thing" represented is omission of a
stated value, and that the significance of the omission of a stated
may be dependent on the meaning of the structure from which the
is omitted and the process employing that structure.

One practical consequence of this is that nulls in a programming
(absent special, contextual knowledge of special semantics) should
appear in
XML as omission of the corresponding element, and visa versa.

Andrew Layman
http://strongbrains.com -- Resources for Self-Education


I'd like to add to my earlier mail [1] regarding the equivalence of
and omitted elements by making another distinction:
representation of data versus update of data.

This is important towards thinking about sparse arrays versus
partially-transmitted arrays.

But first, I'd like to look at the contrast between representation and
update as it applies to structures whose members are distinguished by
name. Consider a simple data structure for medical patient data:

Class Patient {
String name;
Boolean male;
Int age;

An instance of this might have values of its members respectively

{ "Joe Green", True, 46 }.

We might change some of those values by writing

patient.name = "Jo Green";
patient.age = 42;

In each of these cases, what is being described is fairly clear. First
have a structure with three member values (representation). Then we have
instructions to change two of those values (update).

But suppose I write the XML

<name>Jo Green</name>

Is that equivalent to

{ "Jo Green", NULL, 42 }

or to

patient.name = "Jo Green";
patient.age = 42;


If transmitted over a wire, does it result in a data structure having
{ "Jo Green", NULL, 46 } or { "Jo Green", True, 42 }?

That is, is the XML a *representation* of the current state of data or a
command to *update* the state of the data?

You cannot fully tell without knowing the context. Whereas the
language used different syntax to distinguish representing a value from
changing a value, XML only provides the former, representation of a
That is, you cannot determine merely by looking at some XML outside of
context what the purpose of the XML is.

Regarding ordinary structures, SOAP Section 5 only describes how to
translate between a non-XML data structure and an XML data structure. It
has no special rules for annotating or altering the data structure if
purpose is to be used in a command that updates data.

In particular, SOAP section 5 never advances the concept of a
"partially-represented" structured value. Consequently, we can say with
confidence that the XML described above represents the same structured
as the programming language would with { "Jo Green", NULL, 42 }.

Unfortunately, the specification is not quite so clear when it comes to
arrays. Section introduces the idea of a "partially transmitted
array". This is not the representation of all of any array, but only of
part of one, and this fact plus earlier discussions on this list [2]
make it
clear that one possible purpose of the partially-transmitted array is to
carry out a data-update command. Although the phrasing of section
on sparse arrays is not so ambiguous, I believe that a good case can be
for either side of the argument. That is, the spec and the discussion of
DCE cited above support reasonable arguments that sparse arrays are, or
not, for data update; are or are not for data representation.

This leads to a difficulty in interpreting a structure such as

<colors soap:arrayType="xsd:string[7]" >
<item soap:pos="[1]">red</item>
<item soap:pos="[7]">violet</item>

This might be a sparse array, that is, a complete data representation of
array with members [ "red", NULL, NULL, NULL, NULL, NULL, "violet"].

But it might also be a partially-transmitted array, that is, a
for the purpose of update, of only items 1 and 7, and leaving all other
elements untouched.

Both of these interpretations are possible given the SOAP 1.1
This is an unfortunate ambiguity that deserves to be cleaned up by the
1.2 effort at the W3C. I will give my opinion here on what I believe to
the best course.

If an array has any elements NULL, that is, without recorded values,
then it
is necessary to somehow represent this in XML. Using absent elements to
represent NULL is consistent with the interpretation given to absent
elements in structures and suggested earlier in this thread. [1]. Under
this interpretation, the XML structure

<name>Jo Green</name>

definitely represents the same facts as the programming language

{ "Jo Green", NULL, 42 }

and the XML structure

<colors soap:arrayType="xsd:string[7]" >
<item soap:pos="[1]">red</item>
<item soap:pos="[7]">violet</item>

represents exactly the same facts as does the programming language

[ "red", NULL, NULL, NULL, NULL, NULL, "violet"].

I do not deny that there is utility to being able to represent commands
update preexisting data structures. However, this is equally true of
arrays and structures whose members are distinguished by member name.
Rather than having a special form of array representation that might be
interpreted as an update command, and having this remarkable facility
available only for arrays, we would be better served to have (a) a
interpretation and representation for nulls/omitted-values that applies
both structures and arrays, and (b) decide that update commands and
semantic contexts for structures are beyond the immediate scope of SOAP
section 5.

In short, keep sparse arrays but make it very clear that omitted
elements in
XML correspond exactly to NULLs in programming languages, not to partial
update commands.

[1] http://groups.yahoo.com/group/soapbuilders/message/6194?threaded=1
[2] http://groups.yahoo.com/group/soapbuilders/message/5139?threaded=1

Andrew Layman
http://strongbrains.com -- Resources for Self-Education


Yet more on nulls and omitted elements.

Consider again a simple XML structure of medical patient data:

<name>Joe Green</name>

If stored into, or interpreted relative to, a structure layed out as

Class Patient {
String name;
Boolean male;
Int age;

It would be interpreted as

name male age
--------- --------- ---------
Joe Green NULL 46

But, if interpreted relative to a structure like

Class Patient {
String name;
Boolean male;
Int age;
String diagnosis;

Then it would be interpreted as

name male age diagnosis
--------- --------- --------- ----------
Joe Green NULL 46 NULL

And so on. There are two equivalent ways to look at this:

1. The XML structure only states values for certain members. When
interpreted in the context of a programming or database structure that
defines the potentiality for more members, the programming or database
structure will only have values where the XML structure has values, and
have nulls (indication of no stated value) associated with all other

2. The XML structure only states values for certain members. If the XML
structure is interpreted in the context of an XML schema with an <any>
particle, then there are an unlimited number of other subelements that
potential but not actual. This is equivalent to a programming structure
padded with an infinite number of members, of every possible name, all
which are filled with NULL markers, or a database row padded with an
infinite number of columns, of every possible name, all of which are
with NULL markers.

For this reason, it is important, when mapping an XML structure to a
programming structure, to map every omitted element to a NULL marker.
Unless the programming structure is defined by a finite number of
known to both the writer and the reader of the XML, it is not possible
rely on any explicit marking of an element to indicate that it is a null
marker. That is, while xsi:nil or similar attribute can work when data
structures are closed and agreed on by both reader and writer, they do
work in the broader circumstances. XML Schemas and many scripting
allow potentially unlimited members to be part of any structure.
Consider a
schema like

<type name="patient">
<element ref="name"/>
<element ref="male"/>
<element ref="age"/>
<any namespace="##other"/>

It is not possible to write an XML instance that enumerates the infinite
number of subelements without values. E.g. one cannot complete the

<name>Joe Green</name>
<a xsi:nil='1'/>
<aa xsi:nil='1'/>
<aaa xsi:nil='1'/>
<aaaa xsi:nil='1'/>

However, it is easily possible to omit the infinite number of
without values, to whit:

<name>Joe Green</name>

Omission must map to NULL. To support simple round-tripping, NULL must
to omission.

Andrew Layman
http://strongbrains.com -- Resources for Self-Education
Received on Thursday, 13 December 2001 13:02:01 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 22:01:17 UTC