Re: The meaning of "representation"

Xiaoshu Wang wrote:

> First, Shannon quantifies the information to investigate communication. 

Yes.

> Second, information is embedded in a message, i..e, it is the 
> content of the message, yes?

No.  Assuming binary coding is used, the message is a sequence of bits. It 
is presumed that the sender and receiver agree in advance on the range of 
possible information values (my term, not Shannon's), that a given message 
might convey;  each distinct message essentially selects one of those 
values.    From Shannon's 1948 paper [1]:

"The fundamental problem of communication is that of reproducing at one 
point either exactly or approximately a message selected at another point. 
 Frequently the messages have meaning; that is they refer to or are 
correlated according to some system with certain physical or conceptual 
entities. These semantic aspects of communication are irrelevant to the 
engineering problem. The significant aspect is that the actual message is 
one selected from a set of possible messages."

-- and --

"If the number of messages in the set is finite then this number or any 
monotonic function of this number
can be regarded as a measure of the information produced when one message 
is chosen from the set, all
choices being equally likely."

So, messages convey information, in that (when successfully transmitted) 
they cause the sender to agree on a choice between N possibilities.  By 
the way, for Shannons purposes, information is quantified:  a message that 
allows you to choose one of 1024 possible possibilities is conveying more 
information than one that allows you to choose between just two:  the 
former is eliminating 1023 options while the latter is eliminating only 1. 
 That's why it takes at minimum 10 bits to convey the first message, but 
only one bit to convey the second.

Revisiting what Tim said on 25 November:

> Information has been quantified by Shannon, who allows us to 
> measure  it and so some math about it.  You can model it in 
> various ways.  one way is to imagine that I have  very little 
> idea of your state of mind, or your situation. Then you  you 
> send me information: you publish something I read on the web. 
> As a result of reading it, I have significantly cut down the 
> possibilities for what I imagine your state of mind to me.

> Yes, exactly.  I think that's a pretty good informal statement 
> of what I quoted from Shannon above, and is consistent with his
> usage in the rest of Shannon's paper.

The N bits sent on the wire allow you to get agreement between sender and 
receiver on a choice between at most 2^N possible choices.  The higher 
level significance of those choices is usually also the subject of 
agreement in advance between sender and receiver, even though Shannon 
didn't need to say much about that, since he was mainly concerned with 
reliable transmission of the bits (or other code).  So, when I return to 
you a text/html page with an HTTP 200, you get an entity body that is a 
sequence of bits.  You and I agree which sequence, of all the possible 
ones, I have sent you.  The HTTP specification then delegates to the 
specification for the text/plain media type to tell you more:  the page 
has a body, perhaps with certain paragraphs and headings.   So, with those 
specifications on hand, you and I agree that I have sent you an HTML 
document with certain constructs in it. 

By the way, I have been hoping to ground the TAG's discussion of 
versioning more firmly in this view of information as being choices among 
predetermined options, but that's a subject for a different permathread.

Noah






Then, you said the information is abstract

[1] http://plan9.bell-labs.com/cm/ms/what/shannonday/shannon1948.pdf
--------------------------------------
Noah Mendelsohn 
IBM Corporation
One Rogers Street
Cambridge, MA 02142
1-617-693-4036
--------------------------------------

Received on Tuesday, 4 December 2007 17:54:28 UTC