Information Theory

Information theory measures the amount of information in a message by the average number of bits needed to encode all possible messages in an optimal encoding
SEX field in a database: only one bit of information (Male:0; Female:1)
Encoded in ASCII – more space, but no more information

Information theory measures the amount of information in a message by the average number of bits needed to encode all possible messages in an optimal encoding