Information Theory

• Text files can be reduced by about 40% without losing information

• Because 1/p(x) decreases as p(x) increases, an optimal encoding uses short codes for frequently occurring messages; longer codes for infrequent

• Morse code E •, T –, J • – – –, Z – – • •