Korva is a scroll of the internet’s best ideas.
All information reduces to bits
Information existed before Shannon, just as objects had inertia before Newton, but there was little sense of it as a measurable quantity, an object fitted out for hard science. Before Shannon, information was a telegram, a photograph, a paragraph, a song. After him it was abstracted entirely into bits, which he treated as its fundamental unit.
His 1948 paper introduced many of the concepts that underpin the modern digital world, and it is widely regarded as one of the most influential scientific papers of the 20th century. It laid the theoretical foundation for the internet, data compression, error-correcting codes, digital telecommunications, modern computing, and much of today's AI infrastructure.
All the advanced signal processing that lets us send high-speed data grew out of his work on information theory.
How did the word bit get coined?
John Tukey, a Princeton statistician who also worked at Bell Labs, coined bit as a short form of binary digit in 1947. Shannon credited Tukey with the word in his 1948 paper, which is the first published use. The unit measures the information in a choice between two equally likely options, such as a fair coin flip.
How does entropy measure information?
Shannon defined the entropy of a source as the average surprise of its messages, found by summing each outcome's probability times the logarithm of one over that probability. Rare events carry more bits than common ones, and a fair coin flip carries exactly one bit. The measure sets a limit on how far a message can be compressed without losing anything. Perfectly random data has maximum entropy and cannot be compressed at all.
How can codes defeat noise?
Shannon's noisy-channel coding theorem says every channel has a capacity, and below that rate codes exist that make errors as rare as you like. His proof showed such codes exist without constructing them. Practical codes that approach the limit took decades to arrive. Turbo codes, introduced in 1993, were the first practical codes to come close to it.
More from Founders
