Dan Kohn's Blog

In 1948, a 32-year-old at Bell Labs published a paper nobody fully understood.

Engineers found it too mathematical. Mathematicians found it too engineering-focused. One prominent mathematician reviewed it negatively.

That paper – “A Mathematical Theory of Communication”, became the founding document of the digital age.

The man was Claude Shannon. Father of Information Theory.

At 21, he wrote the most important master’s thesis of the 20th century.

Working at MIT on an early mechanical computer, Shannon noticed its relay switches had exactly two states – open or closed. He had just taken a philosophy course introducing Boolean algebra, which also operated on two values: true and false.

Nobody had ever connected these two things.

His 1937 thesis proved that Boolean algebra and electrical circuits are mathematically identical, and that any logical operation could be built from simple switches.

Howard Gardner called it “possibly the most important, and also the most famous, master’s thesis of the century.”

Every digital computer ever built traces back to this insight.

At 29, he proved that perfect encryption exists.

During WWII, Shannon worked on classified cryptography at Bell Labs. His work contributed to SIGSALY, the secure voice system used for confidential communications between Roosevelt and Churchill.

In a classified 1945 memorandum, he mathematically proved the one-time pad provides perfect secrecy, unbreakable not just computationally, but provably, permanently, against an adversary with infinite power.

When declassified in 1949, it transformed cryptography from an art into a science. It laid the foundations for DES, AES, and every modern encryption standard.

At 32, he defined what information is.

His 1948 paper introduced one equation:
H = −Σ p(x) log p(x)

Shannon entropy. The average uncertainty in a probability distribution. The minimum bits required to encode a message.

Three things followed:

> He defined the bit – the fundamental unit of all information. His colleague John Tukey coined the name.

> He proved the channel capacity theorem, every communication channel has a maximum rate of reliable transmission. You can approach it. You can never exceed it.

> He unified telegraph, telephone, and radio into a single mathematical framework for the first time.

Robert Lucky of Bell Labs called it the greatest work “in the annals of technological thought.”

Where his equation lives in AI today:

Cross-entropy loss – the function training every classifier and language model, is derived directly from H. Decision tree splits use information gain, which is H applied to data. Perplexity, the standard LLM evaluation metric, is an exponentiation of cross-entropy.

Every time a neural network trains, Shannon’s formula runs inside it.

He also built the first AI learning device.

In 1950, Shannon built Theseus, a mechanical mouse that navigated a maze through trial and error, learned the correct path, and repeated it perfectly. Mazin Gilbert of Bell Labs said: “Theseus inspired the whole field of AI.”

That same year he published the first paper on programming a computer to play chess. He co-organized the 1956 Dartmouth Workshop, the founding event of AI as a field.

The man:
He rode a unicycle through Bell Labs hallways while juggling. He built a flame-throwing trumpet, a rocket-powered Frisbee, and Styrofoam shoes to walk on the lake behind his house.

He called his home Entropy House.

When asked what motivated him: “I was motivated by curiosity. Never by the desire for financial gain. I just wondered how things were put together.”

In 1985, he appeared unexpectedly at a conference in Brighton. The crowd mobbed him for autographs. Persuaded to speak at the banquet, he talked briefly, then pulled three balls from his pockets and juggled instead.

One engineer said: “It was as if Newton had showed up at a physics conference.”

He died in 2001 after a decade with Alzheimer’s, the cruel irony of information slowly leaving the mind of the man who defined what information was.

Claude, the AI model, is named after Claude Shannon, the mathematician who laid the foundation for the digital world we rely on today.

From: https://www.facebook.com/share/p/1BAAXrxA2j/