#428 How Claude Shannon Worked

Claude Shannon changed your life and you probably don't even know it. Claude is the father of information theory. He transformed information into bits and created the intellectual foundation for computers, the Internet, digital media, high-speed communication, and Artificial Intelligence. The way Claude Shannon worked is what interested me the most. He combined extraordinary abstract reasoning with an engineer’s instinct to build. He reduced complex problems to their essential structure. He connected ideas across mathematics, engineering, cryptography, artificial intelligence, investing, chess, juggling, and robotics. He treated every experience as material that he might later combine into a new insight. He was also relentlessly independent, guided more by curiosity and freedom than recognition, money, or usefulness. This episode is what I learned from rereading A Mind at Play: How Claude Shannon Invented the Information Age by Jimmy Soni and Rob Goodman. Made possible by: Ramp: ⁠⁠https://ramp.com⁠⁠ AppLovin: ⁠⁠https://www.applovin.com Vanta: ⁠⁠https://vanta.com/founders 0:00 — "The great insights don't spring from curiosity alone, but from dissatisfaction... A genius is simply someone who is usefully irritated." 1:36 — "It seems to be much easier to make two small jumps than one big jump in any kind of mental thinking." — Shannon 4:41 — "He was a man almost entirely written out of a history that's defined by self-promoters. His was a life spent in the pursuit of curious, serious play." 5:20 — "He worked with levity and played with gravity. He never acknowledged a distinction between the two." 6:00 — "Geniuses are the luckiest of mortals because what they do is the same as what they most want to do." 6:59 — "I do what comes naturally, and usefulness is not my main goal. I keep asking myself, 'How would you do this? Is it possible to make a machine to do this? Can I prove this theorem?'" — Shannon 11:50 — "Chemistry always seemed dull to me. There's too many isolated facts and too few general principles for my taste." — Shannon 19:09 — "Specialization is the death of genius." — Vannevar Bush 19:18 — "The possibilities of being at once both broad and deep did not pass with Leonardo da Vinci or Benjamin Franklin." — Vannevar Bush 25:13 — "I had the freedom to do anything I wanted from almost the day I started. They never told me what to work on." — Shannon on Bell Labs 26:41 — "I've been working on three different ideas simultaneously, and strangely enough, it seems a more productive method than sticking to one problem." — Shannon 32:55 — "My fondest dream is to someday build a machine that thinks, learns, communicates, and manipulates its environment in a fairly sophisticated way." — Shannon 35:05 — "Before Shannon, information was a telegram, a photograph, a paragraph, a song. After Shannon, information was entirely abstracted into bits." 38:04 — "After the effort of discovery, the effort of communication was secondary by far. He had solved a problem to his own satisfaction, and that, as far as he was concerned, was enough." 40:55 — "I've spent lots of time on totally useless things... He made no distinction between his interest in information and his interest in unicycles. They were all moves in the same game." 46:54 — "Shannon seemed to think with ideas more than with words or formulas. A new problem was like a sculptor's block of stone, and Shannon's ideas chiseled away the obstacles until an approximate solution emerged." — Ed Thorp 47:19 — "I simply removed everything that was not David." — Michelangelo 47:25 — "I don't think I was ever motivated by the notions of winning prizes. I was more motivated by curiosity, never by the desire for financial gain. I just wondered how things were put together." — Shannon 47:49 — "I think the history of science has shown that valuable consequences often proliferate from simple curiosity." — Shannon 48:37 — "The important people and events of history are the thinkers and innovators, the Darwins, the Newtons, the Beethovens, whose work continues to grow influence in a positive fashion." — Shannon

• 0:00Speaker 00

  The great insights don't spring from curiosity alone, but from dissatisfaction. Not the depressive kind of dissatisfaction, but rather a constructive dissatisfaction, or a slight irritation when things don't look quite right. A genius is simply someone who is usefully irritated. A genius must delight in finding solutions. Genius must derive joy from applications of intellect. Claude Shannon once said, "I get a big kick out of seeing a clever way of doing some engineering problem, a clever design for a circuit which uses a very small amount of equipment and gets a great deal of results out of it." For Shannon, there was no substitute for the pleasure of seeing net results.

• 0:43Speaker 00

  So how would such a person go about solving a problem? Shannon proposed six strategies. Strategy number one, "You might," he said, "start by simplifying." Almost every problem that you come across is befuddled with all kinds of extraneous data of one sort or another. And if you can bring this problem down into the main issues, you can see more clearly what you're trying to do. Simplification is an art form in and of itself. It requires a knack for exercising everything from a problem, except what makes it interesting. Number two, encircle your problem with existing answers to similar questions, and then deduce what it is that the answers have in common.

• 1:27Speaker 00

  You'll need a vocabulary of questions already answered. You can call this ingenious incrementalism. As Shannon put it, "It seems to be much easier to make two small jumps than one big jump in any kind of mental thinking." Number three, restate the question. Change the words, change the viewpoint. Break loose from certain mental blocks which are holding you in certain ways of looking at a problem. Do not become trapped by the sunk cost, the work that you've already put in. There's a reason, after all, why someone who is quite green to a problem will sometimes solve it on their first attempt. They are unconstrained by the biases that build up over time.

• 2:10Speaker 00

  Number four, break an overwhelming problem into small pieces. Many proofs in mathematics have been actually found by extremely roundabout processes. Shannon pointed out that a man often starts out and proves many great results which don't seem to be leading anywhere, and then eventually ends up at the back door on the solution of his given problem. Number five, invert. If you can't use your premises to prove your conclusion, just imagine that the conclusion is already true and see what happens. Try proving the premises instead. And number six, take time and see how far it will stretch. Someone always comes along and starts generalizing it. So why not do it yourself?

• 2:55Speaker 00

  That is an excerpt from one of my favorite chapters in the book that I'm going to talk to you about today. That chapter is called "Constructive Dissatisfaction." The book that I'm going to talk to you about today is "A Mind at Play: How Claude Shannon Invented the Information Age," and it was written by Jimmy Soni and Rob Goodman. So even though Claude Shannon is one of the most important people to ever live in history, many people don't know his name. I've been telling friends this week the book that I'm reading, and they're like, "Who's Claude Shannon?" And the thing that resonated the most is like, "Well, Claude Shannon is the reason that Anthropic called Claude Claude." And so I actually broke down this book and my notes on the book in the way that Claude Shannon would break down a problem. So I essentially simplified what I think are the most important lessons, and then

• 3:38Speaker 00

  extracted them out of the book, and then organized them in a way I think is just going to be really interesting for you and I to go over. So one I want to start is, I'm not going to go over like, even though there's a biography of Shannon, and it's wonderfully written, I just, I'm not even going to talk about really much about his personal life, or his childhood, or anything else. There's just a certain way and an indifference to the outside world that I find very intriguing by Shannon. You and I have talked about this maximum over and over again, that you mute the world and then build your own. Well, Claude Shannon muted the world, built his own, and then in turn, by doing that, built the foundation of our world, the digital world that we live in now. So I want to start with just giving you an outline of this very unusual personality he had. He says, and there's a bunch of descriptions all throughout the book, about how he was,

• 4:23Speaker 00

  how he approached life, how he approached work, that I thought was interesting. Here's a few. "He was a man immune to scientific fashion and insulated from opinions of all kinds on all subject. A man of closed doors and long silences, who thought his best thoughts in Spartan apartments and empty office buildings. He was a man almost entirely written out of a history that's defined by self-promoters. His was a life spent in the pursuit of curious, serious play." The "a mind at play," that idea is so important. The book is perfectly titled. So this idea is like, "Hey, I'm spending my entire life just following my natural curiosity, my natural drift. I'm only going to work on things that I'm more interested in. I don't care what other people think." That is technically one of the main takeaways from this book.

• 5:07Speaker 00

  "His life was spent in pursuit of curious, serious play. He was that rare scientific genius who was just as content rigging up a juggling robot or a flamethrower trumpet," both inventions and actually things that he made, "as he was pioneering digital circuits. He worked with levity and played with gravity. He never acknowledged a distinction between the two. Rarely has a thinker who devoted his life to the study of communication been so uncommunicative." And so what they're talking about is the fact that he is the person that wrote, that came up with information theory. The book goes into great detail about how important it is. There's different ways to describe it. In many cases, it's very confusing. This is just a very simple way to describe the impact of the theory that Claude Shannon came up with.

• 5:48Speaker 00

  "All the advanced signal processing that enables us to send high-speed data was done as an outgrowth of Claude Shannon's work on information theory." There's a great quote at the beginning of the book. It says, "Geniuses are the luckiest of mortals, because what they do is the same as what they most want to do." That is exactly a great description of Shannon. What he did was just what he was most interested in doing. And it goes back to what they were saying in that, some of the ideas he had in how to solve problems, that six-step, you know, framework on how to solve problems, that the genius must delight in finding solutions. You could think about Shannon as, he did a lot of theoretical work, but he was most interested in intellect and intelligence that could be applied.

• 6:33Speaker 00

  In fact, he talks a lot about, you know, he's saying this back in the 1930s, the 1940s, 1950s, that is inevitable, that we will build machines that can think and that will be smarter than us. I think it's already obvious, he made no distinction between work and play. And so later in his life, he was, he gave a series of interviews, and there's a bunch of great quotes from those interviews in the book, and this is one of the ones I found most fascinating. "What's your secret in remaining so carefully?" And Shannon replied, "I do what comes naturally, and usefulness is not my main goal. I keep asking myself, 'How would you do this? Is it possible to make a machine to do this? Can I prove this theorem?'" And the way to think about what he's talking about is like, he didn't look at, "The world

• 7:14Speaker 00

  isn't there to be used, but to be played with and to be manipulated by hand and mind." He loved working with his hands. In fact, he has all, there's all kinds of stories in the book, and some of these I'll talk to you about, where he's just making, he just loved to make little machines, little gadgets, and he went from unicycles to chess-playing robots. He never, this is some of my favorite parts about his personality. I read this book for the first time, I think, like, six years ago. I think originally it was like episode number 95 of Founders. And some of these lines are some I've never forgot. "He never argued his ideas. If people didn't believe in them, he ignored those people. Shannon could neither explain himself to others nor care to. He preferred solitude and kept his professional associations to a minimum.

• 7:59Speaker 00

  He was terribly, terribly secretive. He was not someone who would listen to other people about what to work on. Few of his papers were co-authored." Before we get back into this, I want to tell you about the presenting sponsor of this podcast, Ramp. I have been reading a lot about SpaceX lately. SpaceX is one of the most valuable businesses in the world, and one of the main themes in the history of SpaceX is constantly attacking and questioning your cost. Ramp helps many of the most innovative businesses in the world do exactly that, and they do this by using first principles thinking. The median company running on Ramp cuts their expenses by 5%. And one thing that SpaceX has demonstrated is that a religious dedication to controlling your costs helps increase revenue because you can pursue opportunities you couldn't

• 8:44Speaker 00

  otherwise, and we see that in the Ramp data too. The median company running on Ramp also grows their revenue by 16%. So when you're running your business on Ramp and your competitors are not, you have a massive competitive advantage that compounds over time. Ramp is the only platform designed to make your finance team faster and happier. Many of the top founders and CEOs that I know run their business on Ramp. I run my business on Ramp, and you should too. Go to ramp.com today to learn how they can help your business save time, save money, and grow revenue. That is ramp.com. Shannon made a principle of indifference. It was central to a career in which he chased his instincts, often at the expense of more prestigious options.

• 9:28Speaker 00

  That is exactly, he was chasing his instincts when he wrote the paper on information theory. This is why it's so important. And then, once it, once the paper's published, it starts, gradually over time, it starts to build up this like huge following. He becomes the scientific celebrity. You know, he's offered, you sent carte blanche at Bell Labs. You know, you can come to MIT, you can do, basically do whatever you want. We just want to be associated with you. Very similar to what happened with Einstein later in his life. And in fact, it says that Shannon is to communications as Einstein is to physics. But Shannon didn't give a shit about any of that. So it says, "Unlike many scientists who parlayed successful research careers into lives as public intellectuals, he did not seem to consider using his growing standing as an opportunity to expand his network outside of it.

• 10:13Speaker 00

  If anything, he closed himself off further, ignoring letters, colleagues, and projects, and spending his time and attention absorbed by puzzles that interested him the most." This is what he was just saying. He's like, "Well, usefulness was my goal. I just, I find myself curious about something, and it's like, 'Oh, can I, can I solve this problem? Can I prove this theorem? Can I build a machine that does all this?'" That's, he's just, he was just obsessed with puzzles and following his own curiosity. And then, you might wonder, it's like, okay, well, you know, he could have made a lot more money, he could have been, he was awarded a lot of awards, but he never like chased them. He could have been, you know, out there speaking all the time, increasing his public profile. And so like, why, like, what was it, why is this person so different? Like, why is he acting so differently than others who were in a similar position?

• 10:57Speaker 00

  And the answer that you obviously arrive at after you read the book is like, oh, he's just following his natural curiosity in everything he does. And this is what he says, "I think that history of science has shown that valuable consequences often proliferate from simple curiosity." Goes back to the importance of following your natural drift. Shannon would say that his interest in mathematics, even any of the interest in mathematics from a young age, has a very simple source. It just came easily to him. And he says, "I think one tends to get into work that you find easy for yourself." And then what you realize, you can also tell, like, okay, you can tell a lot about a person by what they choose to work on, but you can also tell a lot about a person by what they choose not to do. And he started studying chemistry.

• 11:39Speaker 00

  He's like, "Oh, this isn't, I don't like this subject because it has too many facts and too few principles." So he says, "He disliked the kind of facts that he couldn't bring under a rule and abstract his way out of." He says, "Chemistry always seemed dull to me. There's too many isolated facts and too few general principles for my taste." Now, as he gets to, it's at the end of high school, he's going to college, trying to figure out what to study. And something that he'll be described by his entire life, from when he was a younger kid to even later on, when, you know, unfortunately, he winds up getting Alzheimer's and passing away, but even later, in later age, that he was just extremely indecisive. He liked working on multiple things at one time. He was a natural born tinkerer.

• 12:23Speaker 00

  He was never just focused on one thing, even when he was writing his papers and even when he was at his most productive, when he was, you know, his early twenties, early to late twenties. But this indecisive nature inadvertently is going to help him later in life because, you know, when he's a teenager, he's like, "What the hell am I going to study?" And he's like, "Well, I like mathematics, but I like engineering too." And he talks about later on that he actually thinks like engineers are some of the most important people in the world, which I'll get to. He gives a great talk about that. But, so, this idea is like, "Well, I'm so indecisive, I can't figure out what I should study." It winds up helping him later on. So he says, "Why? Because a generation later, this two curriculum that he's studying in college, mathematics and engineering, are kind of merging into one."

• 13:04Speaker 00

  And so, this is the reason this dual degree appealed to him and why he was drawn to both mathematics and engineering. And he says, "He admitted that his choice of a dual degree wasn't part of a grand design for his career. It was simply adolescent indecision. I wasn't really quite sure what I liked best. Those studies gave him his first taste of communication engineering, which he found especially to his liking. Why? Because it blended practice and theory, so did he his entire life." That's just his personality, this blend of practice and theory. Shannon's variety of indecision, which he never really entirely outgrew, would prove crucial to his later work. This is why it's important. Someone content to build things might have been happy with a single degree in engineering. Someone drawn more to theory might have been satisfied with studying math alone. Shannon, mathematically and mechanically inclined, could not make up his mind, but

• 13:49Speaker 00

  the result left him trained in two fields that would prove essential to his later successes. And so, after he's done with school, right? Right when he's done with his undergraduate degrees, one of the most important things to ever happen in Claude Shannon's life is he sees a job opening, okay? Typed up on a postcard, posted to an engineering bulletin board, and it's saying, "Come to MIT and run what at the time is going to be the largest analog computer in the world." And I will tell you why it's not only running the analog, the largest analog computer in the world at the time, it's going to be really important to his future innovations and the foundation of information theory, what he derives from that, but more importantly, it puts him in touch with his mentor, which I'll get to in one second. So it says, "It was an invitation to help build a mechanical brain.

• 14:34Speaker 00

  Shannon noticed it in the spring of 1936. The job was to be a master's student and assistant on the differential analyzer at MIT, and it was tailor-made for a young man who could find equal joy in equations and construction, which you and I just talked about. Thinking and building." This was Shannon said about this, "I pushed hard for that job and I got it. That was one of the luckiest things of my life. Luck may have played a role, but the application's acceptance was also a testament to the keen eye of a figure who would shape the rest of Shannon's life and the course of American science." Vannevar Bush. Okay, so Vannevar Bush is one of the most important people in American history.

• 15:17Speaker 00

  If you read a book about anybody doing important science and engineering in the United States in 1930s, 1940s, 1950s, Vannevar Bush will probably pop up as some supporting character. If you want more details about him, I've done two episodes on him, it was episode 270, 271, but this book also gives you insight and a little overview into some of his accomplishments. But I would say, most important thing in terms of the relation to the story you and I are going over right now, is he's the first person to actually see Claude Shannon for what he was, which is a near universal genius. So it says, "Vannevar Bush would have presided over a custom-made brain the size of a room. He'd counsel presidents. He'd direct the nation's scientists during World War II. He was called the man who may win or lose the war and the general of physics. He was the first person to see Claude Shannon for who he was.

• 16:02Speaker 00

  Bush believed Shannon to be an almost universal genius whose talents might be channeled into any direction." And also speaks to Claude Shannon's intelligence that this winds up being his mentor, but he also heeds most of the advice and accepts the direction of Vannevar Bush, especially when he was a young man. Then they talk about, this is what Bush is hiring a young Claude Shannon to do, which is like, "I want you to run this differential analyzer." So I do think, there's some fantastic images online about just, you know, essentially this brain the size of a room, if you want to go look at it, but this is a description of, "One of the world's first large-scale analog computers. The differential analyzer was a brain the size of a room, a metal calculus machine that could whirr away at a problem for days and nights on end before it ground to a halt.

• 16:46Speaker 00

  One problem, which measured the effects of the Earth's magnetic field on cosmic rays, took 30 weeks of spinning gears. But when it was done, the differential analyzer had solved by brute force equations so complex that even trying to attack them with human brain power would have been pointless. Indeed, Bush's lab now owned the..." This is fantastic. "Bush's lab now owned the computational power to turn from the problems of industry to some of the fundamental designs of physics. This was the computer before the digital revolution." And so, essentially, Bush locks Claude Shannon in a room with this machine that is built to automate thought, and it is built in the name of industry and efficiency, to remove

• 17:30Speaker 00

  the art from math. And in the midst of his work, he came to understand that he knew another way of automating thought, one that would ultimately prove far more powerful than the analog machine." So this again goes back to his work on information theory, which we'll get to in a minute. During this, Claude comes up with a very interesting insight. He says, "Logic, just like a machine, was a tool for democratizing force. Built with enough precision and skill, it could multiply the power of the gifted and the average alike." I don't think I've ever heard it described that way, it's very interesting. And then the book does a great job of describing this transition from, which, you know, Claude Shannon obviously played a huge role in bringing about this transition from analog computers to digital ones. Less than a decade after Shannon's paper, the great analog machine was effectively obsolete,

• 18:15Speaker 00

  replaced by digital computers that could do that same work literally a thousand times faster, answering questions in real time, driven by thousands of logic gates. The design of these computers was a direct descendant of Shannon's discovery." And then Vannevar Bush makes a few interesting observations about Shannon here, and I think it was really interesting. He says that Shannon was distinguished less by quantitative horsepower than by his mastery of model making, that his main skill was the reduction of big problems to their essential core. And so again, that's why you could have this universal genius. He did not want, actually, Vannevar Bush, and he's about to talk about this right now, he was anti-specialization. And so, he obviously pushed Shannon in that direction.

• 18:57Speaker 00

  He had him go down different fields of science, and in many cases, he wanted Shannon's insights on a field of science that Shannon previously knew nothing about. One of Vannevar Bush's deepest convictions was that specialization is the death of genius. And he has a great quote about this. He says, "In these days, when there's a tendency to specialize so closely, it is well for us to be reminded that the possibilities of being at once both broad and deep did not pass with Leonardo da Vinci or Benjamin Franklin." And so, let's get into how Bush applies this idea to Shannon. He's like, "Hey, Claude, do you know anything about genetics?" And Claude knows nothing about genetics, so, "Okay, I want you to go study this." And he winds up going down the path. It was actually hilarious.

• 19:40Speaker 00

  I'm going to give you just a summary, because there's a lot more detail in the book, but this is Vannevar Bush's summary of just pointing this universal genius at a field of study he knew nothing about. The project had been Bush's initiative and the hypothesis was his. This is the hypothesis. The subject, this 23-year-old genius working in a scientific field in which he had no training and he didn't even know what the words meant. Can the scientific, can the scientific genius produce original findings in less than one year? Conclusion: confirmed.

• 20:13Speaker 00

  And there's something else that's hilarious. Uh, so not only is Vannevar Bush realizing, you know, the genius of Claude Shannon, was obvious to everybody around him, to the point where while he's in, uh, college, Claude Shannon gets interested in learning how to fly. And his flight instructor is like, oh, wait, no, no, no. Flying is way too dangerous. We cannot risk this guy's mind. And so the flight instructor actually writes a letter to the president of MIT. And this is what he says. I am convinced that Shannon is not only unusual, but is in fact a near genius of most unusual promise. And he's essentially saying, uh, with your permission, I, I will ban Shannon from the cockpit because such a life wasn't worth risking in a crash.

• 20:54Speaker 00

  The president of MIT writes back a few days later, and it's very level headed reply. He goes, somehow I doubt the advisability of urging a young man to refrain from flying or arbitrarily to take the opportunity away from him on the ground of his being intellectually superior. I doubt whether it would be good for the development of his own character and personality. And so it says Shannon was allowed to continue flying. And before we get back into this, I want to tell you about AppLovin. One of my all time favorite quotes is from the book Zero to One. In that book, Peter Thiel writes, he says, the single most powerful pattern I have noticed is that successful people find value in unexpected places. And they do this by thinking about business from first principles instead of formulas. And that is exactly what AppLovin has done with their advertising platform.

• 21:35Speaker 00

  AppLovin connects you with over a billion potential new customers in mobile games. AppLovin allows you to capture undivided attention. AppLovin ads are full screen videos that are watched for an average of 35 seconds. That is retention that blows other ad platforms out of the water. And you can launch on AppLovin in minutes. You set the goal and AppLovin achieves it. No complex setup, no expertise needed and AppLovin scales quickly. They can put your ads in front of over a billion potential customers. Other businesses have seen immediate results scaled to hundreds of thousands of dollars of spend per day and increased their revenue by millions. So you want to get started quickly before all of your competitors are on AppLovin. And you can do that by going to applovin.com. That's applovin.com.

• 22:20Speaker 00

  And then I want to tell you about Vanta. Vanta, Vanta, Vanta. Vanta helps your company prove you're secure so more customers will use your product or service. Vanta is an AI powered security expert that scales with you. The more your business grows, the more complex your security needs get. And that complexity turns into chaos. Vanta tames that chaos for you. Vanta automates compliance, continuously monitors your controls and gives you a single source of truth for compliance and risk. So whether you're a fast growing startup or an enterprise company, Vanta fits easily into your existing workflows. Many companies won't sign contracts unless you're certified and this is causing you to lose out on sales. That is why the average Vanta customer reports a 526% return on investment after becoming a Vanta customer.

• 23:03Speaker 00

  Automate your compliance, security and trust with Vanta. Vanta will help you win trust, close deals and stay secure faster and with less effort. Go to vanta.com/founders and you'll get a thousand dollars off. That is vanta.com/founders. So after he go, after he finished graduate school, he's trying to figure out what to work on. He decides to go work for Bell Labs. There is a great book on this that I'll eventually do an episode on. It's called Idea Factory. Uh, but this, this book has a great overview of just how important it was that Shannon chose to go down this path and how it's going to lead to his work on information theory. Says he was headed to what was perhaps the world's foremost technology company. Remember at this time, phone companies, it's the most complicated communication network in the world and they're complete and utter monopolies.

• 23:46Speaker 00

  And so they just shoot off cash and then they took a bunch of that cash. They actually, you know, a few weeks ago, when you and I were talking about the founder of Honda, he had that original insight. He actually thought that research and development departments were so important that they had to be spun out completely and had a different set of incentives. They can't just be inside of your company. Bell Labs was spun out of the phone company too. So there's a lot of thinking and similarities and thinking between spinning out R and D, between what happened is happening here in the story and what the founder of Honda was doing in Japan as well. So, uh, Shannon is headed to the world's most foremost technology company and it's the home of the best communications minds in America. The goal of Bell Labs wasn't simply clear and faster phone calls. The labs were tasked with dreaming up a future in which every form of communication would be machine aided.

• 24:30Speaker 00

  Right place, right time, right set of skills, right person. This is exactly the perfect place for Claude Shannon. So the amount of innovation that comes out of Bell Labs is insane. Uh, let me just read this paragraph to you. In the span of a few decades, Bell researchers engineered the first ever long distance phone call. They synchronized the sounds and images in movies and demonstrated some of the earliest facts in television systems. During World War II, they improved radar, sonar and the bazooka. And they created a secure phone line connecting Franklin Roosevelt and Winston Churchill. They would invent touch tone dialing and the solar battery cell. And they would also pioneer the communication satellite. And in 1947, Bell researchers also created the transistor, the foundation of modern electronics. And so this is why Shannon said he wanted to work at Bell Labs.

• 25:14Speaker 00

  I had the freedom to do anything I wanted from almost the day I started. They never told me what to work on. He also had no responsibilities other than to tinker and to think and to publish papers and to do science. Why is this important? Because Shannon was allergic to administrative work and bureaucracies of almost every kind. He has a very deep, he's like a level 10 introvert, which we'll get to in a moment. But he's, he's naturally a loner. He long solitudes, Spartan apartments, empty offices. This is what he liked, where he does his best work. His freedom from obligation played into his lifelong tendency to isolate himself. Most days were spent shut indoors, alternating between his notepad and the clarinet and back again.

• 25:58Speaker 00

  So he, I already mentioned it a few times and he's going to mention it a few more times that he loved working, uh, back and forth on multiple things at one time. Again, there's no separation between, you know, work and play. So he might be thinking about, you know, how can we, uh, he did a lot of work in cryptography or how can we, you know, improve the telephone network? And he'd think about that for a little bit and then he'd put his pad down and he would listen to jazz or he'd play his clarinet or he'd juggle or he'd ride his unicycle. And then he'd go back and forth, back and forth. Like this is essentially, you know, his entire day and how he lives his life. Um, then it goes back into, oh, it says it right here. He was actually writing this letter to Vannevar Bush and he talked about, you know, essentially what we'd call this today is like his productivity hack. Uh, he says, I've been working on three different ideas simultaneously and strangely enough,

• 26:43Speaker 00

  it seems a more productive method than sticking to one problem. Now, this is, uh, early 1940s. He's, you know, in his twenties and so he's going to be drafted into the war. It terrified him. Now, you know, you could think, okay, might terrified him because he was worried about dying. Obviously he didn't want to do that. But what was interesting is a description of his introvert nature. It's like, wait a minute, I'm going to have to live in barracks. I'm going to be in the army. I'm going to have all these strangers around me. Freaked them out. So he wanted to find a way to, to, to use his mind to serve the country, not his body. So it says Shannon worried not only about the dangers of an overseas deployment, but also the close quarters of army life. I think he did the work with the fear that he might have to go into the army, which means being with lots of people around, which he couldn't stand.

• 27:28Speaker 00

  He could not stand being in groups. He was phobic about crowds and people he didn't know. He would put his mind rather than his body to work on the country's behalf. So this, not only Shannon, but almost everybody at Bell Labs is now transitioned into doing work for the war. And one of the things is interesting is like nothing really go, when you study Shannon, nothing that he works on ever goes to waste. He'll find other ways to essentially make an abstraction or analogy and use it in his later work. And so this description of what he's actually working on, uh, it's called fire control, which I'll read to you in one second, actually plays a role into him, uh, cause he's writing the, the paper on information theory. I think it takes him like eight years on and off. So he's doing a bunch of this, these things at the same time. And then he'll draw these analogies. So, uh, one analogy he drew was from fire control.

• 28:12Speaker 00

  So this is what fire control is. Fire control was essentially the study of hitting moving targets. The targets were anything and everything the enemy could hurl through the air to cause damage, planes, rockets, ballistics. So imagine a gun firing a single shot at a target. Okay. Now imagine that gun is the size of a two story house and it's placed on a moving Navy ship in the middle of the ocean and that it's trying to shoot down an enemy fighter moving at 350 miles per hour. This is the math he's got to figure out. That's a rough description of the challenge of fire control. Goes back to this idea, these always finding analogies. There are surprisingly close invalid analogies between fire control prediction problem and certain basic problems in communications engineering.

• 28:53Speaker 00

  At the most basic level, the speed and quality of information was vital to both phone systems and fire control systems. A phone call reaching its intended recipient was a struggle against noise. An anti aircraft missile hitting its target presented the same conceptual challenge. This is what he says. He's like, he's just a man of abstractions. He's like, oh, I'm actually working on the same problem. You might think, and many people, again, that I think that's part of his genius. I wouldn't even see the similarities between these two problems. And to him, everything was the same. He says, both required high level statistical inference. Both presented the challenge of building machines to accurately translate math into action. Then he goes into some of the other things that his colleagues at Bell Labs were working on.

• 29:36Speaker 00

  Some of the questions the scientists at Bell Labs had to tackle for the war effort. They're working six days a week on this. And so, again, the army and the government's giving them all these problems here, find solutions to these questions. How many tons of explosive force must a bomb release to create a certain amount of damage? In what sorts of formations should bombers fly? Should an airplane be heavily armored or should it be stripped of defenses so it can fly faster? At what depths should an anti-submarine weapon drop from an airplane explode? How many anti-aircraft guns should be placed around a critical target? In short, what the government is trying to figure out is precisely how should these new weapons be used to produce the greatest military payoff? Now, he's doing this because, obviously, they're compelled to. He doesn't want to physically serve in the war, but he hated it.

• 30:20Speaker 00

  So, this is his reaction to all this war work. The whole atmosphere left a bitter taste. The secrecy, the intensity, the drudgery, the obligatory teamwork. It goes back to, I don't know if you caught on that, there's like this one sentence I said earlier, that very few of his papers were co-authored. He does not like working on teams. He likes working by himself, alone. And so, he's like, "I'm being compelled to work on a team. I hate this." And it winds up getting to him. He has, you know, he's also going through a divorce at the time. So, he kind of has like this, you know, temporary breakdown. Now, another thing that they have him working on during the war is cryptography. They're trying to figure out, how can we down, not only do we want to be able to send

• 31:01Speaker 00

  secure communications, you know, across long distances, but we also want to hack into, you know, the Germans and the Japanese doing the same thing. We want to find out, we want to find their secrets and make sure they don't find ours. And so, Shannon is tasked with checking the algorithms that would allow the messages to be securely reproduced on the receiving end. The work gave him a window into the world of encoded speech, transmission of information and cryptography, a synthesis that at that moment in history, may not have taken place anywhere other than at Bell Labs, as Shannon observed. Not a lot of laboratories had voice encoding devices for scrambling speech. And so, this is when Claude Shannon meets Alan Turing, who's also working on very similar things, and they wind up getting along so well. They met daily in the Bell Labs cafeteria and they'd have tea.

• 31:46Speaker 00

  And later in life, Shannon's being interviewed and he was being asked all these questions. He's like, "Well, with your own passion for, you know, cryptographic puzzles, why didn't you probe Turing, you know, deeper and further?" And Shannon's response was simple and to the point. "Well, in wartime, you didn't ask too many questions." But he did talk about some of the stuff that Alan Turing and him would talk about. Remember, this is happening in 1940, so it's pretty wild, because now we are literally living in the future that these two guys envisioned, helped envision too. And we're essentially, you know, having these daily, you know, conversations about. And he says, "We would talk about the notion of building computers that will think." And so, Shannon was obsessed with artificial intelligence. He says, "We had dreams. Turing and I used to talk about the possibility of simulating the human brain.

• 32:31Speaker 00

  Could we really get a computer which would be the equivalent of the human brain or even a lot better? We both thought that this would be possible in not very long time, in 10 or 15 years. Such was not the case." And so, in this book, there's a lot of quotes from Shannon about his deep desire to invent and to have artificial intelligence, to have AI. In fact, he called it his fondest dream. He says, "My fondest dream is to someday build a machine that thinks, learns, communicates, and manipulates its environment in a fairly sophisticated way." Goes on over and over again. There's multiple quotes throughout the book like this. "I believe we are going to invent machines which are smarter than we are." We talk about the fact that when he would discuss this, you know, it had to seem like crazy talk, especially back then.

• 33:14Speaker 00

  He says that he was adamant and he thought the people that thought he was wrong were actually the ones that were incorrect. "The thought that a machine could never exceed its creator was just foolish logic." He called it "wrong and incorrect logic." Now, Shannon winds up meeting, you know, all the other great minds of his time. He met John von Neumann. He said, Shannon called John von Neumann "the smartest person" that he ever met. Later in life, when he was asked a lot of other questions, like, you know, they were, all these scientists were advising the CIA, the Defense Department, the NSA, and Shannon was very reluctant to talk about that, even decades later. But this surprising story will give you an idea of the type of classified work that Shannon

• 33:56Speaker 00

  was involved in, because John von Neumann was involved in as well. "One of Shannon's fellow NSA scientific advisers, John von Neumann, was watched around the clock by uniform military personnel when he was on his deathbed. Impressive though von Neumann's mind may have been, it wasn't immune from infiltration, or so the government feared. And what better time to infiltrate it and grab the precious state secrets it held than when it was in a medically induced haze?" So, it's so important that it was guarded around the clock on his deathbed, because they did not want anybody hacking his brain. So, there is, I don't know, I bet you 50 to 100 pages of this book about information theory. I think for our purposes, I'm just going to give you a really simple overview, because

• 34:40Speaker 00

  I think more important than that is the way he approaches his work. I think that's what you and I are most interested in. But I do think, you know, there is a couple paragraphs that give you, I think, a simplified version of, a simplified understanding, rather, of information theory. "So, information existed before Shannon, just as objects had inertia before Newton. But before Shannon, there was precious little sense of information as an idea, as a measurable quantity, an object fitted out for hard science. Before Shannon, information was a telegram, a photograph, a paragraph, a song. After Shannon, information was entirely abstracted into bits. Information theory was summed up by his recognition that all information, no matter the source, the sender, the recipient, or the meaning, could be efficiently represented by a sequence

• 35:24Speaker 00

  of bits. He felt that bits was information's fundamental unit." So, he writes this paper, like I said, I think it's published in 1948. So, his paper is going to introduce many of the concepts that underpin the modern digital world, the world that we inhabit, the fact that most of our, probably most of our communications is digital today. "And Shannon's work is widely regarded as one of the most influential scientific papers of the 20th century, and laid theoretical foundation for things like the internet, data compression, error correcting codes, digital telecommunications, modern computing, and much of today's AI infrastructure." Now, there's a couple interesting descriptions in the book of him writing the paper. Again, it just plays out over many, many years. "Napkins decorate the table.

• 36:06Speaker 00

  Strands of thought and stray sections of equations accumulate around him. He writes a neat script on lined paper, but there are raw materials everywhere. Eight years like this, scribbling, refining, crossing out, staring into speculative equations, knowing that at the end of all this effort, they may reveal nothing. There are breaks for music and cigarettes, and bleary-eyed walks to work in the morning. But mostly, it's this ceaseless drilling, again, alone, at night, by himself, back to the desk, where he senses perhaps that he's on to something significant, something even more fundamental. But what?" And then he talks about there was many flashes of intuition, that his work was not linear. Ideas came when they came and he had no control of it. One night, I remember, I woke up in the middle of the night and I had an idea and stayed up all night working on that.

• 36:50Speaker 00

  And one of the most remarkable things about Shannon is, he publishes the paper, he knows it's good. He never doubts that his work is good, and even to the point where, you know, he kind of was like, he trusted, he trusted his judgment so much that when he said, it's like, you know, if you want to argue ideas, like, I'm not even going to argue with you. But, once it comes out, then we see, again, he goes back to his personality, he's just like, he wants to do great work, but he's doing great work because he's just curious about it. He's not doing it. If you go back to what Munger says about the importance of having an inner clock, Buffett calls it an inner scorecard, just like, "Man, I'm just doing what I want to do, regardless of what's going on outside, in the outside world." And we see this because, says, "Having completed his path-breaking work by the age of 32, he might have spent his remaining decades as a scientific celebrity. Instead, he spent all his time tinkering." So, he'd build, he's a funny dude, too.

• 37:35Speaker 00

  He built an electronic maze-solving mouse. He had a chess, these are all things he built with his hands, by the way. He built a chess-playing computer. He built the first ever wearable computer. He created a calculator that operated in Roman numerals. He had a fleet of customized unicycles. He spent years devoted to the scientific study of juggling. And his whole point is that working on what naturally interests you is time well spent. Shannon would be adamant on this point. "After the effort of discovery, the effort of communication was secondary by far. He had solved a problem to his own satisfaction, and that, as far as he was concerned, was enough." Shannon explains this viewpoint later on.

• 38:17Speaker 00

  "After I had found the answers, it was always painful to write them up or to publish them." Again, this is why it's so important. The writing up and the publishing is actually how you get the acclaim. He was just interested in solving the puzzle. So, winds up, and now, he's, you know, a scientific celebrity, Bell Labs does not want to lose him. Essentially, just do whatever you want, Shannon. You're on the payroll, you can do whatever, you know, you can work from home, you can come in, you can do neither or both. He still decides to wind up leaving, because he's getting poached by MIT. And he talks about one of the reasons, there's really two, two main reasons that he winds up doing this. And what's funny is, even after he leaves Bell Labs, they thought he was so important, and he was so disinterested in money, for money's sake, that Bell Labs kept him on the payroll.

• 39:02Speaker 00

  So, he winds up getting paid from MIT and Bell Labs. And so, now, at this point in the book, he's talking about why he leaves Bell Labs and decides to go to MIT. "The general freedom in academic life is one of its most important features. There was a certain restlessness on Shannon's part after spending more than a decade and a half in a single institution. Having spent 15 years at Bell Labs, I felt myself getting a little stale and unproductive, and a change of scene and colleagues is very stimulating." And so, then I love this paragraph, which describes the result of that very important decision, saying, "Hey, I had 15 years here, I loved it here, but I need a change of scenery, I need a change of place, I need a change of colleagues." And so, it says, "Once he got to MIT, what resulted were some of Shannon's most creative and whimsical endeavors.

• 39:44Speaker 00

  There was a trumpet that shot fire when played, handmade unicycles. There was a chairlift that took guests down from the porch to the edge of the lake. A machine that solved Rubik's cubes, more chess-playing machines, more handmade robots. Shannon's mind, it seems, was finally free to bring its most outlandish ideas to mechanical life." He loved building machines. What I would describe, the activities that he's doing now, and, you know, especially now at this point in the story, before, earlier in life as well, his activities were autotelic. So, autotelic is one of my favorite words. In fact, I've told you this before, but in case you remember, the original name of this podcast that I started 10 years ago next month.

• 40:23Speaker 00

  Was autotelic. Why was that such an important name? Autotelic is an activity done for the sake of itself. I was saying with the title of the podcast that I never thought I was going to listen to, by the way, I'm going to podcast even if no one listens to it. I just feel compelled to do it. I love it. I'm obsessed with it. You see the exact same idea here. All of his ideas, all of his activities, they're just autotelic. Their activities done for the sake of himself. Shannon summed all of his work up, the work that he's doing now, as happily pointless. I've always pursued my interests without much regard to financial value or value to the world. I've spent lots of time on totally useless things. He made no distinction between his interest in information and his interest in unicycles. They were all moves in the same game. This is just, and this is just great writing here.

• 41:09Speaker 00

  What other people called hobbies, he thought of as experiments. Exercises in the practice of simplification. Models that filed a problem down to its barest interesting form. He was so convinced of a machine-enabled future and so eager to explore its boundaries that he was willing to tolerate a degree of ridicule to bring it to pass. He was preoccupied with, quote, "the possible capabilities and applications of large-scale electronic computers" at a time when nothing like even existed. Considered in the light of that future, which is our present, his machines were not hobbies. They were proofs. That is great, great writing. Here's another great quote from Shannon.

• 41:54Speaker 00

  "A very small percentage of the population produces the greatest proportion of the most important ideas." There are some people, if you shoot one idea into their brain, you will get half an idea out. There are other people who produce two ideas for each idea sent in. Now, another thing, we go back to puzzles. He didn't really give a shit about money, but he was very rich. How did he get very rich? Because he just started getting obsessed with stocks and the stock market. He's like, oh, this is just another puzzle. And they wind up making, uh, there's, there's this documentary and no disrespect to documentary makers, but everybody's like, you gotta watch this documentary. It's called The Bit Player, and I think I've watched it before. We watched it again to prep for this episode. Not good.

• 42:39Speaker 00

  But if you want to maybe watch it on one five, I don't know, or maybe two X. Maybe it gives you an insight. You're probably just better off just listening to the audiobook of this book or maybe listening to this podcast again, again. But one of the interesting things about documentary was they interview his kids. You know, Shannon has now passed on, I think he died in 2001. But they talk about the fact that they made investing a family hobby. And I thought that was interesting. And so his wife Betty says Betty and Claude did play the markets obsessively. The process became a family affair. And so this is what his daughter Peggy would talk about this. "Much of the conversation around the home would be about the stock market because much of my parents' focus was on what the market was doing. They taught me to read the Wall Street Journal and stocks, and they taught me about stocks very early. You'd come down and open the newspaper and they'd have me read because at that time their eyesight, my eyesight was better than theirs.

• 43:25Speaker 00

  And it was a way for them to engage their kids. Then eventually they set up a small personal computer to carry out the quotes during the day and then check again at the end of day. So there were computer printouts floating all around the house with stock quotes on them." Now, there's a couple crazy stories. And again, this is one of my favorite parts of reading his biography is like, you know, the higher you go, the world gets smaller and smaller. And so, you know, he's work, he's talking to John Van Neumann, he's meeting Albert Einstein, he's having tea with Alan Turing. In, in a twist of fate and just remarkable is the fact that Henry Singleton, who again, if you just go back and listen to Buffett and Munger, Buffett and Munger talk about Henry Singleton. Henry Singleton built a conglomerate before Buffett and Munger did. And they took a lot of ideas from Singleton. The ideas that I thought were Buffett and Munger's, they actually learned from Singleton.

• 44:08Speaker 00

  Munger said that, that Singleton was the single smartest person that he ever met in life. Buffett said that it was a crime that business schools did not study this guy. Claude Shannon and Henry Singleton wind up being, meeting in college. He was a friend in college. Uh, the, the conglomerate that Singleton builds is called Teledyne. Claude Shannon is going to be on the board of it. Okay. And he, Shannon does something really smart and he makes a large investment into Teledyne. And that investment winds up compounding, gave him a compounded return of 20%, 20, sorry, 27% over 25 years. And as Shannon retold the story, "He made the investment simply because I had a good opinion of him."

• 44:50Speaker 00

  And there's all these stories in the book where, you know, Singleton benefits from when he's doing a series of acquisitions, you know, being able to bounce those ideas off of Claude Shannon. So I want to go, there's another book I read a long time ago called Fortune's Formula that I think actually has a better description of just how remarkable Shannon was at investing and you're turning the, of trying to solve the puzzle of the stock market. I'm going to read a quote, a paragraph from there, from that book in one second to you, but I just want to give you an overview of why he's doing this. Again, it just was just another puzzle. Shannon's interest in money resembled his other passions. He was not out to accrue wealth for wealth's sake, nor did he have any burdening desire to own the finer things in life, but money created markets and math puzzles,

• 45:31Speaker 00

  problems that could be analyzed and interpreted and played out. So then let me read just how good he was at this. This is a paragraph from Fortune's Formula. "In 1986, Barron's ran an article ranking the recent performance of 77 money managers. Claude Shannon, though not mentioned in the article, had done better than all but three of the pros. The Barron's money managers were mostly firms with up to 100 people. Shannon worked with his wife and an Apple II computer. Barron's reported on the recent performance of 1,026 funds. Shannon achieved a higher return than 1,025 of them." Over, this is incredible, over 30 years.

• 46:13Speaker 00

  "From the late 1950s through 1986, Shannon's returns on his stock portfolio was 28% a year." Another legendary person that Shannon winds up meeting, this is a young Ed Thorp. Ed Thorp and Claude Shannon wind up building the world's first wearable computer. They're trying to solve, Shannon was obsessed with trying to solve gambling. Uh, Thorpe wrote the book on how to count cards at Black Jack's. Thorpe's still alive. It's episode 222, uh, one of the best audiobiofs I've ever read. It's called A Man for All Markets, if you want to learn more about Thorpe. But Thorpe, uh, he's all in this book, but I thought this, this one description of Thorpe on how Shannon dealt with problems was very fascinating. And he says, "Shannon seemed to think with ideas more than with words or formulas.

• 46:58Speaker 00

  A new problem was like a sculptor's block of stone and Shannon's ideas chiseled away the obstacles until an approximate solution emerged like an image, which he proceeded to refine as desired with more ideas." It reminded me of this great quote from Michelangelo when he was saying that how he carved the statue of David. He says, "I simply removed everything that was not David." Back to following his curiosity. He says, "I don't think I was ever motivated by the notions of winning prizes, although I have a couple dozen of them in the other room. I was more motivated by curiosity, never by the desire for financial gain. I just wondered how things were put together or what laws or rules govern a situation or if there are theorems about one, what one can or cannot do,

• 47:43Speaker 00

  mainly because I just wanted to know myself." And he goes back to that, one of my favorite quotes of him. He says, "I think the history of science has shown that valuable consequences often proliferate from simple curiosity." Later in his life, you know, he, he would pick, you know, he'd be invited all over the world to travel and to speak. Many times they wanted to give him an award. You know, he didn't really, he just liked being at home. He liked eating the same thing all the time. He liked just being in his, his, uh, you know, room, building gadgets. But he did go and, and, uh, accept this prize in Japan, which I thought was very interesting. And he talked about the fact that he actually thinks that, at least in the United States, they were actually teaching history incorrect and it's just the wrong way to go about it. And he's got some really interesting insights about how he thinks history should be taught,

• 48:27Speaker 00

  but also tells you a lot about what he admired and what he thought were valuable. And he says, "Most of the time was spent on the study of political leaders and wars, the Caesars, the Napoleons and the Hitlers. I think this is totally wrong. The important people and events of history are the thinkers and innovators, the Darwins, the Newtons, the Beethovens, whose work continues to grow influence in a positive fashion." Talks about we need in the same, in the same talk. He says, "We need to encourage, we need more engineers. We need to encourage people to go into engineering. Thanks for some of the most important people that ever exist." "One category of innovation he singled out for special mention. The discoveries of science are wonderful achievements in themselves, but would not affect the life of the common man without the intermediate efforts of engineers and inventors, people like Thomas Edison and Alexander Graham Bell."

• 49:11Speaker 00

  And then in a cool twist of fate, somebody who was born with one of the most gifted minds that has ever graced the planet. Unfortunately, the last like decade of his life slowly starts to degrade, uh, winds up having Alzheimer's, winds up having to put into, you know, full-time care. But before he died, he outlined what he wanted his funeral to be. And I think, again, this gives you a great insight and says it's just a very unique personality. It's a person that was obsessed with play and following his own curiosity. And so this is Claude Shannon's idea for his own funeral. "Shannon had set to his mind the question of his funeral and imagined something very different. For him, it was an occasion that called for humor, not grief.

• 49:52Speaker 00

  He outlined a grand procession, a Macy's-style parade to amuse and delight and to sum up the life of Claude Shannon. The parade would be led by somebody playing a clarinet. Behind them would be a jazz combo. Next in line would be six unicycling pallbearers somehow balancing Shannon's coffin. Behind them would come the grieving widow, then a juggling octet, and then a juggling machine. Next would come three black chess pieces bearing $100 bills and then three rich men from the West, California tech investors, following the money. They would march in front of a chess float. Atop that float would be British chess master David Levy, who would be squaring off an alive chess match against a computer.

• 50:37Speaker 00

  Then the scientists and mathematicians, a fae links of joggers, and a 417 instrument band would bring up the rear." And that is how Claude Shannon wanted to be remembered. That is where I will leave it for the full story. Highly recommend reading the book. That is 428 books down, 1,000 to go, and I'll talk to you again soon.